Skip to content

Commit

Permalink
fix(build): Updating git submodules to avoid recursive cloning of int…
Browse files Browse the repository at this point in the history
…erdependent submodules; documentation to support this change
  • Loading branch information
monotasker committed Jan 8, 2025
1 parent de54ca3 commit 8123db2
Show file tree
Hide file tree
Showing 3 changed files with 120 additions and 44 deletions.
24 changes: 12 additions & 12 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,6 @@
path = site/kcworks/dependencies/invenio-vocabularies
url = https://github.com/MESH-Research/invenio-vocabularies.git
branch = local-working
[submodule "site/kcworks/dependencies/invenio-group-collections-kcworks"]
path = site/kcworks/dependencies/invenio-group-collections-kcworks
url = https://github.com/MESH-Research/invenio-group-collections-kcworks.git
branch = main
[submodule "site/kcworks/dependencies/invenio-remote-user-data-kcworks"]
path = site/kcworks/dependencies/invenio-remote-user-data-kcworks
url = https://github.com/MESH-Research/invenio-remote-user-data-kcworks.git
branch = main
[submodule "site/kcworks/dependencies/invenio-record-importer-kcworks"]
path = site/kcworks/dependencies/invenio-record-importer-kcworks
url = https://github.com/MESH-Research/invenio-record-importer-kcworks.git
branch = main
[submodule "site/kcworks/dependencies/invenio-remote-api-provisioner"]
path = site/kcworks/dependencies/invenio-remote-api-provisioner
url = https://github.com/MESH-Research/invenio-remote-api-provisioner.git
Expand All @@ -38,3 +26,15 @@
path = site/kcworks/dependencies/invenio-modular-detail-page
url = https://github.com/MESH-Research/invenio-modular-detail-page.git
branch = main
[submodule "site/kcworks/dependencies/invenio-remote-user-data-kcworks"]
path = site/kcworks/dependencies/invenio-remote-user-data-kcworks
url = https://github.com/MESH-Research/invenio-remote-user-data-kcworks.git
branch = main
[submodule "site/kcworks/dependencies/invenio-group-collections-kcworks"]
path = site/kcworks/dependencies/invenio-group-collections-kcworks
url = https://github.com/MESH-Research/invenio-group-collections-kcworks.git
branch = main
[submodule "site/kcworks/dependencies/invenio-record-importer-kcworks"]
path = site/kcworks/dependencies/invenio-record-importer-kcworks
url = https://github.com/MESH-Research/invenio-record-importer-kcworks.git
branch = main
54 changes: 54 additions & 0 deletions docs/source/developing.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,59 @@
# Developing KCWorks

## Version Numbering

KCWorks uses semantic versioning (https://semver.org/). When a new release is made, the version number should be incremented in the following files:

- `README.md`
- `site/pyproject.toml`
- `site/kcworks/__init__.py`

While in beta, the version number should be followed by a numbered `-beta` suffix: e.g., `0.3.3-beta6`. This suffix should be updated continuously (without starting over again for minor releases) until version 1.0.0 is reached and KCWorks leaves beta.

Bug fixes and other changes that do not introduce new features (including changes to documentation, build processes, etc.) should be considered `patch` releases.

New features should be considered `minor` releases.

## Version Control

### Git Branching Strategy

KCWorks employs a modified version of the Gitlab Flow branching strategy for version control. The repository has four persistent branches:

- `main` is the default branch and is the reference point for active development. It will usually not be ready for production deployment.
- `staging` is the branch that is deployed to the staging server. It is created from the `main` branch when changes are ready to be tested. No commits should be made directly to the `staging` branch except to merge changes from `main`.
- `production` is the branch that is deployed to the development server. It is created from the `staging` branch when changes are ready to be deployed to the production server. No commits should be made directly to the `production` branch except to merge changes from `staging`.
- `gh_pages` is the branch that is used to generate the static documentation site for KCWorks on Github Pages. This branch is automatically updated from the `main` branch.

New changes should be made to the `main` branch or to a temporary `feature` or `bugfix` branch created from the `main` branch. These temporary branches should be merged back into `main` as often as possible, and the temporary branches deleted.

No commits should be made directly to the `staging` or `production` branches. All changes should be made to the `main` branch and then merged into `staging` and `production` via pull requests. This is especially important because changes pushed to `staging` and `production` branches will be automatically uploaded to the respective servers.

### Commit strategy

Wherever possible, a commit should represent a single completed change. We try to avoid `wip` commits in order to keep the commit history readable.

In practice, though, we will often need to make incremental commits, particularly on feature and bugfix branches. The practice should be to squash all of the related commits into a single commit before merging the changes into `main`.

### Naming Commits

### Tagging Releases

Whenever the KCWorks version number changes, it that commit should be tagged with the new version number. This can be done by running the following command:

```shell
git tag -a <version-number> -m "Release <version-number>"
```
We do not create branches for each numbered release.

### Git Submodules

KCWorks uses git submodules to manage dependencies. The submodules are located in the `site/kcworks/dependencies` folder. The submodules are cloned from the upstream repositories when the KCWorks instance is first created. They are updated from the upstream repositories when the KCWorks instance is updated.

Note that in some cases there are inter-dependencies between these submodules. For example, the `invenio-record-importer-kcworks` submodule has its own dependency on the `invenio-group-collections-kcworks` submodule. When cloning the KCWorks repository, you **should not use the `--recurse-submodules` option** because this will clone redundant copies of these inter-dependent submodules. Instead, you should clone the KCWorks repository and then initialize the submodules in a separate step with `git submodule update --init`. Likewise, when updating the KCWorks submodules, you should use the `git submodule update --remote` command **without the `--recursive` option**.



## Updating the running KCWorks instance with development changes

### Changes to html template files
Expand Down
86 changes: 54 additions & 32 deletions docs/source/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,36 +8,58 @@ First you will need to have the correct versions of Docker (20.10.10+ with Docke

From there, installation involves these steps. Each one is further explained below, but here is a quick reference:

1. Clone the git repository
1. From your command line, navigate to the parent folder where you want the cloned repository code to live
2. Clone the knowledge-commons-works repository with `git clone --recurse-submodules [email protected]:MESH-Research/knowledge-commons-works.git`
2. Create your configuration files
- `cd knowledge-commons-works`
- Create and configure the `.env` file in this folder as described {ref}`here <in_depth:add and configure an environment file>`
- Create the `.invenio.private` file with the following contents:
```shell
[cli]
services_setup = True
instance_path = /opt/invenio/var/instance
```
3. Start the docker-compose project
- `docker-compose --file docker-compose.yml up -d`
4. Initialize the database and other services, and build asset files
- enter the `web-ui` container by running `docker exec -it kcworks-ui bash`
- *note*: The container name may be different depending on your local docker setup. You can find the correct name by running `docker ps`
- run the script to set up the instance services and build static assets `bash ./scripts/setup-services.sh`
- *note*: Some of the commands in this script may take a while to run. Patience is required! The `invenio rdm-records fixtures` command in particular may take up to an hour to complete during which time it provides no feedback. Don't despair! It is working.
5. Create your own admin user
- enter the `web-ui` container by running `docker exec -it kcworks-ui bash`
- run the commands:
- `invenio users create <email> --password <password>`
- `invenio users activate <email>`
- `invenio access allow administration-access user <email>`
6. View the application
- The Knowledge Commons Works app is now running at `https://localhost`
- The REST API is running at `https://localhost/api`
- pgAdmin is running at `https://localhost/pgadmin`
- OpenSearch Dashboards is running at `https://localhost:5601`
### 1. Clone the git repository

- From your command line, navigate to the parent folder where you want the cloned repository code to live
- Clone the knowledge-commons-works repository with `git clone [email protected]:MESH-Research/knowledge-commons-works.git && git submodule update --init`

> **Note**: Do not use the `--recurse-submodules` option when cloning the repository or the `--recursive` option when initializing the submodules. This will clone redundant copies of the inter-dependent submodules.
### 2. Create your configuration files

- `cd knowledge-commons-works`
- Create and configure the `.env` file in this folder as described {ref}`here <in_depth:add and configure an environment file>`
- Create the `.invenio.private` file with the following contents:

```shell
[cli]
services_setup = True
instance_path = /opt/invenio/var/instance
```
### 3. Start the docker-compose project

- `docker-compose --file docker-compose.yml up -d`

### 4. Initialize the database and other services, and build asset files

- enter the `web-ui` container by running `docker exec -it kcworks-ui bash`

> **Note**: The container name may be different depending on your local docker setup. You can find the correct name by running `docker ps`
- run the script to set up the instance services and build static assets `bash ./scripts/setup-services.sh`

> **Note**: Some of the commands in this script may take a while to run. Patience is required! The `invenio rdm-records fixtures` command in particular may take up to an hour to complete during which time it provides no feedback. Don't despair! It is working.
### 5. Create your own admin user

- enter the `web-ui` container by running `docker exec -it kcworks-ui bash`

> **Note**: The container name may be different depending on your local docker setup. You can find the correct name by running `docker ps`
- run the commands:

```shell
invenio users create <email> --password <password>
invenio users activate <email>
invenio access allow administration-access user <email>
```

### 6. View the application

- The Knowledge Commons Works app is now running at `https://localhost`
- The REST API is running at `https://localhost/api`
- pgAdmin is running at `https://localhost/pgadmin`
- OpenSearch Dashboards is running at `https://localhost:5601`

This setup will allow you to make changes to the core Knowledge Commons Works codebase and see those changes reflected in the running application.

Expand All @@ -50,12 +72,12 @@ To do this, you will need to do the following:

1. Ensure the required git submodules are cloned by running the following commands in the `knowledge-commons-works` folder:
```shell
git submodule update --init --recursive
git submodule update --init
```
This will clone the following repositories:
```shell
main [email protected]:MESH-Research/invenio-record-importer-kcworks.git
main [email protected]:MESH-Research/invenio-group-collections.git
main [email protected]:MESH-Research/invenio-group-collections-kcworks.git
main [email protected]:MESH-Research/invenio-modular-deposit-form.git
main [email protected]:MESH-Research/invenio-modular-detail-page.git
main [email protected]:MESH-Research/invenio-remote-api-provisioner.git
Expand Down

0 comments on commit 8123db2

Please sign in to comment.