diff --git a/README.md b/README.md index 223b601f2..b818dd5dc 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,16 @@ Loculus is a software package to power microbial genomial databases. ### [Visit the Loculus documentation website](https://loculus-project.github.io/loculus/) +## Development + Additional documentation for development is available in each folder's README. This file contains a high-level overview of the project and shared development information that is best kept in one place. +If you would like to develop loculus locally you need to first: + +1. Deploy a local kubernetes instance: [kubernetes](/kubernetes/README.md) +2. Deploy the backend: [backend](/backend/README.md) +3. Deploy the frontend/website: [website](/website/README.md) + ## Architecture - Backend code is in `backend`, see [`backend/README.md`](/backend/README.md) @@ -27,7 +35,6 @@ While the documentation is still a work in progress, a look at the [`.github/wor - [`website.yml`](/.github/workflows/website.yml) runs the website tests and builds the website docker image - [`e2e-k3d.yml`](/.github/workflows/e2e-k3d.yml) runs the end-to-end tests - ## Authorization ### User management diff --git a/backend/README.md b/backend/README.md index 0e74271b4..9c1a550c8 100644 --- a/backend/README.md +++ b/backend/README.md @@ -6,8 +6,8 @@ All commands mentioned in this section are run from the `backend` directory unle ### Prerequisites -* Java 21 installed on your system -* A running PostgreSQL database (e.g. via a local [Kubernetes deployment](../kubernetes/README.md))]) +- Java 21 installed on your system +- A running PostgreSQL database (e.g. via a local [Kubernetes deployment](../kubernetes/README.md))]) ### Starting the backend @@ -24,18 +24,24 @@ The backend is configured via [Spring properties](https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config) that need to be passed on startup, e.g. via command line argument. You need to set: -* the database URL, username and password: + +- the database URL, username and password: + ``` --spring.datasource.url=jdbc:postgresql://localhost:5432/loculus --spring.datasource.username=postgres --spring.datasource.password=unsecure ``` -* the path to the config file (use `../generate_local_test_config.sh` to generate this file): + +- the path to the config file (use `../generate_local_test_config.sh` to generate this file): + ``` --loculus.config.path=../website/tests/config/backend_config.json ``` -* the url to fetch the public key for JWT verification + +- the url to fetch the public key for JWT verification (corresponding to the `jwks_uri` value in the `/.well-known/openid-configuration` endpoint of the Keycloak server): + ``` --spring.security.oauth2.resourceserver.jwt.jwk-set-uri=http://localhost:8083/realms/loculus/protocol/openid-connect/certs ``` @@ -44,13 +50,15 @@ We use Flyway, so that the service can provision an empty/existing DB without an The service listens, by default, to **port 8079**: . +Note: When using a postgresSQL development platform (e.g. pgAdmin) the hostname is 127.0.0.1 and not localhost - this is defined in the `deploy.py` file. + ### Operating the backend behind a proxy When running the backend behind a proxy, the proxy needs to set X-Forwarded headers: -* X-Forwarded-For -* X-Forwarded-Proto -* X-Forwarded-Prefix +- X-Forwarded-For +- X-Forwarded-Proto +- X-Forwarded-Prefix ## Development @@ -98,7 +106,7 @@ The backend provides a Swagger UI at dag.png +``` + +![snakemake DAG](dag.png) + ### Download data from NCBI virus Using NCBI `datasets` CLI, download all sequences and corresponding NCBI curated metadata for a configurable taxon. The taxon is specified using the NCBI Taxonomy ID, and includes all child taxa, i.e. dowloading sequences for the Ebola virus taxon ID includes all sequences for more specific Ebola virus (sub)species taxon ids. diff --git a/ingest/dag.png b/ingest/dag.png new file mode 100644 index 000000000..26667977b Binary files /dev/null and b/ingest/dag.png differ diff --git a/kubernetes/README.md b/kubernetes/README.md index 2b2543a8d..f98ed30f0 100644 --- a/kubernetes/README.md +++ b/kubernetes/README.md @@ -32,6 +32,7 @@ helm install loculus kubernetes/loculus -f my-values.yaml Install [k3d](https://k3d.io/v5.6.0/) and [helm](https://helm.sh/). +We deploy kubernetes via the `../deploy.py` script, this requires you to have `pyyaml` and `requests` installed. We recommend following the steps in [ingest](/ingest/README.md) to activate the loculus-ingest mamba environment which includes these packages, but the packages can also be installed individually. ### Setup for local development