From cd9cc7ca35829c4080ed536ce97cac528a30e80e Mon Sep 17 00:00:00 2001 From: spwoodcock Date: Wed, 22 May 2024 20:25:48 +0100 Subject: [PATCH] docs: update backend dev guide, remove redundant, add tunneling info --- docs/dev/Backend.md | 159 +++++++++++++++++++++----------------------- 1 file changed, 77 insertions(+), 82 deletions(-) diff --git a/docs/dev/Backend.md b/docs/dev/Backend.md index 9dfa08c491..16a1b7a5aa 100644 --- a/docs/dev/Backend.md +++ b/docs/dev/Backend.md @@ -21,8 +21,6 @@ dependencies, and creates a container for each the database, the api, and the frontend. These containers talk to each other via the URLs defined in the docker-compose file and your env file. -### 1A: Starting the Containers - 1. You will need to [Install Docker](https://docs.docker.com/engine/install/) and ensure that it is running on your local machine. @@ -43,11 +41,12 @@ URLs defined in the docker-compose file and your env file. > Note: the database host `fmtm-db` is automatically > resolved by docker compose to the database container IP. -#### Bundled ODK Central +### Bundled ODK Central - FMTM uses ODK Central to store ODK data. - To facilitate faster development, the Docker setup includes a Central server. -- The credentials are provided via the `.env` file. +- The credentials are provided via the `.env` file, and the default URL to + access Central from within containers is: `https://proxy`. > Alternatively, you may provide credentials to an external Central server > in the `.env`. @@ -58,48 +57,12 @@ To run the local development setup without ODK Central (use external server): dc --profile no-odk up -d ``` -### 1B: Import Test Data - -Some test data is available to get started quickly. - -1. Navigate to the `import-test-data` endpoint in the API docs page: - -2. Click `Try it out`, then `execute`. - ## 2. Start the API without Docker -To run FMTM without Docker, you will need to start the database, then the API. - -### 2A: Starting the Database - -#### Option 1: Run the Database (only) in Docker - -Running the database in Docker means postgres does -not need to be installed on your local machine. - -1. You will need to - [Install Docker](https://docs.docker.com/engine/install/) - and ensure that it is running on your local machine. -2. Start an instance of Postgres (with Postgis): - -```bash -docker compose up -d fmtm-db -``` - -The database should be accessible at localhost:5438. - -> Note: if port 5438 is already taken, then change the -> `-p ANY_PORT:5432` declaration. - -#### Option 2: Run the database locally - -For advanced users, it is also possible to run a -postgresql/postgis database locally, however you will -need to set it up yourself and make it accessible on a port. - -### 2B. Starting the API - -After starting the database, from the command line: +- To run FMTM without Docker, you will need to start the database, then the API. +- First start a Postgres database running on a port on your machine. + - The database must have the Postgis extension installed. +- After starting the database, from the command line: 1. Navigate to the top level directory of the FMTM project. 2. Install PDM with: `pip install pdm` @@ -109,26 +72,8 @@ After starting the database, from the command line: The API should now be accessible at: -## 3. Hybrid Docker/Local - -- It is not recommended to run FMTM in a container while - using a local database on your machine. -- It is possible, but complicates the docker networking slightly. -- The FMTM container cannot see the local machine's - `localhost`, so we need a workaround. -- **Option 1**: add `network_mode: "host"` under the `api:` - service in the **docker-compose.yml** file. -- **Option 2**: use the direct container IP address for the - database for **FMTM_DB_HOST**, found via `docker inspect fmtm_db`. - ## Backend Tips -### Implement authorization on an endpoints - -To add authentication to an endpoint, import `login_required` -from `auth` module, you can use it as a decorator or use -fastapi `Depends(login_required)` on endpoints. - ### Database Migration #### Creating Migration Files @@ -164,7 +109,7 @@ docker compose up -d migrations } ``` -## Backend Debugging +### Interactive Debugging - The `docker-compose.yml` builds FMTM using the `debug` target in the Dockerfile. - The debug image contains `debugpy` to assist debugging in the container. @@ -215,7 +160,7 @@ Example launch.json config for vscode: > Note: either port 5678 needs to be bound to your localhost (default), > or the `host` parameter can be set to the container IP address. -## Backend Testing +### Running Tests To run the backend tests locally, run: @@ -236,15 +181,7 @@ To assess performance of endpoints: - While in debug mode (DEBUG=True), access any endpoint. - Add the `?profile=true` arg to the URL to view the execution time. -## Using the local version of ODK Central - -- During project creation a Central ODK URL must be provided. -- If you set up FMTM with docker and have ODK Central - running in a container, you can use the URL: - `https://proxy` -- The credentials should be present in your `.env` file. - -## Debugging osm-fieldwork +### Debugging osm-fieldwork `osm-fieldwork` is an integral package for much of the functionality in FMTM. @@ -266,7 +203,7 @@ Creating a new release during development may not always be feasible. > Note: this is useful for debugging features during active development. -## Accessing S3 Files use s3fs +### Accessing S3 Files use s3fs The s3fs tool allows you to mount an S3 bucket on your filesystem, to browse like any other directory. @@ -309,14 +246,14 @@ url=http://s3.fmtm.localhost:7050 0 0` > Note: you should replace USERNAME with your linux username. -## Running JOSM in the dev stack +### Running JOSM in the dev stack - Run JOSM with FMTM: ```bash docker compose \ -f docker-compose.yml \ - -f josm/docker-compose.yml \ + -f contrib/josm/docker-compose.yml \ up -d ``` @@ -326,10 +263,68 @@ Access the JOSM GUI in browser: You can now call the JOSM API from FMTM and changes will be reflected in the GUI. -## Conclusion +### Debugging local FMTM on mobile + +- It's difficult to debug services running on localhost from your mobile phone. +- An easy way to do this is by tunneling: Cloudflare provides a great free + solution for this (an alternative is Ngrok). +- To run the tunnel to the FMTM API: + +```bash +docker compose \ + -f docker-compose.yml \ + -f contrib/tunnel/fmtm/docker-compose.yml \ + up -d +``` + +View the website to access FMTM remotely (e.g. via mobile): + +```bash +docker compose \ + -f docker-compose.yml \ + -f contrib/tunnel/fmtm/docker-compose.yml \ + logs api-tunnel +``` + +- Now the final step is to add the provided tunnel URL to the allowed CORS origins. +- Add to your `.env` file: + +```dotenv +EXTRA_CORS_ORIGINS=https://the-url-you-were-given.trycloudflare.com +``` + +- Then restart your API service: + +```bash +docker compose restart api +``` + +### Using local ODK Central on mobile + +- Sometimes you wish to use a project in your local ODK Central, via ODK Collect + on your mobile. +- To run the tunnel to the ODK Central API: + +```bash +docker compose \ + -f docker-compose.yml \ + -f contrib/tunnel/odk/docker-compose.yml \ + up -d +``` + +- During project creation, set the ODK Central server URL to the provided tunnel + address for the ODK Central API: + +```bash +docker compose \ + -f docker-compose.yml \ + -f contrib/tunnel/odk/docker-compose.yml \ + logs central-tunnel +``` + +> The credentials for the local ODK Central instance are: +> Username: +> Password: Password1234 -Running the FMTM project is easy with Docker. You can also run the -project locally outside of Docker, but it requires more setup. The -frontend is built with React and Typescript, and the backend is built -with FastAPI. Use the tips provided to customize and extend the -functionality of the project. +- Now when you access the project via a QRCode on mobile, the connection to ODK + Central should work.