Skip to content

Commit

Permalink
docs: update backend dev guide, remove redundant, add tunneling info
Browse files Browse the repository at this point in the history
  • Loading branch information
spwoodcock committed May 22, 2024
1 parent 8efef02 commit cd9cc7c
Showing 1 changed file with 77 additions and 82 deletions.
159 changes: 77 additions & 82 deletions docs/dev/Backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ dependencies, and creates a container for each the database, the api,
and the frontend. These containers talk to each other via the
URLs defined in the docker-compose file and your env file.

### 1A: Starting the Containers

1. You will need to
[Install Docker](https://docs.docker.com/engine/install/)
and ensure that it is running on your local machine.
Expand All @@ -43,11 +41,12 @@ URLs defined in the docker-compose file and your env file.
> Note: the database host `fmtm-db` is automatically
> resolved by docker compose to the database container IP.
#### Bundled ODK Central
### Bundled ODK Central

- FMTM uses ODK Central to store ODK data.
- To facilitate faster development, the Docker setup includes a Central server.
- The credentials are provided via the `.env` file.
- The credentials are provided via the `.env` file, and the default URL to
access Central from within containers is: `https://proxy`.

> Alternatively, you may provide credentials to an external Central server
> in the `.env`.
Expand All @@ -58,48 +57,12 @@ To run the local development setup without ODK Central (use external server):
dc --profile no-odk up -d
```

### 1B: Import Test Data

Some test data is available to get started quickly.

1. Navigate to the `import-test-data` endpoint in the API docs page:
<http://api.fmtm.localhost:7050/docs#/debug/import_test_data_debug_import_test_data_get>
2. Click `Try it out`, then `execute`.

## 2. Start the API without Docker

To run FMTM without Docker, you will need to start the database, then the API.

### 2A: Starting the Database

#### Option 1: Run the Database (only) in Docker

Running the database in Docker means postgres does
not need to be installed on your local machine.

1. You will need to
[Install Docker](https://docs.docker.com/engine/install/)
and ensure that it is running on your local machine.
2. Start an instance of Postgres (with Postgis):

```bash
docker compose up -d fmtm-db
```

The database should be accessible at localhost:5438.

> Note: if port 5438 is already taken, then change the
> `-p ANY_PORT:5432` declaration.
#### Option 2: Run the database locally

For advanced users, it is also possible to run a
postgresql/postgis database locally, however you will
need to set it up yourself and make it accessible on a port.

### 2B. Starting the API

After starting the database, from the command line:
- To run FMTM without Docker, you will need to start the database, then the API.
- First start a Postgres database running on a port on your machine.
- The database must have the Postgis extension installed.
- After starting the database, from the command line:

1. Navigate to the top level directory of the FMTM project.
2. Install PDM with: `pip install pdm`
Expand All @@ -109,26 +72,8 @@ After starting the database, from the command line:

The API should now be accessible at: <http://api.fmtm.localhost:7050/docs>

## 3. Hybrid Docker/Local

- It is not recommended to run FMTM in a container while
using a local database on your machine.
- It is possible, but complicates the docker networking slightly.
- The FMTM container cannot see the local machine's
`localhost`, so we need a workaround.
- **Option 1**: add `network_mode: "host"` under the `api:`
service in the **docker-compose.yml** file.
- **Option 2**: use the direct container IP address for the
database for **FMTM_DB_HOST**, found via `docker inspect fmtm_db`.

## Backend Tips

### Implement authorization on an endpoints

To add authentication to an endpoint, import `login_required`
from `auth` module, you can use it as a decorator or use
fastapi `Depends(login_required)` on endpoints.

### Database Migration

#### Creating Migration Files
Expand Down Expand Up @@ -164,7 +109,7 @@ docker compose up -d migrations
}
```

## Backend Debugging
### Interactive Debugging

- The `docker-compose.yml` builds FMTM using the `debug` target in the Dockerfile.
- The debug image contains `debugpy` to assist debugging in the container.
Expand Down Expand Up @@ -215,7 +160,7 @@ Example launch.json config for vscode:
> Note: either port 5678 needs to be bound to your localhost (default),
> or the `host` parameter can be set to the container IP address.
## Backend Testing
### Running Tests

To run the backend tests locally, run:

Expand All @@ -236,15 +181,7 @@ To assess performance of endpoints:
- While in debug mode (DEBUG=True), access any endpoint.
- Add the `?profile=true` arg to the URL to view the execution time.

## Using the local version of ODK Central

- During project creation a Central ODK URL must be provided.
- If you set up FMTM with docker and have ODK Central
running in a container, you can use the URL:
`https://proxy`
- The credentials should be present in your `.env` file.

## Debugging osm-fieldwork
### Debugging osm-fieldwork

`osm-fieldwork` is an integral package for much of the functionality in FMTM.

Expand All @@ -266,7 +203,7 @@ Creating a new release during development may not always be feasible.

> Note: this is useful for debugging features during active development.

## Accessing S3 Files use s3fs
### Accessing S3 Files use s3fs

The s3fs tool allows you to mount an S3 bucket on your filesystem,
to browse like any other directory.
Expand Down Expand Up @@ -309,14 +246,14 @@ url=http://s3.fmtm.localhost:7050 0 0`

> Note: you should replace USERNAME with your linux username.

## Running JOSM in the dev stack
### Running JOSM in the dev stack

- Run JOSM with FMTM:

```bash
docker compose \
-f docker-compose.yml \
-f josm/docker-compose.yml \
-f contrib/josm/docker-compose.yml \
up -d
```

Expand All @@ -326,10 +263,68 @@ Access the JOSM GUI in browser: <http://localhost:8112>

You can now call the JOSM API from FMTM and changes will be reflected in the GUI.

## Conclusion
### Debugging local FMTM on mobile

- It's difficult to debug services running on localhost from your mobile phone.
- An easy way to do this is by tunneling: Cloudflare provides a great free
solution for this (an alternative is Ngrok).
- To run the tunnel to the FMTM API:

```bash
docker compose \
-f docker-compose.yml \
-f contrib/tunnel/fmtm/docker-compose.yml \
up -d
```

View the website to access FMTM remotely (e.g. via mobile):

```bash
docker compose \
-f docker-compose.yml \
-f contrib/tunnel/fmtm/docker-compose.yml \
logs api-tunnel
```

- Now the final step is to add the provided tunnel URL to the allowed CORS origins.
- Add to your `.env` file:

```dotenv
EXTRA_CORS_ORIGINS=https://the-url-you-were-given.trycloudflare.com
```

- Then restart your API service:

```bash
docker compose restart api
```

### Using local ODK Central on mobile

- Sometimes you wish to use a project in your local ODK Central, via ODK Collect
on your mobile.
- To run the tunnel to the ODK Central API:

```bash
docker compose \
-f docker-compose.yml \
-f contrib/tunnel/odk/docker-compose.yml \
up -d
```

- During project creation, set the ODK Central server URL to the provided tunnel
address for the ODK Central API:

```bash
docker compose \
-f docker-compose.yml \
-f contrib/tunnel/odk/docker-compose.yml \
logs central-tunnel
```

> The credentials for the local ODK Central instance are:
> Username: <[email protected]>
> Password: Password1234

Running the FMTM project is easy with Docker. You can also run the
project locally outside of Docker, but it requires more setup. The
frontend is built with React and Typescript, and the backend is built
with FastAPI. Use the tips provided to customize and extend the
functionality of the project.
- Now when you access the project via a QRCode on mobile, the connection to ODK
Central should work.

0 comments on commit cd9cc7c

Please sign in to comment.