Dynamic qrcode generation via frontend

[spwoodcock]

What type of PR is this? (check all applicable)

• 🍕 Feature
• 🐛 Bug Fix
• 📝 Documentation Update
• 🎨 Style
• 🧑‍💻 Code Refactor
• 🔥 Performance Improvements
• ✅ Test
• 🤖 Build
• 🔁 CI
• 📦 Chore (Release)
• ⏩ Revert

Related Issue

Fixes #736
Partially fixes #966
Partially fixes #1077

Describe this PR

Current Setup

Lack of dynamic fields in QR code:

• We generate QR codes in bulk when a project is created, then save to the DB.
• The issue is that the QR code wants to contain dynamic fields, such as the user who is completing the task.

Other:

• Unencrypted ODK credentials.
• Inefficient calling of tasks-list every time the task boundary is clicked.

New Setup

Dynamic QR codes:

• I moved the QR code generation to the frontend.
• The backend generates the ODK Central appuser token, and stores it encrypted in the database.
• This token is returned to the frontend from the tasks-list endpoint.
• The ODK Collect JSON is generated via the frontend, base64 encoded, then embedded in a QR code.
  • The JSON also includes field metadata_username where the OSM username is included.
  • I also misuse the metadata_email field to include the task id from FMTM, to hopefully be passed through to submissions.

Other:

• I added default ODK credential fields (encrypted) to the organisations table (which we can use later to solve Map ODK Central Credentials with organization name #966).
• I updated the project ODK credentials to be encrypted by default too.
• QR code generation on the frontend is very performant.
• Updated odk password to SecretStr type, so defaults to obfuscated ****. To extract the actual value, use .get_secret_value()
• Updated fmtm-splitter to the first stable major release: v1.0.0.

Screenshots

Important Considerations

• Relies on a new release from osm-fieldwork
• This is a big change, and hence has some migrations for the database.
• However, to preserve QR codes from existing test data, and to encrypt the ODK credentials prior to usage, I added a script that can do this.
• It must be ran prior to applying the migrations.
  • The script is under migrations/qrcode_to_odktoken.py.
• To use:

1. Pull the latest code or checkout to feat/frontend-qr-codes
2. Enter the database container:

docker compose exec fmtm-db bash

# Apply required migrations
psql -U fmtm
ALTER TABLE IF EXISTS public.tasks
    ADD COLUMN IF NOT EXISTS odk_token VARCHAR;
exit

3. Install extra dependencies in the api container:

docker compose exec -u root api bash
apt update && apt install libzbar0 --no-install-recommends
exit
docker compose exec api bash
pip install pillow pyzbar cryptography
# Generate a encryption key (save this for later)
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
exit

4. Add the variable ENCRYPTION_KEY=xxx to your .env file, using the value generated above.
5. Restart the api container with your .env set: docker compose up -d api
6. Run the script:

docker cp migrations/qrcode_to_odktoken.py fmtm-api-1:/opt/app/
docker compose exec api bash
cd app
python qrcode_to_odktoken.py --apply

Checklist before requesting a review

• 📖 Read the FMTM Contributing Guide: https://github.com/hotosm/fmtm/blob/main/CONTRIBUTING.md
• 📖 Read the FMTM Code of Conduct: https://github.com/hotosm/fmtm/blob/main/CODE_OF_CONDUCT.md
• 👷‍♀️ Create small PRs. In most cases, this will be possible.
• ✅ Provide tests for your changes.
• 📝 Use descriptive commit messages.
• 📗 Update any related documentation and include any relevant screenshots.

[optional] What gif best describes this PR or how it makes you feel?

[nrjadkry]

I am getting such error when I scan the QR Code from odk collect.
QR Code does not contain valid settings

@spwoodcock , I think we needed to compress the json using zlib before encoding the result into base64.
I guess, this is missing in the frontend ??

[spwoodcock]

Thanks @nrjadkry #1159