diff --git a/README.md b/README.md index f5be819d3..ab4da964f 100644 --- a/README.md +++ b/README.md @@ -177,43 +177,7 @@ All have the password: `password` If you modify the database models (e.g. tables or indexes), you must create a migration for the changes. We use `Alembic` (via `flask-migrate`) which compares our database models with the running database to generate a suggested migration. -The new migration can be autogenerated in two main ways: - -1. Make sure the development setup is running (`docker-compose up`) while doing the changes. - -2. Reset the container using `docker-compose down` and then remove `flask init-db $$DB_TYPE &&` in `docker-compose.yml`. It will prevent the population of the database, allowing you to install the old schema and build your migration on top of it by running `docker-compose up` - -To create the migration, run the command `flask db migrate -m ` in the running backend: - -```bash -docker exec dds_backend flask db migrate -m -``` - -This will create a migration in the folder `migrations/versions`. Confirm that the changes in the file match the changes you did, otherwise change the `upgrade` and `downgrade` functions as needed. Keep an eye out for changes to the `apscheduler` tables and indexes, and make sure they are not included in the migration. Once the migration looks ok, test it by running `flask db upgrade` in the backend: - -```bash -docker exec dds_backend flask db upgrade -``` - -Finally, confirm that the database looks correct after running the migration and commit the migration file to git. Note that you need to run `black` on the generated migration file. - -> ↩️ If you want to start over, restore the content of `migrations/versions` (remove new files, run `git restore` on the folder) and start from autogeneration method 2. - -### Database issues while running `docker-compose up` - -If you run into issues with complaints about the db while running `docker-compose up` you can try to reset the containers by running `docker-compose down` before trying again. If you still have issues, try cleaning up containers and volumes manually. - -> ⚠️ These commands will remove _all_ containers and volumes! -> If you are working on other projects please be more selective. - -```bash -docker rm $(docker ps -a -q) -f -docker volume prune -``` - -Then run `docker-compose up` as normal. The images will be rebuilt from scratch before launch. - -If there are still issues, try deleting the `pycache` folders and repeat the above steps. +For instructions on how to do this, see [the README in the migrations directory](./migrations/README). ## Run tests diff --git a/migrations/README b/migrations/README index 0e0484415..69ee057c9 100644 --- a/migrations/README +++ b/migrations/README @@ -1 +1,94 @@ -Single-database configuration for Flask. +# Database Migration Guide + +If you modify the database models (e.g. tables or indexes), you must create a migration for the changes. We use `Alembic` (via `flask-migrate`) which compares our database models with the running database to generate a suggested migration. + +This document explains how to generating new database migration. + +## Steps for Generating Migrations + +The new migration can be autogenerated in _two main ways_. Method 1 below should be used as the default, and Method 2 should be used in order to start over, e.g. in the case of something going wrong. + +### Method 1 + +1. Make sure the development setup is running (`docker-compose up`) while doing the changes. +2. To create the migration, do **one of the following points**. This will create a migration in the folder `migrations/versions`. + + - Enter the container (`docker exec -it dds_backend /bin/sh`) and generate the migration (`flask db migrate -m `) + - Run the following command (outside of the container): `docker exec dds_backend flask db migrate -m ` + +3. Confirm that the changes in the file match the changes you did, otherwise change the `upgrade` and `downgrade` functions as needed. + + > NB! Keep an eye out for changes to the `apscheduler` tables and indexes, and **make sure they are not included in the migration**. + +4. Once the migration looks ok, test it by running `flask db upgrade` in the backend, either by first entering the container (see first point under step 2 above) or by running the following (outside of the container): `docker exec dds_backend flask db upgrade` + +5. Finally, confirm that the database looks correct after running the migration and commit the migration file to git. + + > Note that you need to run `black` on the generated migration file. + +### Method 2 + +If you want to start over, e.g. if something went wrong when going through method 1: + +1. Restore the content of `migrations/versions`: + + a) Remove the new migrations file(s) + b) Run `git restore` on the `migrations/versions` folder + +2. Reset the container: `docker-compose down` +3. Remove `flask init-db $$DB_TYPE &&` in the `docker-compose.yml` file. This will prevent the population of the database and instead allow the install of the old schema. +4. Start the container again: `docker-compose up` +5. Follow Method 1 + +## Database issues while running `docker-compose up` + +If you run into issues with complaints about the db while running `docker-compose up` you can try to reset the containers by running `docker-compose down` before trying again. If you still have issues, try cleaning up containers and volumes manually. + +> ⚠️ These commands will remove _all_ containers and volumes! +> If you are working on other projects please be more selective. + +```bash +docker rm $(docker ps -a -q) -f +docker volume prune +``` + +Then run `docker-compose up` as normal. The images will be rebuilt from scratch before launch. + +If there are still issues, try deleting the `pycache` folders and repeat the above steps. + + +THE TEXT BELOW IS WHAT ALVARO WROTE. I WILL BE ADDING PARTS OF IT INTO THE TEXT ABOVE. + +### 1. Start the Containers + +Before making any model changes, ensure the containers are running locally. + +### 2. Modify the Database Models + +Edit the `models.py` file to reflect the changes you need. This might include: + +- Adding new tables or columns +- Modifying existing fields +- Deleting tables or columns + +### 3. Access the Backend Container + +Connect a terminal into the backend container. + +### 4. Run the Migration Command + +Inside the container, execute the migration command: + +```bash +flask db migrate -m "Your migration message" +``` + +### 5. Review the Generated Migration File + +A new migration script will be generated in the `migrations/versions` directory. **Review this file carefully** to ensure it correctly represents your intended changes. + +For reference on available operations and customization options, check the [Alembic Documentation](https://alembic.sqlalchemy.org/en/latest/ops.html). + +### 6. Commit the Migration File + +Finally, commit the migration script to version control.