Skip to content

Commit

Permalink
updated info and link to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
i-oden committed Dec 4, 2024
1 parent 4db54b4 commit d148b6e
Show file tree
Hide file tree
Showing 2 changed files with 95 additions and 38 deletions.
38 changes: 1 addition & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,43 +177,7 @@ All have the password: `password`

If you modify the database models (e.g. tables or indexes), you must create a migration for the changes. We use `Alembic` (via `flask-migrate`) which compares our database models with the running database to generate a suggested migration.

The new migration can be autogenerated in two main ways:

1. Make sure the development setup is running (`docker-compose up`) while doing the changes.

2. Reset the container using `docker-compose down` and then remove `flask init-db $$DB_TYPE &&` in `docker-compose.yml`. It will prevent the population of the database, allowing you to install the old schema and build your migration on top of it by running `docker-compose up`

To create the migration, run the command `flask db migrate -m <migration name>` in the running backend:

```bash
docker exec dds_backend flask db migrate -m <migration name>
```

This will create a migration in the folder `migrations/versions`. Confirm that the changes in the file match the changes you did, otherwise change the `upgrade` and `downgrade` functions as needed. Keep an eye out for changes to the `apscheduler` tables and indexes, and make sure they are not included in the migration. Once the migration looks ok, test it by running `flask db upgrade` in the backend:

```bash
docker exec dds_backend flask db upgrade
```

Finally, confirm that the database looks correct after running the migration and commit the migration file to git. Note that you need to run `black` on the generated migration file.

> ↩️ If you want to start over, restore the content of `migrations/versions` (remove new files, run `git restore` on the folder) and start from autogeneration method 2.

### Database issues while running `docker-compose up`

If you run into issues with complaints about the db while running `docker-compose up` you can try to reset the containers by running `docker-compose down` before trying again. If you still have issues, try cleaning up containers and volumes manually.

> ⚠️ These commands will remove _all_ containers and volumes!
> If you are working on other projects please be more selective.

```bash
docker rm $(docker ps -a -q) -f
docker volume prune
```

Then run `docker-compose up` as normal. The images will be rebuilt from scratch before launch.

If there are still issues, try deleting the `pycache` folders and repeat the above steps.
For instructions on how to do this, see [the README in the migrations directory](./migrations/README).

## Run tests

Expand Down
95 changes: 94 additions & 1 deletion migrations/README
Original file line number Diff line number Diff line change
@@ -1 +1,94 @@
Single-database configuration for Flask.
# Database Migration Guide

If you modify the database models (e.g. tables or indexes), you must create a migration for the changes. We use `Alembic` (via `flask-migrate`) which compares our database models with the running database to generate a suggested migration.

This document explains how to generating new database migration.

## Steps for Generating Migrations

The new migration can be autogenerated in _two main ways_. Method 1 below should be used as the default, and Method 2 should be used in order to start over, e.g. in the case of something going wrong.

### Method 1

1. Make sure the development setup is running (`docker-compose up`) while doing the changes.
2. To create the migration, do **one of the following points**. This will create a migration in the folder `migrations/versions`.

- Enter the container (`docker exec -it dds_backend /bin/sh`) and generate the migration (`flask db migrate -m <migration name>`)
- Run the following command (outside of the container): `docker exec dds_backend flask db migrate -m <migration name>`

3. Confirm that the changes in the file match the changes you did, otherwise change the `upgrade` and `downgrade` functions as needed.

> NB! Keep an eye out for changes to the `apscheduler` tables and indexes, and **make sure they are not included in the migration**.

4. Once the migration looks ok, test it by running `flask db upgrade` in the backend, either by first entering the container (see first point under step 2 above) or by running the following (outside of the container): `docker exec dds_backend flask db upgrade`

5. Finally, confirm that the database looks correct after running the migration and commit the migration file to git.

> Note that you need to run `black` on the generated migration file.

### Method 2

If you want to start over, e.g. if something went wrong when going through method 1:

1. Restore the content of `migrations/versions`:

a) Remove the new migrations file(s)
b) Run `git restore` on the `migrations/versions` folder

2. Reset the container: `docker-compose down`
3. Remove `flask init-db $$DB_TYPE &&` in the `docker-compose.yml` file. This will prevent the population of the database and instead allow the install of the old schema.
4. Start the container again: `docker-compose up`
5. Follow Method 1

## Database issues while running `docker-compose up`

If you run into issues with complaints about the db while running `docker-compose up` you can try to reset the containers by running `docker-compose down` before trying again. If you still have issues, try cleaning up containers and volumes manually.

> ⚠️ These commands will remove _all_ containers and volumes!
> If you are working on other projects please be more selective.

```bash
docker rm $(docker ps -a -q) -f
docker volume prune
```

Then run `docker-compose up` as normal. The images will be rebuilt from scratch before launch.

If there are still issues, try deleting the `pycache` folders and repeat the above steps.


THE TEXT BELOW IS WHAT ALVARO WROTE. I WILL BE ADDING PARTS OF IT INTO THE TEXT ABOVE.

### 1. Start the Containers

Before making any model changes, ensure the containers are running locally.

### 2. Modify the Database Models

Edit the `models.py` file to reflect the changes you need. This might include:

- Adding new tables or columns
- Modifying existing fields
- Deleting tables or columns

### 3. Access the Backend Container

Connect a terminal into the backend container.

### 4. Run the Migration Command

Inside the container, execute the migration command:

```bash
flask db migrate -m "Your migration message"
```

### 5. Review the Generated Migration File

A new migration script will be generated in the `migrations/versions` directory. **Review this file carefully** to ensure it correctly represents your intended changes.

For reference on available operations and customization options, check the [Alembic Documentation](https://alembic.sqlalchemy.org/en/latest/ops.html).

### 6. Commit the Migration File

Finally, commit the migration script to version control.

0 comments on commit d148b6e

Please sign in to comment.