Skip to content

Commit

Permalink
Merge pull request #165 from mobiusml/docker_info
Browse files Browse the repository at this point in the history
add docker info
  • Loading branch information
HRashidi authored Aug 19, 2024
2 parents 6cb0b9a + cab223d commit ab44386
Showing 1 changed file with 43 additions and 1 deletion.
44 changes: 43 additions & 1 deletion docs/pages/docker.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,45 @@
# Run with Docker

TODO: Add instructions on how to run Aana Apps with Docker.
We provide a docker-compose configuration to run the application in a Docker container in [Aana App Template](https://github.com/mobiusml/aana_app_template/blob/main?tab=readme-ov-file#running-with-docker).

Requirements:

- Docker Engine >= 26.1.0
- Docker Compose >= 1.29.2
- NVIDIA Driver >= 525.60.13

You can edit the [Dockerfile](https://github.com/mobiusml/aana_app_template/blob/main/Dockerfile) to assemble the image as you desire and
and [docker-compose file](https://github.com/mobiusml/aana_app_template/blob/main/docker-compose.yaml) for container instances and their environment variables.

To run the application, simply run the following command:

```bash
docker-compose up
```

The application will be accessible at `http://localhost:8000` on the host server.


!!! warning

If your applications requires GPU to run, you need to specify which GPU to use.

The applications will detect the available GPU automatically but you need to make sure that `CUDA_VISIBLE_DEVICES` is set correctly.

Sometimes `CUDA_VISIBLE_DEVICES` is set to an empty string and the application will not be able to detect the GPU. Use `unset CUDA_VISIBLE_DEVICES` to unset the variable.

You can also set the `CUDA_VISIBLE_DEVICES` environment variable to the GPU index you want to use: `CUDA_VISIBLE_DEVICES=0 docker-compose up`.


!!! Tip

Some models use Flash Attention for better performance. You can set the build argument `INSTALL_FLASH_ATTENTION` to `true` to install Flash Attention.

```bash
INSTALL_FLASH_ATTENTION=true docker-compose build
```

After building the image, you can use `docker-compose up` command to run the application.

You can also set the `INSTALL_FLASH_ATTENTION` environment variable to `true` in the `docker-compose.yaml` file.

0 comments on commit ab44386

Please sign in to comment.