Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored the project building and deploying procedure. Added frontend #43

Merged
merged 9 commits into from
Oct 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,25 @@ jobs:
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip --no-cache-dir
pip install flake8 ruff --no-cache-dir
pip install flake8 ruff isort --no-cache-dir
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Git Clone Action
uses: actions/checkout@v2
with:
repository: ${{ github.repository }}
path: backend/

- name: Run flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 backend --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings
flake8 backend --count --exit-zero --max-complexity=10 --max-line-length=79 --statistics --config=setup.cfg

- name: Run ruff
run: |
python -m ruff check .

- name: Run isort
run: |
isort -c .

63 changes: 41 additions & 22 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,33 +37,51 @@ jobs:
push: true
tags: ${{ secrets.DOCKER_USERNAME }}/volunteers_backend:latest

# build_frontend_and_push_to_docker_hub:
# name: Push frontend Docker image to DockerHub
# runs-on: ubuntu-latest
# steps:
# - name: Check out the repo
# uses: actions/checkout@v3
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
# - name: Login to Docker
# uses: docker/login-action@v2
# with:
# username: ${{ secrets.DOCKER_USERNAME }}
# password: ${{ secrets.DOCKER_PASSWORD }}
# - name: Push to DockerHub
# uses: docker/build-push-action@v4
# with:
# context: [email protected]:volunteers-for-city-projects/volunteers-frontend.git
# file: /infra_bt/Dockerfile
# push: true
# tags: ${{ secrets.DOCKER_USERNAME }}/volunteers_frontend:latest
build_frontend_and_push_to_docker_hub:
name: Push frontend Docker image to DockerHub
runs-on: ubuntu-latest
steps:
- name: Login to Docker
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Push to DockerHub
uses: docker/build-push-action@v4
with:
context: https://github.com/volunteers-for-city-projects/volunteers-frontend.git#develop
push: true
tags: ${{ secrets.DOCKER_USERNAME }}/volunteers_frontend:latest

build_gateway_and_push_to_docker_hub:
name: Push gateway Docker image to DockerHub
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Docker
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Push to DockerHub
uses: docker/build-push-action@v4
with:
build-args:
NGINX_VERSION=${{ vars.NGINX_VERSION_BUILD }}
context: ./infra_bt/
push: true
tags: ${{ secrets.DOCKER_USERNAME }}/volunteers_gateway:latest

deploy:
runs-on: ubuntu-latest
needs:
# Дождёмся билда всех образов
- build_backend_and_push_to_docker_hub
# - build_frontend_and_push_to_docker_hub
- build_frontend_and_push_to_docker_hub
- build_gateway_and_push_to_docker_hub
steps:
- name: Checkout repo
uses: actions/checkout@v3
Expand All @@ -77,7 +95,8 @@ jobs:
key: ${{ secrets.SSH_KEY }}
passphrase: ${{ secrets.SSH_PASSPHRASE }}
source: "infra_bt/docker-compose.production.yml"
target: "volunteers-for-city-projects"
target: "better-together/"
strip_components: 1
- name: Executing remote ssh commands to deploy
uses: appleboy/ssh-action@master
with:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -167,3 +167,6 @@ cython_debug/
# static
**/collected_static/
media/

# pre-commit
.pre-commit-config.yaml
4 changes: 2 additions & 2 deletions backend/backend/settings.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import os
from pathlib import Path

from dotenv import load_dotenv
from pathlib import Path

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
Expand Down Expand Up @@ -153,7 +153,7 @@
STATIC_ROOT = Path(BASE_DIR, 'collected_static')

MEDIA_URL = 'media/'
MEDIA_ROOT = '/media'
MEDIA_ROOT = Path(BASE_DIR, 'media')

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field
Expand Down
5 changes: 5 additions & 0 deletions infra_bt/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.env
.env.example
docker-compose.production.yml
docker-compose.yml
Dockerfile
3 changes: 3 additions & 0 deletions infra_bt/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ARG NGINX_VERSION_BUILD
FROM nginx:$NGINX_VERSION_BUILD
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
26 changes: 12 additions & 14 deletions infra_bt/docker-compose.production.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ version: '3'

volumes:
pg_data:
static:
media:
static_data:
media_data:


services:
Expand All @@ -27,27 +27,25 @@ services:
image: 1yunker/volunteers_backend
env_file: .env
volumes:
- static:/backend_static
- media:/media
Copy link
Collaborator

@1yunker 1yunker Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Поясни зачем настройки media совсем убрал, а не исправил на media_data:/media?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Удалял, так как был "сюрприз" с той строчкой, которую ты прописал, а корректную версию прописать забыл. Было бы не плохо созвониться обсудить, там еще есть пара "моментов", которые на счет медиа хотел прояснить.

- static_data:/backend_static
- media_data:/app/media
depends_on:
db:
condition: service_healthy

# frontend:
# image: 1yunker/volunteers_frontend
# command: cp -r /app/result_build/. /static/
# volumes:
# - static:/static
frontend:
image: 1yunker/volunteers_frontend
command: cp -r /app/build/. /static_files/
volumes:
- static_data:/static_files

gateway:
image: nginx:1.25.2-alpine3.18-slim
image: 1yunker/volunteers_gateway
ports:
- 8000:80
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
# - ../docs/:/usr/share/nginx/html/api/docs/
- static:/static
- media:/media
- static_data:/usr/share/nginx/html
- media_data:/usr/share/nginx/html/media
depends_on:
backend:
condition: service_started
39 changes: 17 additions & 22 deletions infra_bt/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ version: '3.3'

volumes:
pg_data:
static:
media:
static_data:
media_data:


name: volunteers
Expand Down Expand Up @@ -32,45 +32,40 @@ services:
build: ../backend/
env_file: .env
volumes:
- static:/backend_static
- media:/media
- static_data:/backend_static
- media_data:/app/media
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Через /app/media пробовал, почему-то не работает так. В корне есть /media по умолчанию в контейнере, он почему-то на нее ссылаться начинает.

Copy link
Collaborator Author

@ArtemKAF ArtemKAF Oct 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Вот из-за того, что в контейнер вольюм подключается по пути /media, мы и получимаем в итоге ссылки на подключаемое оборудование в ОС контейнера (особенность линукса, у которого /media - это директория куда автоматом монтируются подключаемые носители flopy, cdrom и прочее). Это и есть "сюрприз". Поэтому и подключил /app/media, так как при сборке контейнера бэкэнд складывается в директорию /app.
А при настройке MEDIA_ROOT = BASE_DIR / 'media' бэкэнд и будет сохранять медиа по пути /app/media

command:
- /bin/sh
- -c
- |
python manage.py collectstatic -c --noinput
cp -r /app/collected_static/. /backend_static/static/
cp -r /app/collected_static/. /backend_static/static
python manage.py migrate
gunicorn --bind 0.0.0.0:8000 backend.wsgi
restart: unless-stopped
depends_on:
db:
condition: service_healthy

# frontend:
# build:
# context: [email protected]:volunteers-for-city-projects/volunteers-frontend.git
# dockerfile: Dockerfile
# command:
# - /bin/sh
# - -c
# - |
# npm run build result_build
# cp -r /app/result_build/ static_files/
# volumes:
# - ../frontend:/app/
# - static:/static_files
frontend:
build:
context: https://github.com/volunteers-for-city-projects/volunteers-frontend.git#develop
command:
- /bin/sh
- -c
- |
cp -r /app/. /static_files/
volumes:
- static_data:/static_files

nginx:
image: nginx:1.25.2-alpine3.18-slim
ports:
- "8000:80"
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
# - ../frontend/build:/usr/share/nginx/html/
# - ../docs/:/usr/share/nginx/html/api/docs/
- static:/static
- media:/media
- static_data:/usr/share/nginx/html
- media_data:/usr/share/nginx/html/media/
depends_on:
backend:
condition: service_started
23 changes: 9 additions & 14 deletions infra_bt/nginx.conf
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
server {
listen 80;
server_tokens off;
client_max_body_size 20M;

root /usr/share/nginx/html;

location /api/ {
proxy_set_header Host $http_host;
Expand All @@ -9,26 +13,17 @@ server {
proxy_set_header Host $http_host;
proxy_pass http://backend:8000/swagger/;
}
location /redoc/ {
proxy_set_header Host $http_host;
proxy_pass http://backend:8000/redoc/;
}
location /admin/ {
proxy_set_header Host $http_host;
proxy_pass http://backend:8000/admin/;
}
location /static/ {
root /static;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тоже не понимаю зачем затирать эти настройки...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Здесь эти строчки становятся не нужными из-за изменения настроек монтирования тома static_data и переноса настройки root вне секции location / {}

location /media/ {
root /;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

И эти настройки тоже зачем удалены?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А вот тут я тоже погорячился убрать и не вернуть прежде чем обсудить где именно расположить папку медиа относительно всей статики вцелом.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Предлагаю в settings изменить MEDIA_ROOT = '/media'

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Да, я тоже к этому пришел и внес эти изменения в свой ПР.


# location / {
# proxy_set_header Host $http_host;
# proxy_pass http://backend:8000/;
# }

location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri /index.html;
try_files $uri $uri/ =404;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down