Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/cd #78

Merged
merged 32 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
881b21d
Update adding staging mechanism
zacdezgeo Oct 11, 2024
8464ed8
Add CD pipeline
zacdezgeo Oct 11, 2024
0249d8d
Solve f-string matching
zacdezgeo Oct 11, 2024
59905e9
Update event to push on main
zacdezgeo Oct 11, 2024
965fefe
Add dev CD
zacdezgeo Oct 11, 2024
6343331
Remove npm and node specific jobs
zacdezgeo Oct 11, 2024
7f52fe0
Update variables
zacdezgeo Oct 11, 2024
5e7a1a7
Update environment
zacdezgeo Oct 11, 2024
6753e28
Update role
zacdezgeo Oct 11, 2024
0f315b8
Remove cdk bootstrap
zacdezgeo Oct 11, 2024
6399e79
Add working directory for cdk deployment
zacdezgeo Oct 11, 2024
222255a
Add cdk requirements
zacdezgeo Oct 11, 2024
bd5cc2b
Update cdk requirements
zacdezgeo Oct 11, 2024
4131cfd
Update deployment variables
zacdezgeo Oct 11, 2024
5e933c2
Refactor deployment tooling (#85)
alukach Nov 5, 2024
324c3cb
Fix destroy
alukach Nov 5, 2024
0f58c07
Prevent unnecessary deployments
alukach Nov 5, 2024
bf068fc
Pass in PR number
alukach Nov 5, 2024
544387a
Prevent testing on closed PRs
alukach Nov 5, 2024
7d97095
Refactor
alukach Nov 5, 2024
80f788f
Mv to workflows
alukach Nov 5, 2024
ad97522
Fix destroy
alukach Nov 5, 2024
c092e63
Rework trigger
alukach Nov 5, 2024
a2e2096
Refine
alukach Nov 5, 2024
d04b313
Rm old needs
alukach Nov 6, 2024
f50fd0b
Rework trigger
alukach Nov 6, 2024
e4a2a09
Try fix if condition
alukach Nov 7, 2024
ab4bd0d
Mv deployment trigger back to ci.yml
alukach Nov 7, 2024
e15f207
Rework triggers
alukach Nov 7, 2024
090f1da
Rename jobs
alukach Nov 7, 2024
d7d3f4c
Set concurrency to stage
alukach Nov 7, 2024
38c9a95
Only trigger on pushes to main
alukach Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 68 additions & 40 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,47 +1,75 @@
name: Run Tests
name: "Run Tests"

on: [push, pull_request]
on:
push:
branches:
- main
pull_request:
types:
- opened
- synchronize
- reopened
- closed

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11

- name: Install Poetry
run: |
python -m pip install --upgrade pip
python -m pip install poetry

- name: Install dependencies
working-directory: ./space2stats_api/src
run: |
poetry install --with test

- name: install lib postgres
uses: nyurik/action-setup-postgis@v2

- name: Run pre-commit
working-directory: ./space2stats_api/src
run: |
poetry run pre-commit run --all-files

- name: Run tests
working-directory: ./space2stats_api/src
run: |
poetry run python -m pytest --benchmark-skip tests
env:
PGHOST: localhost
PGPORT: 5432
PGDATABASE: mydatabase
PGUSER: myuser
PGPASSWORD: mypassword
PGTABLENAME: space2stats
S3_BUCKET_NAME: test-bucket
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.11

- name: Install Poetry
run: |
python -m pip install --upgrade pip
python -m pip install poetry

- name: Install dependencies
working-directory: ./space2stats_api/src
run: |
poetry install --with test

- name: install lib postgres
uses: nyurik/action-setup-postgis@v2

- name: Run pre-commit
working-directory: ./space2stats_api/src
run: |
poetry run pre-commit run --all-files

- name: Run tests
working-directory: ./space2stats_api/src
run: |
poetry run python -m pytest --benchmark-skip tests
env:
PGHOST: localhost
PGPORT: 5432
PGDATABASE: mydatabase
PGUSER: myuser
PGPASSWORD: mypassword
PGTABLENAME: space2stats
S3_BUCKET_NAME: test-bucket

deploy-to-dev:
if: ${{ github.event_name == 'pull_request' }}
needs: test
uses: "./.github/workflows/reusable-deploy.yml"
with:
environment: Space2Stats API Dev
stage: pr-${{ github.event.pull_request.number }}
pr-number: ${{ github.event.pull_request.number }}
secrets: inherit

deploy-to-production:
if: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
needs: test
uses: "./.github/workflows/reusable-deploy.yml"
with:
environment: Space2Stats API Prod
stage: prod
secrets: inherit
70 changes: 70 additions & 0 deletions .github/workflows/destroy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: Destroy Preview Environment

on:
pull_request:
types:
- closed

permissions:
id-token: write
contents: read
pull-requests: write

jobs:
destroy:
concurrency: Space2Stats API Dev
environment: Space2Stats API Dev
runs-on: ubuntu-latest

steps:
- name: Check out repository code
uses: actions/checkout@v2

- name: Install AWS CDK
run: npm install -g aws-cdk

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::017820688988:role/Space2Stats-Deploy-Role
aws-region: ${{ vars.CDK_DEFAULT_REGION }}

- name: Install CDK dependencies
working-directory: ./space2stats_api/cdk
run: |
pip install -r requirements-cdk.txt

- name: Tear down CDK stack
working-directory: ./space2stats_api/cdk
env:
STAGE: pr-${{ github.event.pull_request.number }}
PGHOST: ${{ secrets.PGHOST }}
PGPORT: ${{ secrets.PGPORT }}
PGDATABASE: ${{ secrets.PGDATABASE }}
PGUSER: ${{ secrets.PGUSER }}
PGPASSWORD: ${{ secrets.PGPASSWORD }}
PGTABLENAME: ${{ secrets.PGTABLENAME }}
CDK_CERTIFICATE_ARN: ${{ vars.CDK_CERTIFICATE_ARN }}
CDK_DEFAULT_ACCOUNT: ${{ vars.CDK_DEFAULT_ACCOUNT }}
CDK_DEFAULT_REGION: ${{ vars.CDK_DEFAULT_REGION }}
CDK_DOMAIN_NAME: ${{ vars.CDK_DOMAIN_NAME }}
run: cdk destroy --require-approval never

- name: Find Comment
uses: peter-evans/find-comment@v3
id: find-comment
if: ${{ github.event.pull_request.number }}
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: "github-actions[bot]"
body-includes: "PR Deployment Details:"

- name: Create or update comment with removal confirmation
uses: peter-evans/create-or-update-comment@v4
if: ${{ github.event.pull_request.number }}
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
body: |
Removed PR Preview Environment.
edit-mode: append
85 changes: 85 additions & 0 deletions .github/workflows/reusable-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: Deploy

on:
workflow_call:
inputs:
environment:
type: string
required: true
stage:
type: string
required: true
pr-number:
type: number
required: false

permissions:
id-token: write
contents: read
pull-requests: write

jobs:
deploy:
concurrency: ${{ inputs.stage }}
environment: ${{ inputs.environment }}
runs-on: ubuntu-latest

steps:
- name: Check out repository code
uses: actions/checkout@v2

- name: Install AWS CDK
run: npm install -g aws-cdk

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::017820688988:role/Space2Stats-Deploy-Role
aws-region: ${{ vars.CDK_DEFAULT_REGION }}

- name: Install CDK dependencies
working-directory: ./space2stats_api/cdk
run: |
pip install -r requirements-cdk.txt
- name: Deploy CDK stack to staging
working-directory: ./space2stats_api/cdk
env:
STAGE: ${{ inputs.stage }}
PGHOST: ${{ secrets.PGHOST }}
PGPORT: ${{ secrets.PGPORT }}
PGDATABASE: ${{ secrets.PGDATABASE }}
PGUSER: ${{ secrets.PGUSER }}
PGPASSWORD: ${{ secrets.PGPASSWORD }}
PGTABLENAME: ${{ secrets.PGTABLENAME }}
CDK_CERTIFICATE_ARN: ${{ vars.CDK_CERTIFICATE_ARN }}
CDK_DEFAULT_ACCOUNT: ${{ vars.CDK_DEFAULT_ACCOUNT }}
CDK_DEFAULT_REGION: ${{ vars.CDK_DEFAULT_REGION }}
CDK_DOMAIN_NAME: ${{ vars.CDK_DOMAIN_NAME }}
run: cdk deploy --require-approval never --outputs-file outputs.json

- name: Get API URL
id: get-api-url
working-directory: ./space2stats_api/cdk
run: |
echo "api-url=$(jq -r '."Space2Stats-${{ inputs.stage }}".ApiGatewayUrl' outputs.json)" >> $GITHUB_OUTPUT
- name: Find Comment
uses: peter-evans/find-comment@v3
id: find-comment
if: ${{ inputs.pr-number }}
with:
issue-number: ${{ inputs.pr-number }}
comment-author: "github-actions[bot]"
body-includes: "PR Deployment Details:"

- name: Create or update comment with URL
uses: peter-evans/create-or-update-comment@v4
if: ${{ inputs.pr-number }}
with:
issue-number: ${{ inputs.pr-number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
body: |
PR Deployment Details:
🚀 PR deployed to ${{ steps.get-api-url.outputs.api-url }}
edit-mode: replace
10 changes: 8 additions & 2 deletions space2stats_api/cdk/app.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
import os

from aws_cdk import App, Environment
from aws_stack import Space2StatsStack
from settings import DeploymentSettings

settings = DeploymentSettings(_env_file="aws_deployment.env")
settings = DeploymentSettings(
_env_file=f"aws_deployment_{os.environ.get('STAGE', 'dev')}.env"
)

env = Environment(
account=settings.CDK_DEFAULT_ACCOUNT, region=settings.CDK_DEFAULT_REGION
)

app = App()

Space2StatsStack(app, "Space2StatsStack", env=env)
Space2StatsStack(
app, f"Space2Stats-{settings.STAGE}", env=env, deployment_settings=settings
)

app.synth()
42 changes: 27 additions & 15 deletions space2stats_api/cdk/aws_stack.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from aws_cdk import Duration, Stack
from aws_cdk import CfnOutput, Duration, Stack
from aws_cdk import aws_apigatewayv2 as apigatewayv2
from aws_cdk import aws_apigatewayv2_integrations as integrations
from aws_cdk import aws_certificatemanager as acm
Expand All @@ -10,11 +10,16 @@


class Space2StatsStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
def __init__(
self,
scope: Construct,
id: str,
deployment_settings: DeploymentSettings,
**kwargs,
) -> None:
super().__init__(scope, id, **kwargs)

app_settings = AppSettings(_env_file="./aws_app.env")
deployment_settings = DeploymentSettings(_env_file="./aws_deployment.env")

bucket = s3.Bucket(
self,
Expand Down Expand Up @@ -43,13 +48,6 @@ def __init__(self, scope: Construct, id: str, **kwargs) -> None:
self, "Certificate", deployment_settings.CDK_CERTIFICATE_ARN
)

domain_name = apigatewayv2.DomainName(
self,
"DomainName",
domain_name=deployment_settings.CDK_DOMAIN_NAME,
certificate=certificate,
)

http_api = apigatewayv2.HttpApi(
self,
"Space2StatsHttpApi",
Expand All @@ -58,10 +56,24 @@ def __init__(self, scope: Construct, id: str, **kwargs) -> None:
),
)

apigatewayv2.ApiMapping(
CfnOutput(
self,
"ApiMapping",
api=http_api,
domain_name=domain_name,
stage=http_api.default_stage,
"ApiGatewayUrl",
key="ApiGatewayUrl",
value=http_api.url,
)

if deployment_settings.CDK_DOMAIN_NAME:
domain_name = apigatewayv2.DomainName(
self,
"DomainName",
domain_name=deployment_settings.CDK_DOMAIN_NAME,
certificate=certificate,
)
apigatewayv2.ApiMapping(
self,
"ApiMapping",
api=http_api,
domain_name=domain_name,
stage=http_api.default_stage,
)
1 change: 1 addition & 0 deletions space2stats_api/cdk/requirements-cdk.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
aws-cdk-lib==2.130.0
aws-cdk.aws-lambda-python-alpha==2.130.0-alpha.0
constructs==10.3.0

pydantic_settings>=2.0
5 changes: 4 additions & 1 deletion space2stats_api/cdk/settings.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from typing import Optional

from pydantic_settings import BaseSettings


Expand All @@ -14,4 +16,5 @@ class DeploymentSettings(BaseSettings):
CDK_DEFAULT_ACCOUNT: str
CDK_DEFAULT_REGION: str
CDK_CERTIFICATE_ARN: str
CDK_DOMAIN_NAME: str
CDK_DOMAIN_NAME: Optional[str]
STAGE: str = "dev"
Loading