EREGCSC-2859 Migrate the site lambdas and database to CDK (#1522)

* EREGS-2858: Convert Static-Assets serverless to CDK * add api stack * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * EREGS-2859: Add API stack CDK resources * Add Dockerfile * Add Dockerfile * Add SSm permission * Add SSm permission * Add SSm permission * Add SSm permission * Add SSm permission * Add SSm permission * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * Add secrets manager for secret * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * only prepend stage name to url on amazonaws.com domains * print out the lambda response * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * add static asset build * in VITE_ENV, replace dev with eph- * remove api_gateway_base_path param from Mangum config * Create a re-implementation of serverless-wsgi's lambda handler * linting and file description for handler.py * update requirements.txt: remove mangum, upgrade Werkzeug * properly parse hostname to avoid CodeQL alert * add database configs * add dev SG to Ephemeral env api stack * revert wsgi.py and remove unused mangum dependency * add dev SG to Ephemeral env api stack * add dev SG to Ephemeral env api stack * Add DEV CDK workflow yaml and update database constructor * Add SG export for ephmeral env to access database * Add SG export for ephmeral env to access database and deploy backend stack * Add SG export for ephmeral env to access database and deploy backend stack * Use ssm instead of exports * rename django_app to application to fix serverless deploy * Use ssm instead of exports * Use ssm instead of exports * Use ssm instead of exports * Clean up uneeded files * update postgresql to use port 3306 * Show notification on successful CDK deploy * Attempt 2 for notify on successful CDK deploy * Use the correct apiendpoint url * Use the correct apiendpoint url * Use the correct apiendpoint url * Use the correct apiendpoint url * Use the correct apiendpoint url * Use the correct apiendpoint url * add debug * add debug * output api url for notify step --------- Co-authored-by: Caleb Godwin <[email protected]>
Enterprise-CMCS · Feb 6, 2025 · fbf81f6 · fbf81f6
1 parent 80c9d5d
commit fbf81f6
Show file tree

Hide file tree

Showing 34 changed files with 3,577 additions and 1,394 deletions.
diff --git a/.github/workflows/deploy-cdk-dev.yml b/.github/workflows/deploy-cdk-dev.yml
@@ -13,7 +13,59 @@ permissions:
 
 concurrency: ${{ github.workflow }}-${{ github.ref }}
 
-jobs:
+jobs:  
+  deploy-static-asset-cdk:    
+    strategy:      
+      max-parallel: 1      
+      matrix:        
+        environment: ["dev"]    
+    runs-on: ubuntu-22.04    
+    environment:      
+      name: ${{ matrix.environment }}
+    outputs:
+      static_url: ${{ steps.get-static-url.outputs.static_url }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          submodules: true     
+
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_TO_ASSUME }}
+          aws-region: us-east-1
+
+      - name: Deploy Static-Asset Infrastructure
+        env:
+          AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
+          AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
+          CDK_DEBUG: true
+        run: |
+          pushd cdk-eregs
+          npm install -g aws-cdk@latest @aws-sdk/client-ssm
+          npm install
+          
+          # Get exact stack name
+          STATICASSET_STACK="cms-eregs-${{ matrix.environment }}-static-assets"
+          
+          cdk deploy ${STATICASSET_STACK} \
+          -c environment=${{ matrix.environment }} \
+          -c deploymentType=infrastructure \
+          --require-approval never \
+          --exclusively \
+          --app "npx ts-node bin/static-assets.ts" \
+          --outputs-file static-outputs.json
+          popd
+
+      - name: Get Static URL
+        id: get-static-url
+        run: |
+          pushd cdk-eregs
+          STATIC_STACK="cms-eregs-${{ matrix.environment }}-static-assets"
+          STATIC_URL=$(cat static-outputs.json | jq -r ".[\"$STATIC_STACK\"].StaticURL")
+          echo "static_url=${STATIC_URL}" >> $GITHUB_OUTPUT
+          popd
   deploy-zip-lambdas-cdk:    
     strategy:
       max-parallel: 1
@@ -92,7 +144,57 @@ jobs:
           --exclusively \
           --app "npx ts-node bin/docker-lambdas.ts"
           popd
+  deploy-site-lambda-cdk:
+    needs: [deploy-static-asset-cdk, deploy-zip-lambdas-cdk, deploy-text-extractor-cdk]
+    strategy:      
+      max-parallel: 1      
+      matrix:        
+        environment: ["dev"]    
+    runs-on: ubuntu-22.04    
+    environment:      
+      name: ${{ matrix.environment }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          submodules: true     
+
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_TO_ASSUME }}
+          aws-region: us-east-1
+
+      - name: Deploy Site-Lambda
+        env:
+          AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
+          AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
+          CDK_DEBUG: true
+        run: |
+          pushd cdk-eregs
+          npm install -g aws-cdk@latest @aws-sdk/client-ssm
+          npm install
+          
+          # Get exact stack name
+          API_STACK="cms-eregs-${{ matrix.environment }}-api"
+          
+          cdk deploy $API_STACK \
+          -c environment=${{ matrix.environment }} \
+          -c buildId="${GITHUB_RUN_ID}" \
+          --require-approval never \
+          --exclusively \
+          --app "npx ts-node bin/docker-lambdas.ts" \
+          --outputs-file api-outputs.json
+          popd
 
+      - name: Get API URL
+        id: get-api-url
+        run: |
+          pushd cdk-eregs
+          API_STACK="cms-eregs-${{ matrix.environment }}-api"
+          API_URL=$(cat api-outputs.json | jq -r ".[\"$API_STACK\"].ApiUrl")
+          echo "api_url=${API_URL}" >> $GITHUB_OUTPUT
+          popd
   deploy-fr-parser-cdk:
     needs: [deploy-zip-lambdas-cdk, deploy-text-extractor-cdk]
     strategy:
@@ -170,4 +272,89 @@ jobs:
           --require-approval never \
           --exclusively \
           --app "npx ts-node bin/docker-lambdas.ts"
+          popd
+  build-and-deploy-vue-cdk:
+    needs: [deploy-site-lambda-cdk, deploy-static-asset-cdk]
+    strategy:      
+      max-parallel: 1      
+      matrix:        
+        environment: ["dev"]    
+    runs-on: ubuntu-22.04    
+    environment:      
+      name: ${{ matrix.environment }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          submodules: true
+
+      - name: Setup Node
+        uses: actions/setup-node@v3
+        with:
+          node-version: 18.14
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install Python dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r ./solution/static-assets/requirements.txt
+          pip install -r ./solution/backend/requirements.txt
+
+      - name: Make envfile
+        uses: SpicyPizza/[email protected]
+        with:
+          envkey_VITE_API_URL: ${{ needs.deploy-django.outputs.url }}
+          directory: solution/ui/regulations/eregs-vite
+          file_name: .env
+
+      - name: Run collectstatic
+        env:
+          STATIC_URL: ${{ needs.deploy-static-assets-cdk.outputs.static_url }}
+          STATIC_ROOT: ../static-assets/regulations
+          VITE_ENV: ${{ matrix.environment }}
+        run: |
+          pushd solution/backend
+          python manage.py collectstatic --noinput
+          cd ..
+          popd
+
+      - name: Build Vite assets
+        env:
+          STATIC_URL: ${{ needs.deploy-static-assets-cdk.outputs.static_url }}
+          STATIC_ROOT: ../static-assets/regulations
+          VITE_ENV: ${{ matrix.environment }}
+        run: |
+          pushd solution
+          make regulations
+          popd
+
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_TO_ASSUME }}
+          aws-region: us-east-1
+
+      - name: Deploy Static Assets
+        env:
+          AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
+          AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
+          CDK_DEBUG: true
+        run: |
+          pushd cdk-eregs
+          npm install -g aws-cdk@latest @aws-sdk/client-ssm
+          npm install
+          
+          # Get exact stack name
+          STATICASSET_STACK="cms-eregs-${{ matrix.environment }}-static-assets"
+          
+          cdk deploy ${STATICASSET_STACK} \
+          -c environment=${{ matrix.environment }} \
+          -c deploymentType=content \
+          --require-approval never \
+          --exclusively \
+          --app "npx ts-node bin/static-assets.ts" \
+          --outputs-file static-outputs.json
           popd