Skip to content

Commit

Permalink
Tracking pull request to merge release-2.22.0 to master (#2913)
Browse files Browse the repository at this point in the history
* initial updates 2.20.0

* update pr number

* fix: fix for total previous credit reductions (#2902)

Co-authored-by: Your Name <you@example.com>

* update pip to 24.0 as celery build failed

* update pip to 24.0 as scanhandler failed

* Fix: Pending Deductions Logic for Previously Submitted Reports (#2904)

* fix: fix for total previous credit reductions

* fix: added pending deductions logic for previous submitted reports

---------

Co-authored-by: Your Name <you@example.com>

* update approvers

* initial updates for 2.22.0

* update pr number

* add knp to allow lcfs backend to access tfrs rabbit

* update the diagram to allow lcfs backend to access tfrs rabbit

* udpate rabbitmq dev knp

* update comment in knp

* feat: implement TFRS teardown for BCeID and IDIR users

* add redis settings for backend

* add REDIS arguments

* correct typo

* TFRS - Credit transfer incorrectly categorized in TFRS #2916 (#2917)

* TFRS - Credit transfer incorrectly categorized in TFRS #2916

* .

---------

Co-authored-by: prv-proton <prashanth.venkateshappa@gov.bc.ca>

* Add changes for new pipeline (#2921)

* New pipeline and replace deployment config (#2924)

* New pipeline 2.22.0 (#2925)

* update deployment name

* test deployonly

* open all builds

* combine build

* replace dc with d on Test

* update test pipeline name

* move test-ci-dc-2-d.yaml to release-2.21.0

* add test-ci

* update test-ci suffix validation

* update dev-ci

* update prod release pipeline

* add new prod pipeline

* Feat: TFRS: Add Redis Service for organization balance (#2919)

* Feat: TFRS: Setup Redis Service and Call LCFS Cache for Available Balances

* update docker compose yml

* review comments fix.

* .

* .

---------

Co-authored-by: prv-proton <prashanth.venkateshappa@gov.bc.ca>

* feat: lcfs transaction message on director transaction (#2920)

* fix: Convert Decimal to int

Decimal is not JSON serializable to fails to send message, convert to int before sending

* use artifactory pull secret

* use artifactory pull secret

* update artifactory pull secret

* feat: Sync Reports TFRS -> LCFS

* Change logic to sync report with status instead of just transaction
* Call when report is created, submitted, and approved
* Add more fields to message

* install oc command

* add install-oc.sh

* add install-oc.sh

* remove install-oc

* update test-ci

* update pipeline title

* update approval title

* add approval step

* restore oc command from cache

* update issue title

* move approval steps

* update dev and test ci

* update dev-ci

* reorg test-ci

* uncomment deployment

* reorg pro-ci

* reorg pro-ci

* test pro-ci after ubuntu image change

* open the prod deployment

* set approval to 2

* feat: supplemental variables (#2945)

* feat: enable tear-down feature flags for dev testing (#2946)

* update frontend node to 20.18.1

* fix: resolve issue with tear_down.idir.widgets.balance flag (#2948)

---------

Co-authored-by: Alex Zorkin <47334977+AlexZorkin@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Hamed Valiollahi Bayeki <hamedv90@gmail.com>
Co-authored-by: Prashanth <130073308+prv-proton@users.noreply.github.com>
Co-authored-by: prv-proton <prashanth.venkateshappa@gov.bc.ca>
Co-authored-by: Daniel Haselhan <daniel.haselhan@quartech.com>
Co-authored-by: Daniel Haselhan <dhaselhan@gmail.com>
  • Loading branch information
8 people authored Jan 29, 2025
1 parent f1432f9 commit f96317a
Show file tree
Hide file tree
Showing 52 changed files with 2,435 additions and 967 deletions.
398 changes: 398 additions & 0 deletions .github/workflows/dev-ci.yaml

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions .github/workflows/dev-release.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
## For each release, the value of name, branches, RELEASE_NAME and PR_NUMBER need to be adjusted accordingly
## For each release, update lib/config.js: version and releaseBranch

name: TFRS Dev release-2.21.0
name: TFRS Dev release-2.22.0

on:
push:
branches: [release-2.21.0]
branches: [release-2.22.0]
paths:
- frontend/**
- backend/**
Expand All @@ -15,8 +15,8 @@ on:
env:
## The pull request number of the Tracking pull request to merge the release branch to main
## Also remember to update the version in .pipeline/lib/config.js
PR_NUMBER: 2908
RELEASE_NAME: release-2.21.0
PR_NUMBER: 2913
RELEASE_NAME: release-2.22.0

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down
164 changes: 164 additions & 0 deletions .github/workflows/prod-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
name: TFRS New Pipeline Prod release-2.22.0

on:
workflow_dispatch:

env:
GIT_URL: https://github.com/bcgov/tfrs.git
TEST_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-test
PROD_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-prod

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:

install-oc:
runs-on: ubuntu-latest
outputs:
cache-hit: ${{ steps.cache.outputs.cache-hit }}
steps:
- name: Set up cache for OpenShift CLI
id: cache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc # Path where the `oc` binary will be installed
key: oc-cli-${{ runner.os }}

- name: Install OpenShift CLI (if not cached)
if: steps.cache.outputs.cache-hit != 'true'
run: |
curl -LO https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-linux.tar.gz
tar -xvf openshift-client-linux.tar.gz
sudo mv oc /usr/local/bin/
oc version --client
- name: Confirm OpenShift CLI is Available
run: oc version --client

get-build-suffix:
name: Find Test deployment build suffix
runs-on: ubuntu-latest
needs: [install-oc]

outputs:
BUILD_SUFFIX: ${{ steps.get-build-suffix.outputs.BUILD_SUFFIX }}

steps:
- name: Restore oc command from Cache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc
key: oc-cli-${{ runner.os }}

- name: Log in to Openshift
uses: redhat-actions/oc-login@v1.3
with:
openshift_server_url: ${{ secrets.OPENSHIFT_SERVER }}
openshift_token: ${{ secrets.OPENSHIFT_TOKEN }}
insecure_skip_tls_verify: true
namespace: ${{ env.TEST_NAMESPACE }}

- id: get-build-suffix
run: |
build_suffix=$(oc -n ${{ env.TEST_NAMESPACE }} describe deployment/tfrs-frontend-test | grep Image | awk -F ':' '{print $NF}')
echo "Validating $build_suffix"
if [[ "$build_suffix" =~ ^[0-9]+\.[0-9]+\.[0-9]$ ]]; then
echo "Build suffix $build_suffix format is valid."
echo "BUILD_SUFFIX=$build_suffix" >> $GITHUB_OUTPUT
else
echo "Error: Build suffix $build_suffix format is invalid!"
exit 1
fi
deploy:
name: Deploy on Prod
runs-on: ubuntu-latest
timeout-minutes: 60
needs: [get-build-suffix]

env:
BUILD_SUFFIX: ${{ needs.get-build-suffix.outputs.BUILD_SUFFIX }}

steps:
- id: get-current-time
run: |
echo "CURRENT_TIME=$(TZ='America/Vancouver' date '+%Y-%m-%d %H:%M:%S %Z')" >> $GITHUB_OUTPUT
- name: Ask for approval for TFRS Prod deployment
uses: trstringer/manual-approval@v1.6.0
with:
secret: ${{ github.TOKEN }}
approvers: AlexZorkin,kuanfandevops,prv-proton,JulianForeman,kevin-hashimoto,dhaselhan
minimum-approvals: 2
issue-title: "TFRS ${{ env.BUILD_SUFFIX }} Prod Deployment at ${{ steps.get-current-time.outputs.CURRENT_TIME }}"

- name: Checkout Manifest repository
uses: actions/checkout@v4.1.1
with:
repository: bcgov-c/tenant-gitops-0ab226
ref: main
ssh-key: ${{ secrets.MANIFEST_REPO_DEPLOY_KEY }}

- name: Update tags
uses: mikefarah/yq@v4.40.5
with:
cmd: |
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/backend/values-prod.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/frontend/values-prod.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/notification-server/values-prod.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/celery/values-prod.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/scan-coordinator/values-prod.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/scan-handler/values-prod.yaml
- name: GitHub Commit & Push
shell: bash {0}
run: |
git config --global user.email "actions@github.com"
git config --global user.name "GitHub Actions"
git add -A
git commit -m "Update the image tag to ${{ env.BUILD_SUFFIX }} on Prod"
git push
- name: Restore oc command from Cache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc
key: oc-cli-${{ runner.os }}

- name: Log in to Openshift
uses: redhat-actions/oc-login@v1.3
with:
openshift_server_url: ${{ secrets.OPENSHIFT_SERVER }}
openshift_token: ${{ secrets.OPENSHIFT_TOKEN }}
insecure_skip_tls_verify: true
namespace: ${{ env.TEST_NAMESPACE }}

- name: Tag and deploy to Prod
run: |
helm -n ${{ env.PROD_NAMESPACE }} list
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-backend:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-backend:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-frontend:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-frontend:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-celery:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-celery:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-scan-coordinator:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-scan-coordinator:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-scan-handler:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-scan-handler:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.TEST_NAMESPACE }}/tfrs-notification-server:${{ env.BUILD_SUFFIX }} ${{ env.PROD_NAMESPACE }}/tfrs-notification-server:${{ env.BUILD_SUFFIX }}
cd tfrs/charts/backend
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-backend-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../frontend
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-frontend-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../notification-server
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-notification-server-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../celery
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-celery-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../scan-coordinator
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-scan-coordinator-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../scan-handler
helm -n ${{ env.PROD_NAMESPACE }} -f ./values-prod.yaml upgrade --install tfrs-scan-handler-prod . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
168 changes: 168 additions & 0 deletions .github/workflows/test-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
name: TFRS New Pipeline Test release-2.22.0

on:
workflow_dispatch:

env:
GIT_URL: https://github.com/bcgov/tfrs.git
DEV_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-dev
TEST_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-test

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:

install-oc:
runs-on: ubuntu-latest
outputs:
cache-hit: ${{ steps.cache.outputs.cache-hit }}
steps:

- name: Set up cache for OpenShift CLI
id: cache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc # Path where the `oc` binary will be installed
key: oc-cli-${{ runner.os }}

- name: Install OpenShift CLI (if not cached)
if: steps.cache.outputs.cache-hit != 'true'
run: |
curl -LO https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-linux.tar.gz
tar -xvf openshift-client-linux.tar.gz
sudo mv oc /usr/local/bin/
oc version --client
- name: Confirm OpenShift CLI is Available
run: oc version --client

get-build-suffix:
name: Find Dev deployment build suffix
needs: install-oc
runs-on: ubuntu-latest

outputs:
BUILD_SUFFIX: ${{ steps.get-build-suffix.outputs.BUILD_SUFFIX }}

steps:
- name: Check out repository
uses: actions/checkout@v4.1.1

- name: Restore oc command from Cßache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc
key: oc-cli-${{ runner.os }}

- name: Log in to Openshift
uses: redhat-actions/oc-login@v1.3
with:
openshift_server_url: ${{ secrets.OPENSHIFT_SERVER }}
openshift_token: ${{ secrets.OPENSHIFT_TOKEN }}
insecure_skip_tls_verify: true
namespace: ${{ env.DEV_NAMESPACE }}

- id: get-build-suffix
run: |
build_suffix=$(oc -n ${{ env.DEV_NAMESPACE }} describe deployment/tfrs-frontend-dev | grep Image | awk -F ':' '{print $NF}')
echo "Validating $build_suffix"
if [[ "$build_suffix" =~ ^[0-9]+\.[0-9]+\.[0-9]$ ]]; then
echo "Build suffix $build_suffix format is valid."
echo "BUILD_SUFFIX=$build_suffix" >> $GITHUB_OUTPUT
else
echo "Error: Build suffix $build_suffix format is invalid!"
exit 1
fi
deploy:
name: Deploy on Test
runs-on: ubuntu-latest
timeout-minutes: 60
needs: [get-build-suffix]

env:
BUILD_SUFFIX: ${{ needs.get-build-suffix.outputs.BUILD_SUFFIX }}

steps:
- id: get-current-time
run: |
echo "CURRENT_TIME=$(TZ="America/Vancouver" date '+%Y-%m-%d %H:%M:%S %Z')" >> $GITHUB_OUTPUT
- name: Ask for approval for TFRS Test deployment
uses: trstringer/manual-approval@v1.6.0
with:
secret: ${{ github.TOKEN }}
approvers: AlexZorkin,kuanfandevops,prv-proton,JulianForeman,kevin-hashimoto,dhaselhan
minimum-approvals: 1
issue-title: "TFRS ${{ env.BUILD_SUFFIX }} Test Deployment at ${{ steps.get-current-time.outputs.CURRENT_TIME }}"

- name: Checkout Manifest repository
uses: actions/checkout@v4.1.1
with:
repository: bcgov-c/tenant-gitops-0ab226
ref: main
ssh-key: ${{ secrets.MANIFEST_REPO_DEPLOY_KEY }}

- name: Update tags
uses: mikefarah/yq@v4.40.5
with:
cmd: |
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/backend/values-test.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/frontend/values-test.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/notification-server/values-test.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/celery/values-test.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/scan-coordinator/values-test.yaml
yq -i '.image.tag = "${{ env.BUILD_SUFFIX }}"' tfrs/charts/scan-handler/values-test.yaml
- name: GitHub Commit & Push
shell: bash {0}
run: |
git config --global user.email "actions@github.com"
git config --global user.name "GitHub Actions"
git add -A
git commit -m "Update the image tag to ${{ env.BUILD_SUFFIX }} on Test"
git push
- name: Restore oc command from Cache
uses: actions/cache@v4.2.0
with:
path: /usr/local/bin/oc
key: oc-cli-${{ runner.os }}

- name: Log in to Openshift
uses: redhat-actions/oc-login@v1.3
with:
openshift_server_url: ${{ secrets.OPENSHIFT_SERVER }}
openshift_token: ${{ secrets.OPENSHIFT_TOKEN }}
insecure_skip_tls_verify: true
namespace: ${{ env.TOOLS_NAMESPACE }}

- name: Tag and deploy to Test
run: |
helm -n ${{ env.TEST_NAMESPACE }} list
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-backend:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-backend:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-frontend:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-frontend:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-celery:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-celery:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-scan-coordinator:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-scan-coordinator:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-scan-handler:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-scan-handler:${{ env.BUILD_SUFFIX }}
oc tag ${{ env.DEV_NAMESPACE }}/tfrs-notification-server:${{ env.BUILD_SUFFIX }} ${{ env.TEST_NAMESPACE }}/tfrs-notification-server:${{ env.BUILD_SUFFIX }}
cd tfrs/charts/backend
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-backend-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../frontend
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-frontend-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../notification-server
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-notification-server-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../celery
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-celery-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../scan-coordinator
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-scan-coordinator-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
cd ../scan-handler
helm -n ${{ env.TEST_NAMESPACE }} -f ./values-test.yaml upgrade --install tfrs-scan-handler-test . \
--set podAnnotations.rolloutTriggered="A$(date +%s)E"
Loading

0 comments on commit f96317a

Please sign in to comment.