Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using .env files for centralizing github enviroment variables #1537

Closed
wants to merge 3 commits into from
+76 −32
Closed

Using .env files for centralizing github enviroment variables #1537

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3437839
testing .env
otherview Sep 20, 2023
3a02908
testing
otherview Sep 20, 2023
eddea3e
testing
otherview Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/envs/dev-testnet.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
L2_ENCLAVE_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/enclave:latest
L2_HOST_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/host:latest
L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/hardhatdeployer:latest
RESOURCE_TAG_NAME=testnetlatest
RESOURCE_STARTING_NAME=T
RESOURCE_TESTNET_NAME=testnet
L1_WS_URL=ws://testnet-eth2network.uksouth.cloudapp.azure.com:9000
L1_HTTP_URL=http://testnet-eth2network.uksouth.cloudapp.azure.com:8025
BATCH_INTERVAL=1s
ROLLUP_INTERVAL=10s
10 changes: 10 additions & 0 deletions .github/workflows/envs/sepolia-testnet.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
L2_ENCLAVE_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/sepolia_enclave:latest
L2_HOST_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/sepolia_host:latest
L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/sepolia_hardhatdeployer:latest
RESOURCE_TAG_NAME=sepoliatestnetlatest
RESOURCE_STARTING_NAME=S
RESOURCE_TESTNET_NAME=sepoliatestnet
BATCH_INTERVAL=1s
ROLLUP_INTERVAL=3m
NODE_WALLET_ADDR_0=0x2fe9B92E12a8d94bfb2f19c19024B9554890C0CC
NODE_WALLET_ADDR_1=0xBD0D613bCbDbcC93abE025117564cc4435896A5F
11 changes: 11 additions & 0 deletions .github/workflows/envs/testnet.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
L2_ENCLAVE_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/enclave:latest
L2_HOST_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/host:latest
L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/hardhatdeployer:latest
RESOURCE_TAG_NAME=testnetlatest
RESOURCE_STARTING_NAME=T
RESOURCE_TESTNET_NAME=testnet
L1_WS_URL=ws://testnet-eth2network.uksouth.cloudapp.azure.com:9000
L1_HTTP_URL=http://testnet-eth2network.uksouth.cloudapp.azure.com:8025
BATCH_INTERVAL=1s
ROLLUP_INTERVAL=10s
NODE_WALLET_ADDR_0=
77 changes: 45 additions & 32 deletions .github/workflows/manual-deploy-testnet-l2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ on:
jobs:
build:
runs-on: ubuntu-latest
environment:
name: ${{ github.event.inputs.testnet_type }}

# Map a step output to a job output
outputs:
Expand All @@ -42,6 +40,15 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Load dotenv
id: dotenv
uses: falti/[email protected]
with:
path: .github/workflows/envs/${{ github.event.inputs.testnet_type }}.env
log-variables: true
export-variables: true
keys-case: bypass

- uses: actions/setup-go@v4
with:
go-version: 1.20.4
Expand All @@ -60,12 +67,12 @@ jobs:

- name: 'Build and push obscuro node images'
run: |
DOCKER_BUILDKIT=1 docker build -t ${{vars.L2_ENCLAVE_DOCKER_BUILD_TAG}} -f dockerfiles/enclave.Dockerfile .
docker push ${{vars.L2_ENCLAVE_DOCKER_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{vars.L2_HOST_DOCKER_BUILD_TAG}} -f dockerfiles/host.Dockerfile .
docker push ${{vars.L2_HOST_DOCKER_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{vars.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}} -f tools/hardhatdeployer/Dockerfile .
docker push ${{vars.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{steps.dotenv.outputs.L2_ENCLAVE_DOCKER_BUILD_TAG}} -f dockerfiles/enclave.Dockerfile .
docker push ${{steps.dotenv.outputs.L2_ENCLAVE_DOCKER_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{steps.dotenv.outputs.L2_HOST_DOCKER_BUILD_TAG}} -f dockerfiles/host.Dockerfile .
docker push ${{steps.dotenv.outputs.L2_HOST_DOCKER_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{steps.dotenv.outputs.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}} -f tools/hardhatdeployer/Dockerfile .
docker push ${{steps.dotenv.outputs.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}}

- name: 'Deploy Contracts'
id: deployContracts
Expand All @@ -74,7 +81,7 @@ jobs:
go run ./testnet/launcher/l1contractdeployer/cmd \
-l1_http_url=${{ secrets.L1_HTTP_URL }} \
-private_key=${{ secrets.WORKER_PK }} \
-docker_image=${{vars.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}} \
-docker_image=${{steps.dotenv.outputs.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}} \
-contracts_env_file=./testnet/.env
source ./testnet/.env
echo "Contracts deployed to $MGMTCONTRACTADDR"
Expand All @@ -97,14 +104,14 @@ jobs:
uses: azure/CLI@v1
with:
inlineScript: |
$(az resource list --tag ${{vars.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true
$(az resource list --tag ${{steps.dotenv.outputs.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true

# This will clean up any lingering dependencies - might fail if there are no resources to cleanup
- name: 'Delete VMs dependencies'
uses: azure/CLI@v1
with:
inlineScript: |
$(az resource list --tag ${{vars.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true
$(az resource list --tag ${{steps.dotenv.outputs.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true

- name: 'Upload container logs on failure'
uses: actions/upload-artifact@v3
Expand Down Expand Up @@ -150,6 +157,12 @@ jobs:
host_id: 1

steps:
- name: 'Test_ENV_2'
id: Test_ENV_2
shell: bash
run: |
echo "Loaded environment variables -> ${{ steps.dotenv.outputs.RESOURCE_TESTNET_NAME }}"

- name: 'Extract branch name'
shell: bash
run: |
Expand All @@ -171,19 +184,19 @@ jobs:
uses: azure/CLI@v1
with:
inlineScript: |
az vm create -g Testnet -n "${{vars.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" \
az vm create -g Testnet -n "${{steps.dotenv.outputs.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" \
--admin-username obscurouser --admin-password "${{ secrets.OBSCURO_NODE_VM_PWD }}" \
--public-ip-address-dns-name "obscuronode-${{ matrix.host_id }}-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}" \
--tags deploygroup=ObscuroNode-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }} ${{vars.RESOURCE_TAG_NAME}}=true \
--vnet-name ObscuroHost-${{vars.RESOURCE_TESTNET_NAME}}-01VNET --subnet ObscuroHost-${{vars.RESOURCE_TESTNET_NAME}}-01Subnet \
--public-ip-address-dns-name "obscuronode-${{ matrix.host_id }}-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}" \
--tags deploygroup=ObscuroNode-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }} ${{steps.dotenv.outputs.RESOURCE_TAG_NAME}}=true \
--vnet-name ObscuroHost-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-01VNET --subnet ObscuroHost-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-01Subnet \
--size Standard_DC4s_v2 --image ObscuroConfUbuntu \
--public-ip-sku Basic --authentication-type password

- name: 'Open Obscuro node-${{ matrix.host_id }} ports on Azure'
uses: azure/CLI@v1
with:
inlineScript: |
az vm open-port -g Testnet -n "${{vars.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" --port 80,81,6060,6061,10000
az vm open-port -g Testnet -n "${{steps.dotenv.outputs.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" --port 80,81,6060,6061,10000

# To overcome issues with critical VM resources being unavailable, we need to wait for the VM to be ready
- name: 'Allow time for VM initialization'
Expand All @@ -194,7 +207,7 @@ jobs:
uses: azure/CLI@v1
with:
inlineScript: |
az vm run-command invoke -g Testnet -n "${{vars.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" \
az vm run-command invoke -g Testnet -n "${{steps.dotenv.outputs.RESOURCE_STARTING_NAME}}-${{ matrix.host_id }}-${{ GITHUB.RUN_NUMBER }}" \
--command-id RunShellScript \
--scripts 'mkdir -p /home/obscuro \
&& git clone --depth 1 -b ${{ env.BRANCH_NAME }} https://github.com/obscuronet/go-obscuro.git /home/obscuro/go-obscuro \
Expand Down Expand Up @@ -223,15 +236,15 @@ jobs:
-message_bus_contract_addr=${{needs.build.outputs.MSG_BUS_CONTRACT_ADDR}} \
-l1_start=${{needs.build.outputs.L1_START_HASH}} \
-private_key=${{ secrets[matrix.node_pk_lookup] }} \
-sequencer_id=${{ vars.NODE_WALLET_ADDR_0 }} \
-host_public_p2p_addr=obscuronode-${{ matrix.host_id }}-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com:10000 \
-sequencer_id=${{ steps.dotenv.outputs.NODE_WALLET_ADDR_0 }} \
-host_public_p2p_addr=obscuronode-${{ matrix.host_id }}-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com:10000 \
-host_p2p_port=10000 \
-enclave_docker_image=${{vars.L2_ENCLAVE_DOCKER_BUILD_TAG}} \
-host_docker_image=${{vars.L2_HOST_DOCKER_BUILD_TAG}} \
-enclave_docker_image=${{steps.dotenv.outputs.L2_ENCLAVE_DOCKER_BUILD_TAG}} \
-host_docker_image=${{steps.dotenv.outputs.L2_HOST_DOCKER_BUILD_TAG}} \
-is_debug_namespace_enabled=true \
-log_level=${{ github.event.inputs.log_level }} \
-batch_interval=${{vars.BATCH_INTERVAL}} \
-rollup_interval=${{vars.ROLLUP_INTERVAL}} \
-batch_interval=${{steps.dotenv.outputs.BATCH_INTERVAL}} \
-rollup_interval=${{steps.dotenv.outputs.ROLLUP_INTERVAL}} \
start'


Expand All @@ -251,18 +264,18 @@ jobs:
creds: ${{ secrets.AZURE_CREDENTIALS }}

- name: 'Remove existing backend nodes from the load balancer'
run: ./.github/workflows/runner-scripts/testnet-clear-loadbalancer.sh ${{vars.RESOURCE_TESTNET_NAME}}
run: ./.github/workflows/runner-scripts/testnet-clear-loadbalancer.sh ${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}

- name: 'Add load balancer address pool to the IP configuration'
uses: azure/CLI@v1
with:
inlineScript: |
az network nic ip-config address-pool add \
--address-pool Backend-Pool-Obscuro-${{vars.RESOURCE_TESTNET_NAME}} \
--ip-config-name ipconfig${{vars.RESOURCE_STARTING_NAME}}-1-${{ GITHUB.RUN_NUMBER }} \
--nic-name ${{vars.RESOURCE_STARTING_NAME}}-1-${{ GITHUB.RUN_NUMBER }}VMNic \
--address-pool Backend-Pool-Obscuro-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}} \
--ip-config-name ipconfig${{steps.dotenv.outputs.RESOURCE_STARTING_NAME}}-1-${{ GITHUB.RUN_NUMBER }} \
--nic-name ${{steps.dotenv.outputs.RESOURCE_STARTING_NAME}}-1-${{ GITHUB.RUN_NUMBER }}VMNic \
--resource-group Testnet \
--lb-name ${{vars.RESOURCE_TESTNET_NAME}}-loadbalancer
--lb-name ${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-loadbalancer

check-obscuro-is-healthy:
needs:
Expand All @@ -277,8 +290,8 @@ jobs:
- name: "Wait until obscuro node is healthy"
shell: bash
run: |
./.github/workflows/runner-scripts/wait-node-healthy.sh --host=obscuronode-0-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com
./.github/workflows/runner-scripts/wait-node-healthy.sh --host=obscuronode-1-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com
./.github/workflows/runner-scripts/wait-node-healthy.sh --host=obscuronode-0-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com
./.github/workflows/runner-scripts/wait-node-healthy.sh --host=obscuronode-1-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com

deploy-l2-contracts:
needs:
Expand All @@ -295,15 +308,15 @@ jobs:
shell: bash
run: |
go run ./testnet/launcher/l2contractdeployer/cmd \
-l2_host=obscuronode-0-${{vars.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com \
-l2_host=obscuronode-0-${{steps.dotenv.outputs.RESOURCE_TESTNET_NAME}}-${{ GITHUB.RUN_NUMBER }}.uksouth.cloudapp.azure.com \
-l1_http_url=${{ secrets.L1_HTTP_URL }} \
-l2_ws_port=81 \
-private_key=${{ secrets.WORKER_PK }} \
-l2_private_key=8dfb8083da6275ae3e4f41e3e8a8c19d028d32c9247e24530933782f2a05035b \
-l2_hoc_private_key=6e384a07a01263518a09a5424c7b6bbfc3604ba7d93f47e3a455cbdd7f9f0682 \
-l2_poc_private_key=4bfe14725e685901c062ccd4e220c61cf9c189897b6c78bd18d7f51291b2b8f8 \
-message_bus_contract_addr=${{ needs.build.outputs.MSG_BUS_CONTRACT_ADDR }} \
-docker_image=${{vars.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}}
-docker_image=${{steps.dotenv.outputs.L2_HARDHATDEPLOYER_DOCKER_BUILD_TAG}}

- name: 'Save container logs on failure'
if: failure()
Expand Down