Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update deployment scripts to use standardised env vars #1608

Merged
merged 10 commits into from
Oct 20, 2023
Merged

Update deployment scripts to use standardised env vars #1608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c8e9b03
add sender script
BedrockSquirrel Oct 4, 2023
f66a5f7
WIP updated scripts with new var names
BedrockSquirrel Oct 9, 2023
5419c52
WIP
BedrockSquirrel Oct 16, 2023
c868b7a
update deployment scripts to use standardised var names
BedrockSquirrel Oct 17, 2023
6ba4fd5
removed unintended commit
BedrockSquirrel Oct 17, 2023
5dd3b15
standardise whitespace
BedrockSquirrel Oct 17, 2023
bbdce17
dump all vars at script start
BedrockSquirrel Oct 18, 2023
715cdb5
review comments - renaming vars
BedrockSquirrel Oct 18, 2023
4663c7d
revert duplicated line
BedrockSquirrel Oct 19, 2023
1be6308
Merge remote-tracking branch 'origin/main' into matt/update-l1-deploy…
BedrockSquirrel Oct 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 11 additions & 25 deletions .github/workflows/manual-deploy-testnet-faucet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,14 @@ on:
jobs:
build-and-deploy:
runs-on: ubuntu-latest
environment:
name: ${{ github.event.inputs.testnet_type }}
steps:
- name: 'Print GitHub variables'
# This is a useful record of what the environment variables were at the time the job ran, for debugging and reference
run: |
echo "GitHub Variables = ${{ toJSON(vars) }}"

- run: echo "Workflow_dispatch inputs ${{ github.event.inputs.testnet_type }}"
- run: echo "Workflow_call inputs ${{ inputs.testnet_type }}"

Comment on lines 32 to 45
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new hunk introduces an environment variable based on the testnet_type input and a step to print GitHub variables for debugging. However, the vars object used in line 41 is not defined anywhere. This will cause an error when the workflow runs. You should replace vars with github to print all GitHub context variables.

- echo "GitHub Variables = ${{ toJSON(vars) }}"
+ echo "GitHub Variables = ${{ toJSON(github) }}"

Expand All @@ -41,27 +48,6 @@ jobs:
- name: 'Set up Docker'
uses: docker/setup-buildx-action@v1

- name: 'Sets env vars for testnet'
if: ${{ inputs.testnet_type == 'uat-testnet' }}
run: |
echo "FAUCET_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/faucet_uat_testnet:latest" >> $GITHUB_ENV
echo "TESTNET_ADDR=erpc.uat-testnet.obscu.ro" >> $GITHUB_ENV
echo "DEFAULT_FAUCET_AMOUNT=100" >> $GITHUB_ENV

- name: 'Sets env vars for dev-testnet'
if: ${{ inputs.testnet_type == 'dev-testnet' }}
run: |
echo "FAUCET_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/faucet_dev_testnet:latest" >> $GITHUB_ENV
echo "TESTNET_ADDR=erpc.dev-testnet.obscu.ro" >> $GITHUB_ENV
echo "DEFAULT_FAUCET_AMOUNT=100" >> $GITHUB_ENV

- name: 'Sets env vars for sepolia-testnet'
if: ${{ inputs.testnet_type == 'sepolia-testnet' }}
run: |
echo "FAUCET_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/faucet_sepolia_testnet:latest" >> $GITHUB_ENV
echo "TESTNET_ADDR=erpc.sepolia-testnet.obscu.ro" >> $GITHUB_ENV
echo "DEFAULT_FAUCET_AMOUNT=0.5" >> $GITHUB_ENV

- name: 'Login to Azure docker registry'
uses: azure/docker-login@v1
with:
Expand All @@ -76,20 +62,20 @@ jobs:

- name: Build and Push Docker Image
run: |
DOCKER_BUILDKIT=1 docker build -t ${{env.FAUCET_BUILD_TAG}} -f tools/faucet/Dockerfile .
docker push ${{env.FAUCET_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{ vars.DOCKER_BUILD_TAG_FAUCET }} -f tools/faucet/Dockerfile .
docker push ${{ vars.DOCKER_BUILD_TAG_FAUCET }}

- name: 'Deploy to Azure Container Instances'
uses: 'azure/aci-deploy@v1'
with:
resource-group: ${{ secrets.RESOURCE_GROUP }}
dns-name-label: ${{ inputs.testnet_type }}-faucet
image: ${{ env.FAUCET_BUILD_TAG }}
image: ${{ vars.DOCKER_BUILD_TAG_FAUCET }}
name: ${{ inputs.testnet_type }}-faucet
location: 'uksouth'
restart-policy: 'Never'
environment-variables: PORT=80
command-line: ./faucet --nodeHost ${{ env.TESTNET_ADDR }} --pk ${{ secrets.FAUCET_PK }} --jwtSecret ${{ secrets.FAUCET_JWT_SECRET }} --defaultAmount ${{ env.DEFAULT_FAUCET_AMOUNT }}
command-line: ./faucet --nodeHost ${{ vars.L2_RPC_URL_VALIDATOR }} --pk ${{ secrets.FAUCET_PK }} --jwtSecret ${{ secrets.FAUCET_JWT_SECRET }} --defaultAmount ${{ vars.FAUCET_PAY_AMOUNT }}
ports: '80'
cpu: 2
memory: 2
Comment on lines 62 to 81
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new hunk changes the Docker build and push commands and the Azure Container Instances deployment command to use the new environment variable names. However, the vars object used in these commands is not defined anywhere. This will cause an error when the workflow runs. You should replace vars with the correct object or variable that contains these values.

- DOCKER_BUILDKIT=1 docker build -t ${{ vars.DOCKER_BUILD_TAG_FAUCET }} -f tools/faucet/Dockerfile . 
- docker push ${{ vars.DOCKER_BUILD_TAG_FAUCET }}
+ DOCKER_BUILDKIT=1 docker build -t ${{ env.DOCKER_BUILD_TAG_FAUCET }} -f tools/faucet/Dockerfile . 
+ docker push ${{ env.DOCKER_BUILD_TAG_FAUCET }}

- image: ${{ vars.DOCKER_BUILD_TAG_FAUCET }}
+ image: ${{ env.DOCKER_BUILD_TAG_FAUCET }}

- command-line: ./faucet --nodeHost ${{ vars.L2_RPC_URL_VALIDATOR }} --pk ${{ secrets.FAUCET_PK }} --jwtSecret ${{ secrets.FAUCET_JWT_SECRET }} --defaultAmount ${{ vars.FAUCET_PAY_AMOUNT }}
+ command-line: ./faucet --nodeHost ${{ env.L2_RPC_URL_VALIDATOR }} --pk ${{ secrets.FAUCET_PK }} --jwtSecret ${{ secrets.FAUCET_JWT_SECRET }} --defaultAmount ${{ env.FAUCET_PAY_AMOUNT }}

59 changes: 20 additions & 39 deletions .github/workflows/manual-deploy-testnet-l1.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# WebSocket: 9000, 9001
#
# Exposes the following addresses: (only accessible internally)
# testnet-eth2network-DEPLOYNUMBER.uksouth.azurecontainer.io
# uat-testnet-eth2network-DEPLOYNUMBER.uksouth.azurecontainer.io
# or
# dev-testnet-eth2network-DEPLOYNUMBER.uksouth.azurecontainer.io
#
Expand All @@ -34,32 +34,16 @@ jobs:
name: ${{ github.event.inputs.testnet_type }}

steps:
- name: 'Print GitHub variables'
# This is a useful record of what the environment variables were at the time the job ran, for debugging and reference
run: |
echo "GitHub Variables = ${{ toJSON(vars) }}"

- uses: actions/checkout@v3

- name: 'Set up Docker'
uses: docker/setup-buildx-action@v1

- name: 'Sets env vars for testnet'
if: ${{ github.event.inputs.testnet_type == 'uat-testnet' }}
run: |
echo "ETH2NETWORK_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/uat_testnet_eth2network:latest" >> $GITHUB_ENV
echo "L1_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/uat_testnet_l1network:latest" >> $GITHUB_ENV
echo "L1_CONTAINER_NAME=uat_testnet-eth2network" >> $GITHUB_ENV
echo "RESOURCE_TAG_NAME=uat-testnet-l1-latest" >> $GITHUB_ENV
echo "RESOURCE_NAME=uat-testnet-eth2network" >> $GITHUB_ENV
echo "TEST_BRIDGE_ADDRESS=0xE09a37ABc1A63441404007019E5BC7517bE2c43f" >> $GITHUB_ENV

- name: 'Sets env vars for dev-testnet'
if: ${{ github.event.inputs.testnet_type == 'dev-testnet' }}
run: |
echo "ETH2NETWORK_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/dev_testnet_eth2network:latest" >> $GITHUB_ENV
echo "L1_DOCKER_BUILD_TAG=testnetobscuronet.azurecr.io/obscuronet/dev_testnet_l1network:latest" >> $GITHUB_ENV
echo "L1_CONTAINER_NAME=dev-testnet-eth2network" >> $GITHUB_ENV
echo "RESOURCE_TAG_NAME=dev-testnet-l1-latest" >> $GITHUB_ENV
echo "RESOURCE_NAME=dev-testnet-eth2network" >> $GITHUB_ENV
echo "TEST_BRIDGE_ADDRESS=0xE09a37ABc1A63441404007019E5BC7517bE2c43f" >> $GITHUB_ENV


- name: 'Login via Azure CLI'
uses: azure/login@v1
with:
Expand All @@ -76,53 +60,52 @@ jobs:
# DOCKER_BUILDKIT=1 will enable the new docker build kit that allows us to use build only caches on RUN commands.
# Tag the same image with 2 tags and push with the -a flag which pushes all images
run: |
DOCKER_BUILDKIT=1 docker build -t ${{env.ETH2NETWORK_BUILD_TAG}} -t ${{env.L1_DOCKER_BUILD_TAG}} -f testnet/eth2network.Dockerfile .
docker push ${{env.L1_DOCKER_BUILD_TAG}}
BedrockSquirrel marked this conversation as resolved.
Show resolved Hide resolved
docker push ${{env.ETH2NETWORK_BUILD_TAG}}
DOCKER_BUILDKIT=1 docker build -t ${{ vars.DOCKER_BUILD_TAG_ETH2NETWORK }} -f testnet/eth2network.Dockerfile .
docker push ${{ vars.DOCKER_BUILD_TAG_ETH2NETWORK }}

# This will fail some deletions due to resource dependencies ( ie. you must first delete the vm before deleting the disk)
- name: 'Delete deployed VMs'
uses: azure/CLI@v1
with:
inlineScript: |
$(az resource list --tag ${{env.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true
$(az resource list --tag ${{ vars.AZURE_DEPLOY_GROUP_L1 }}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true

# This will clean up any lingering dependencies - might fail if there are no resources to cleanup
- name: 'Delete VMs dependencies'
uses: azure/CLI@v1
with:
inlineScript: |
$(az resource list --tag ${{env.RESOURCE_TAG_NAME}}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true
$(az resource list --tag ${{ vars.AZURE_DEPLOY_GROUP_L1 }}=true --query '[]."id"' -o tsv | xargs -n1 az resource delete --verbose -g Testnet --ids) || true


- name: 'Create VM for "${{env.RESOURCE_NAME}}-${{ GITHUB.RUN_NUMBER }}" on Azure'
- name: 'Create VM for "${{ github.event.inputs.testnet_type }}-eth2network-${{ GITHUB.RUN_NUMBER }}" on Azure'
uses: azure/CLI@v1
with:
inlineScript: |
az vm create -g Testnet -n "${{env.RESOURCE_NAME}}-${{ GITHUB.RUN_NUMBER }}" \
az vm create -g Testnet -n "${{ github.event.inputs.testnet_type }}-eth2network-${{ GITHUB.RUN_NUMBER }}" \
--admin-username obscurouser --admin-password "${{ secrets.OBSCURO_NODE_VM_PWD }}" \
--public-ip-address-dns-name "${{ env.RESOURCE_NAME }}" \
--tags ${{env.RESOURCE_TAG_NAME}}=true \
--vnet-name ${{env.RESOURCE_NAME}}-01VNET --subnet ${{env.RESOURCE_NAME}}-01Subnet \
--public-ip-address-dns-name "${{ github.event.inputs.testnet_type }}-eth2network" \
--tags ${{ vars.AZURE_DEPLOY_GROUP_L1 }}=true \
--vnet-name ${{ github.event.inputs.testnet_type }}-eth2network-01VNET --subnet ${{ github.event.inputs.testnet_type }}-eth2network-01Subnet \
--size Standard_D3_v2 --image Canonical:0001-com-ubuntu-server-jammy:22_04-lts:latest \
--public-ip-sku Basic --authentication-type password

- name: 'Open Obscuro node-${{ matrix.host_id }} ports on Azure'
uses: azure/CLI@v1
with:
inlineScript: |
az vm open-port -g Testnet -n "${{env.RESOURCE_NAME}}-${{ GITHUB.RUN_NUMBER }}" --port 8025,8026,9000,9001
az vm open-port -g Testnet -n "${{ github.event.inputs.testnet_type }}-eth2network-${{ GITHUB.RUN_NUMBER }}" --port 8025,8026,9000,9001

# To overcome issues with critical VM resources being unavailable, we need to wait for the VM to be ready
- name: 'Allow time for VM initialization'
shell: bash
run: sleep 60

- name: 'Start l1 ${{env.RESOURCE_NAME}}-${{ GITHUB.RUN_NUMBER }} on Azure'
- name: 'Start l1 ${{ github.event.inputs.testnet_type }}-eth2network-${{ GITHUB.RUN_NUMBER }} on Azure'
uses: azure/CLI@v1
with:
inlineScript: |
az vm run-command invoke -g Testnet -n "${{env.RESOURCE_NAME}}-${{ GITHUB.RUN_NUMBER }}" \
az vm run-command invoke -g Testnet -n "${{ github.event.inputs.testnet_type }}-eth2network-${{ GITHUB.RUN_NUMBER }}" \
--command-id RunShellScript \
--scripts 'mkdir -p /home/obscuro \
&& sudo apt-get update \
Expand All @@ -145,10 +128,8 @@ jobs:
datadog/agent:latest \
&& docker run -d \
-p 8025:8025 -p 8026:8026 -p 9000:9000 -p 9001:9001 \
--entrypoint /home/obscuro/go-obscuro/integration/eth2network/main/main ${{ env.ETH2NETWORK_BUILD_TAG }} \
--entrypoint /home/obscuro/go-obscuro/integration/eth2network/main/main ${{ vars.DOCKER_BUILD_TAG_ETH2NETWORK }} \
--blockTimeSecs=15 --slotsPerEpoch=2 --slotsPerSecond=15 \
--numNodes=1 --gethHTTPStartPort=8025 --gethWSStartPort=9000 \
--logToFile=false \
--prefundedAddrs="${{ vars.WORKER_ADDR }},${{ vars.NODE_WALLET_ADDR_0 }},${{ vars.NODE_WALLET_ADDR_1 }},${{ env.TEST_BRIDGE_ADDRESS }}"'


--prefundedAddrs="${{ vars.ACCOUNT_ADDR_WORKER }},${{ vars.ACCOUNT_ADDR_NODE_0 }},${{ vars.ACCOUNT_ADDR_NODE_1 }},${{ vars.ACCOUNT_ADDR_L1_BRIDGE_TEST }}"'
Loading
Loading