Skip to content

Commit

Permalink
Merge branch 'main' into repo-sync-973b7272da018d817323f99f409d2dd3a8…
Browse files Browse the repository at this point in the history
…dbbb8b-38.1
  • Loading branch information
Andrew Huynh committed May 23, 2023
2 parents 973b727 + 674f21a commit 07b8d36
Show file tree
Hide file tree
Showing 1,028 changed files with 18,074 additions and 38,334 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ _Describe what tests you have done._
# Requirements
_Before commit the code, please do the following steps._
1. Run `make fmt` and `make fmt-sh`
2. Run `make linter`
2. Run `make lint`



Expand Down
46 changes: 46 additions & 0 deletions .github/repo_sync_pr_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Description of the issue
An automated PR to kickstart the process of syncing the latest changes from [cw-agent](https://github.com/aws/amazon-cloudwatch-agent/)

# Description of changes

### Follow the git CLI instructions resolve the merge conflicts

```shell
git pull origin main
git checkout repo-sync-<hash>-<run_id>
git merge main # do a regular merge -- we want to keep the commits
# resolve merge conflicts in your preferred IDE
git push -u origin repo-sync-<hash>-<run_id>
```

Some useful commands
* [Restore conflict resolution in a single file](https://stackoverflow.com/questions/14409420/restart-undo-conflict-resolution-in-a-single-file) - `git checkout -m <FILE>`
* Total reset - `git merge --abort`

### Related docs
* Resolving conflicts with:
* [Git CLI](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/addressing-merge-conflicts/resolving-a-merge-conflict-using-the-command-line)
* [IntelliJ](https://www.jetbrains.com/help/idea/resolving-conflicts.html#distributed-version-control-systems)
* [GoLand](https://www.jetbrains.com/help/go/resolve-conflicts.html)
* [VSCode](https://learn.microsoft.com/en-us/visualstudio/version-control/git-resolve-conflicts?view=vs-2022)

### Best practices

* Remember to update all references from `amazon-cloudwatch-agent` to `private-amazon-cloudwatch-agent-staging`
* Resolve the `go.sum` with `go mod tidy`. Don't bother manually resolving conflicts in this file
* When finished, ensure builds work by using `make build` or `make release`
* When unsure or blocked, do a deep dive on the `git blame` for greater context. Maybe even look for the associated PR's and ask the original authors and PR approvers
* If another automated PR arrives before your work is merged, just close your current one and save the branch
* After your PR is approved, **do a regular merge to preserve the commits**.
* Remember to cleanup your commits because none of them will be squashed in a regular merge

# License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

# Tests
n/a

# Requirements
_Before commit the code, please do the following steps._
1. Run `make fmt` and `make fmt-sh`
2. Run `make lint`
2 changes: 1 addition & 1 deletion .github/workflows/PR-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest, windows-2019, windows-latest, macos-latest]
os: [ ubuntu-latest, windows-2019, windows-latest, macos-11]
include:
- os: ubuntu-latest
family: linux
Expand Down
76 changes: 76 additions & 0 deletions .github/workflows/alpha-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT

# alpha-release.yml - releasing CWA artifacts for customers and will be safe to deleted after Alpha release etc.
name: Alpha Release
env:
TERRAFORM_AWS_ASSUME_ROLE: ${{ secrets.TERRAFORM_AWS_ASSUME_ROLE }}
S3_INTEGRATION_BUCKET: ${{ secrets.S3_INTEGRATION_BUCKET }}

on:
release:
types: [published]

jobs:
UploadRelease:
name: "UploadRelease"
runs-on: ubuntu-latest
environment: release
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v2

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2

- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ~1.19.2

- name: Cache go
id: cached_go
uses: actions/cache@v2
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: v1-go-pkg-mod-${{ runner.os }}-${{ hashFiles('**/go.sum') }}

# Required to build rpm again to reset cw agent version to new tag version
- name: Build RPM
run: make build package-rpm

- name: Check current release version against new version and ensure they are different
run: |
aws s3 cp s3://${S3_INTEGRATION_BUCKET}/release/CWAGENT_VERSION /tmp/
current_val=`cat /tmp/CWAGENT_VERSION`
new_val=`cat build/bin/CWAGENT_VERSION`
if [ "$current_val" = "$new_val" ]; then
echo "This version has already been published. Do nothing."
exit 1
fi
- name: Copy RPM To Release
run: |
aws s3 cp build/bin/linux/amd64/amazon-cloudwatch-agent.rpm s3://${S3_INTEGRATION_BUCKET}/release/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
aws s3 cp build/bin/linux/arm64/amazon-cloudwatch-agent.rpm s3://${S3_INTEGRATION_BUCKET}/release/amazon_linux/arm64/latest/amazon-cloudwatch-agent.rpm
aws s3 cp build/bin/CWAGENT_VERSION s3://${S3_INTEGRATION_BUCKET}/release/CWAGENT_VERSION
- name: Copy RPM To Tag
run: |
export tag=$(cat build/bin/CWAGENT_VERSION)
aws s3 cp build/bin/linux/amd64/amazon-cloudwatch-agent.rpm s3://${S3_INTEGRATION_BUCKET}/release/${tag}/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
aws s3 cp build/bin/linux/arm64/amazon-cloudwatch-agent.rpm s3://${S3_INTEGRATION_BUCKET}/release/${tag}/amazon_linux/arm64/latest/amazon-cloudwatch-agent.rpm
aws s3 cp build/bin/CWAGENT_VERSION s3://${S3_INTEGRATION_BUCKET}/release/${tag}/CWAGENT_VERSION
# Assume we do not run 2 releases at the same time
DeployCanary:
needs: [UploadRelease]
uses: ./.github/workflows/deploy-canary.yml
secrets: inherit
20 changes: 19 additions & 1 deletion .github/workflows/clean-aws-resources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,25 @@ jobs:

- name: Clean old ami
run: go run ./tool/clean/clean_ami/clean_ami.go --tags=clean


clean-old-file-systems:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v3

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2

- name: Clean old file system
run: go run ./tool/clean/clean_file_system/clean_file_system.go --tags=clean

clean-opensource-dedicated-hosts:
runs-on: ubuntu-latest
permissions:
Expand Down
27 changes: 0 additions & 27 deletions .github/workflows/codeql-analysis.yml

This file was deleted.

90 changes: 90 additions & 0 deletions .github/workflows/deploy-canary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT

name: Deploy Canary
env:
TERRAFORM_AWS_ASSUME_ROLE: ${{ secrets.TERRAFORM_AWS_ASSUME_ROLE }}
S3_INTEGRATION_BUCKET: ${{ secrets.S3_INTEGRATION_BUCKET }}
KEY_NAME: ${{ secrets.KEY_NAME }}
PRIVATE_KEY: ${{ secrets.AWS_PRIVATE_KEY }}
CWA_GITHUB_TEST_REPO_NAME: "aws/amazon-cloudwatch-agent-test"
CWA_GITHUB_TEST_REPO_URL: "https://github.com/aws/amazon-cloudwatch-agent-test.git"
CWA_GITHUB_TEST_REPO_BRANCH: "main"

on:
schedule:
- cron: "0 15 * * *" # Run daily at 15:00 UTC
workflow_call:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true

jobs:
DeployCanary:
name: "DeployCanary"
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v2
with:
repository: ${{env.CWA_GITHUB_TEST_REPO_NAME}}
ref: ${{env.CWA_GITHUB_TEST_REPO_BRANCH}}

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2

- name: Terminate Last Canary
run: |
if aws s3api wait object-exists --bucket ${S3_INTEGRATION_BUCKET} --key canary/al2/terraform.tfstate ;
then
cd terraform/ec2/linux
aws s3 cp s3://${S3_INTEGRATION_BUCKET}/canary/al2/terraform.tfstate .
terraform --version
terraform init
terraform destroy -auto-approve
aws s3api delete-object --bucket ${S3_INTEGRATION_BUCKET} --key canary/al2/terraform.tfstate
fi
# @TODO we can add a matrix in the future but for alpha we will only deploy to al2
- name: Terraform apply
uses: nick-fields/retry@v2
with:
max_attempts: 3
timeout_minutes: 60
retry_wait_seconds: 5
command: |
cd terraform/ec2/linux
terraform init
if terraform apply --auto-approve \
-var="github_test_repo=${{env.CWA_GITHUB_TEST_REPO_URL}}" \
-var="github_test_repo_branch=${{env.CWA_GITHUB_TEST_REPO_BRANCH}}" \
-var="user=ec2-user" \
-var="ami=cloudwatch-agent-integration-test-al2*" \
-var="arc=amd64" \
-var="binary_name=amazon-cloudwatch-agent.rpm" \
-var="s3_bucket=${S3_INTEGRATION_BUCKET}" \
-var="ssh_key_name=${KEY_NAME}" \
-var="ssh_key_value=${PRIVATE_KEY}" \
-var="test_name=canary" \
-var="is_canary=true" \
-var="test_dir=./test/canary" ; then aws s3 cp terraform.tfstate s3://${S3_INTEGRATION_BUCKET}/canary/al2/terraform.tfstate
else
terraform destroy -auto-approve && exit 1
fi
#This is here just in case workflow cancel
- name: Terraform destroy
if: ${{ cancelled() }}
uses: nick-fields/retry@v2
with:
max_attempts: 3
timeout_minutes: 8
retry_wait_seconds: 5
command: cd terraform/ec2/linux && terraform destroy --auto-approve
25 changes: 8 additions & 17 deletions .github/workflows/integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ jobs:

- name: Build Binaries
if: steps.cached_binaries.outputs.cache-hit != 'true'
run: make amazon-cloudwatch-agent-linux amazon-cloudwatch-agent-windows package-rpm package-deb package-win
run: make amazon-cloudwatch-agent-linux amazon-cloudwatch-agent-windows package-rpm package-deb package-win package-darwin

- name: Sign Build Files
if: steps.cached_binaries.outputs.cache-hit != 'true'
Expand Down Expand Up @@ -254,7 +254,6 @@ jobs:
with:
path: cwa
fetch-depth: 0

- uses: actions/checkout@v3
with:
repository: ${{env.CWA_GITHUB_TEST_REPO_NAME}}
Expand All @@ -271,13 +270,6 @@ jobs:
role-to-assume: ${{ env.TERRAFORM_AWS_ASSUME_ROLE }}
aws-region: us-west-2

- name: Cache binaries
id: cached_binaries
uses: actions/cache@v3
with:
key: "cached-binaries-${{ runner.os }}-${{ github.sha }}"
path: go.mod

- name: Cache pkg
if: steps.cached_binaries.outputs.cache-hit != 'true'
uses: actions/cache@v3
Expand All @@ -286,14 +278,14 @@ jobs:
~/Library/Caches/go-build
~/go/pkg/mod
key: v1-go-pkg-mod-${{ runner.os }}-${{ hashFiles('**/go.sum') }}

- name: Build Binaries
if: steps.cached_binaries.outputs.cache-hit != 'true'
if: steps.cached_pkg.outputs.cache-hit != 'true'
working-directory: cwa
run: make amazon-cloudwatch-agent-darwin package-darwin
run: make build package-darwin

- name: Copy binary
if: steps.cached_binaries.outputs.cache-hit != 'true'
if: steps.cached_pkg.outputs.cache-hit != 'true'
working-directory: cwa
run: |
echo cw agent version $(cat CWAGENT_VERSION)
Expand All @@ -302,7 +294,7 @@ jobs:
cp build/bin/CWAGENT_VERSION /tmp/CWAGENT_VERSION
- name: Create pkg dep folder and copy deps
if: steps.cached_binaries.outputs.cache-hit != 'true'
if: steps.cached_pkg.outputs.cache-hit != 'true'
working-directory: test
run: |
cp -r pkg/tools/. /tmp/
Expand Down Expand Up @@ -404,7 +396,7 @@ jobs:
aws s3 cp packages/amazon-cloudwatch-agent.msi.sig s3://${S3_INTEGRATION_BUCKET}/integration-test/packaging/${{ github.sha }}/amazon-cloudwatch-agent.msi.sig
aws s3 cp packages/amd64/amazon-cloudwatch-agent.pkg.sig s3://${S3_INTEGRATION_BUCKET}/integration-test/packaging/${{ github.sha }}/amd64/amazon-cloudwatch-agent.pkg.sig
aws s3 cp packages/arm64/amazon-cloudwatch-agent.pkg.sig s3://${S3_INTEGRATION_BUCKET}/integration-test/packaging/${{ github.sha }}/arm64/amazon-cloudwatch-agent.pkg.sig
StartLocalStack:
name: 'StartLocalStack'
runs-on: ubuntu-latest
Expand Down Expand Up @@ -448,7 +440,7 @@ jobs:
echo $LOCAL_STACK_HOST_NAME &&
echo "::set-output name=local_stack_host_name::$LOCAL_STACK_HOST_NAME" &&
aws s3 cp terraform.tfstate s3://${S3_INTEGRATION_BUCKET}/integration-test/local-stack-terraform-state/${GITHUB_SHA}/terraform.tfstate
EC2NvidiaGPUIntegrationTest:
needs: [ MakeBinary, BuildMSI, StartLocalStack, GenerateTestMatrix ]
name: 'EC2NVIDIAGPUIntegrationTest'
Expand Down Expand Up @@ -933,7 +925,6 @@ jobs:
retry_wait_seconds: 5
command: cd terraform/ecs_fargate/linux && terraform destroy --auto-approve


EKSIntegrationTest:
name: 'EKSIntegrationTest'
runs-on: ubuntu-latest
Expand Down
Loading

0 comments on commit 07b8d36

Please sign in to comment.