Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI setup for AWS #312

Open
9 tasks
phbnf opened this issue Nov 15, 2024 · 2 comments · May be fixed by #349
Open
9 tasks

CI setup for AWS #312

phbnf opened this issue Nov 15, 2024 · 2 comments · May be fixed by #349
Assignees
Labels
enhancement New feature or request

Comments

@phbnf
Copy link
Contributor

phbnf commented Nov 15, 2024

  • provide basic backend storage setup
    • eventually, use Aurora Serverless instead of setting up a new AuroraDB every time
  • integrate with ECS + Fargate to run the conformance test (I haven't tested this yet with lambda functions, but did so from a VM)
    • add a VPC between the lambda function and S3
    • set-up service accounts
    • use IAM for database authentication
  • integrate with a build pipeilne
  • integrate with a deployment pipeline
  • connect the build pipeline to git
@phbnf
Copy link
Contributor Author

phbnf commented Nov 28, 2024

I spent a bunch of time looking into (integrate with a build pipeline, integrate with a deployment pipeline), and spoke a bit with @patflynn about various options.

We can:

  1. do everything in AWS, using CodeBuild, CodePipeline. It seems like a good idea since everything will be deployed on AWS anyways.
  2. do everything in GCP, in the same way that we do this for the GCP deployment (modulo that the conformance binary and hammer would run on AWS). There would be some wrinkles about a. which container registry we push the AWS conformance binary, b. cross cloud authentication (probably requires using secret keys, etc)
  3. do everything with GitHub action. Same wrinkles as for 2., but there's a lot open source bits that we can re-use to make this work.

I'm going to move forward with 3.:

  • authentication with AWS is greatly simplified by https://github.com/aws-actions/configure-aws-credentials, and https://github.com/aws-actions/amazon-ecr-login
  • after playing with the three options, I come to the conclusion that GithubAction is by far the most frictionless option: a single .yaml file to rule them all, can easily pass data between steps, lowest learning curve, the team is already familiar with this
  • it's different than the GCP setup, but we already use Github Actions for a few things including some integration tests, so it's not breaking ground either.
  • we can always move the GCP setup to Github Actions later if we want consistency

@phbnf
Copy link
Contributor Author

phbnf commented Nov 28, 2024

For authentication, we'll start by storing a secret key in GitHub for a user defined with clickops that has the right permissions. We'll move to a sleeker form of authentication later.

@phbnf phbnf linked a pull request Dec 2, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant