Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(buckets): add CRYOSTAT_BUCKETS for pre-created buckets #4

Merged
merged 7 commits into from
Jan 22, 2024

Conversation

andrewazores
Copy link
Member

@andrewazores andrewazores commented Jan 16, 2024

Welcome to Cryostat3! 👋

Before contributing, make sure you have:

  • Read the contributing guidelines
  • Linked a relevant issue which this PR resolves
  • Linked any other relevant issues, PR's, or documentation, if any
  • Resolved all conflicts, if any
  • Rebased your branch PR on top of the latest upstream main branch
  • Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
  • Signed all commits using a GPG signature

To recreate commits with GPG signature git fetch upstream && git rebase --force --gpg-sign upstream/main


Fixes: #3
Based on #1
Depends on #1

Description of the change:

Adds an environment variable CRYOSTAT_BUCKETS that defines a comma-separated list of bucket names that should be automatically created in the storage volume.

Motivation for the change:

See #1. It is useful to be able to declare the required buckets on the storage container itself, so that applications (like cryostat) that expect particular buckets to exist can simply assume that they do/will, rather than needing to implement behaviour to check and create them all the time on their own.

An alternate approach is to figure out what initialization work actually needs to be done (ie what is weed shell doing under the covers), and see if it is possible to do that work before executing the original entrypoint script and bootstrapping the system. This is probably more fragile to any underlying implementation changes by Seaweed, however.

How to manually test:

  1. ./build.bash
  2. Run the just-built image, ex.: podman run --rm -it -e CRYOSTAT_ACCESS_KEY=a -e CRYOSTAT_SECRET_KEY=b -e CRYOSTAT_BUCKETS=archivedrecordings -e WEED_V=4 -p 8333:8333 -p 8888:8888 cryostat-storage:latest
  3. Wait some time (up to 30 seconds), then use the aws CLI and run: AWS_ACCESS_KEY_ID=a AWS_SECRET_ACCESS_KEY=b aws s3 ls --endpoint-url http://localhost:8333
  4. The result should be a line like 2024-01-16 18:01:26 archivedrecordings, indicating the creation time of the one bucket specified in the CRYOSTAT_BUCKETS env var. Repeat this test with different CRYOSTAT_BUCKETS values, separated by commas, with lengths from 3 to 63 characters.

@andrewazores andrewazores added the feat New feature or request label Jan 16, 2024
@andrewazores andrewazores marked this pull request as ready for review January 17, 2024 15:56
@andrewazores andrewazores requested a review from ebaron January 17, 2024 15:57
@andrewazores andrewazores merged commit e06eae5 into cryostatio:main Jan 22, 2024
1 check passed
@andrewazores andrewazores deleted the precreate-buckets branch January 22, 2024 21:05
andrewazores added a commit to andrewazores/cryostat-storage that referenced this pull request Aug 16, 2024
andrewazores added a commit to andrewazores/cryostat-storage that referenced this pull request Aug 20, 2024
mergify bot pushed a commit that referenced this pull request Sep 9, 2024
andrewazores added a commit that referenced this pull request Sep 9, 2024
…23) (#24)

fix(startup): improve startup detection for bucket creation (#23)

See #4
Related to cryostatio/cryostat-helm#115

(cherry picked from commit ff64237)

Co-authored-by: Andrew Azores <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat New feature or request
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Request] Environment Variable to control pre-created buckets
2 participants