Skip to content

Commit

Permalink
Query all controllers for backups
Browse files Browse the repository at this point in the history
There currently is a problem with ctl-0, which is where we always
fetched backups from.

This extends the fetch script to look in the directories of other
controllers too.

Additionally, it fixes a KeyError when nothing is returned from
the query. It now checks for the presence of the key, rather than
size.

```python
File "/app/openstack_billing_db/fetch.py", line 67, in download_latest_dump_from_s3
    if len(dumps["Contents"]) == 0:
           ~~~~~^^^^^^^^^^^^
KeyError: 'Contents'
```
  • Loading branch information
knikolla committed May 30, 2024
1 parent 5d37060 commit 4745cfe
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions src/openstack_billing_db/fetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,19 @@ def download_latest_dump_from_s3() -> str:
aws_secret_access_key=s3_secret,
)

key = None
today = datetime.today().strftime("%Y%m%d")
dumps = s3.list_objects_v2(Bucket=s3_bucket, Prefix=f"dbs/nerc-ctl-0/nova-{today}")

if len(dumps["Contents"]) == 0:
for ctl in ["nerc-ctl-0", "nerc-ctl-1", "nerc-ctl-2"]:
dumps = s3.list_objects_v2(Bucket=s3_bucket, Prefix=f"dbs/{ctl}/nova-{today}")

if "Contents" in dumps:
key = dumps["Contents"][0]["Key"]
break

if not key:
raise Exception(f"No database dumps found for {today}")

key = dumps["Contents"][0]["Key"]
filename = os.path.basename(key)
download_location = f"/tmp/{filename}"

Expand Down

0 comments on commit 4745cfe

Please sign in to comment.