DD-1763 Voeg --age parameter toe aan deposit-clean-data

[aliassheikh]

Fixes DD-1763 Voeg --age parameter toe aan deposit-clean-data

Description of changes

• added --age parameter to give an int parameter which will be used in 'today - age' as an 'end-date'.
  • end-date will be used to clean de dd-dataverse-ingest work-folder (/tmp subfolders) until the given end-date
• mutually_exclusive parameters:
  • --enddate
  • --age

How to test

Ran the script in a prepared test environment in vagrant with the age parameter --age 612

• tested on 4 March 2025 and the creation day of those deposit was 2023-06-30
• Note also that the delete column is changed automatically and set to false in dd-manage-deposit internal database.

depositor | depositId | bagName | depositState | depositCreationTimestamp | depositUpdateTimestamp | description | location | storageInBytes | deleted
-- | -- | -- | -- | -- | -- | -- | -- | -- | --
jangoogle | d3559071-474d-4825-b0a1-f1d2b07142a3 | all-mappings | PUBLISHED | 2023-06-30T17:35:18.934453+02:00 | 2025-03-03T14:55:03.328627+01:00 | The deposit was successfully ingested in the Data Station and will be automatically archived | /var/opt/dans.knaw.nl/tmp/auto-ingest/outbox/processed | 17723 | true
user001 | b9e2bf38-ec9f-4c29-9b75-ae725c8fcea5 | default-open | PUBLISHED | 2024-12-01T11:31:13.819851+01:00 | 2024-12-01T11:32:15.764606+01:00 | The deposit was successfully ingested in the Data Station and will be automatically archived. | /var/opt/dans.knaw.nl/tmp/auto-ingest/outbox/processed | 1749155 | false
jangoogle | 1c242a44-7c40-42e4-8215-8a0a19e1474f | all-mappings | FAILED | 2023-06-30T17:30:04.055385+02:00 | 2025-03-03T14:47:48.903738+01:00 | Failed /var/opt/dans.knaw.nl/tmp/auto-ingest/inbox/1c242a44-7c40-42e4-8215-8a0a19e1474f: Error creating dataset, deleting draft | /var/opt/dans.knaw.nl/tmp/auto-ingest/outbox/failed | 17660 | true
user001 | 2bacaf7e-7d5c-47e4-96af-60df763eee84 | audiences | PUBLISHED | 2024-12-01T11:36:05.091256+01:00 | 2024-12-01T11:36:10.882592+01:00 | The deposit was successfully ingested in the Data Station and will be automatically archived. | /var/opt/dans.knaw.nl/tmp/auto-ingest/outbox/processed | 3216665 | false
jangoogle | 4ec7b614-f3b1-4c18-8ff7-bab5843f3505 | all-mappings | FAILED | 2023-06-30T17:27:01.781809+02:00 | 2025-03-03T14:48:11.495391+01:00 | Failed /var/opt/dans.knaw.nl/tmp/auto-ingest/inbox/4ec7b614-f3b1-4c18-8ff7-bab5843f3505: Error creating dataset, deleting draft | /var/opt/dans.knaw.nl/tmp/auto-ingest/outbox/failed | 17660 | true

Related PRs

(Add links)

Notify

@DANS-KNAW/core-systems