From 443673b515e837ed422818e403b59b32717cf8a2 Mon Sep 17 00:00:00 2001 From: Michael Reuter Date: Tue, 23 Jul 2024 17:38:08 -0700 Subject: [PATCH] First draft. --- .../Supporting-Infrastructure/index.rst | 80 +++++++++++++++++++ System-Architecture/index.rst | 1 + 2 files changed, 81 insertions(+) create mode 100644 System-Architecture/Supporting-Infrastructure/index.rst diff --git a/System-Architecture/Supporting-Infrastructure/index.rst b/System-Architecture/Supporting-Infrastructure/index.rst new file mode 100644 index 0000000..0c9b464 --- /dev/null +++ b/System-Architecture/Supporting-Infrastructure/index.rst @@ -0,0 +1,80 @@ +.. _System-Architecture-Supporting-Infrastructure: + +######################### +Supporting Infrastructure +######################### + +Storage Retention Policies +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Summit +------ + +.. list-table:: Retention Policies + :widths: auto + :header-rows: 1 + + * - Content + - Retention (days) + - Provider + - Server + * - AuxTel Raw Images (Butler) + - 90 + - Ceph NFS + - yagan + * - AuxTel Quicklook Images (Butler) + - ?? (90) + - Ceph NFS + - yagan + * - Large File Annex + - 30 + - Ceph S3 + - chonchon + * - RubinTV Products + - ?? + - Ceph S3 + - chonchon + * - ComCam Raw Images (Butler) + - 90 + - NFS + - comcam-archiver + * - ComCam Quicklook Images (Butler) + - ?? (90) + - NFS + - comcam-archiver + * - LSSTCam Raw Images (Butler) + - ?? + - Ceph NFS + - chonchon + * - LSSTCam Quicklook Images (Butler) + - ?? + - Ceph NFS + - chonchon + * - LSSTCam Raw Images (Butler) + - 30 + - Ceph S3 + - new cluster + * - LSSTCam Quicklook Images (Butler) + - ?? (30) + - Ceph S3 + - new cluster + * - DREAM Products + - ?? + - Ceph S3 + - new cluster + * - EFD + - 30 + - Ceph PVC + - yagan + +Notes: + * Quicklook images are generated by Rapid Analysis. + * Butler retentions are shared, but independent application is in progress. + * The Ceph S3 on the new cluster entries are for future reference. System is currently not ready for use. +Open issues: + * All retention policies should be reviewed for relevance. + * Anything with ?? needs a decision. + * Is DREAM going to store any products when the new cluster is available? + * Since calibration products are absolved from deletion, they will accumulate over time. How do we decide when and/or if to purge? + * User data products in the butler are not covered by a retention policy. Should they? + * Ceph NFS for LSSTCam and ComCam will have to be shared so will greatly influence the retention duration diff --git a/System-Architecture/index.rst b/System-Architecture/index.rst index 4ac71e8..472d4e0 100644 --- a/System-Architecture/index.rst +++ b/System-Architecture/index.rst @@ -26,6 +26,7 @@ These are generally grouped according to their association with a specific teles Control-Packages/* CSC-Overview/* + Supporting-Infrastructure/* There exist multiple components for which documentation is not yet available including: the EFD, configuration management, deployment, and the scriptQueue.