Skip to content

Commit

Permalink
First draft.
Browse files Browse the repository at this point in the history
  • Loading branch information
mareuter committed Jul 24, 2024
1 parent 7811ddb commit 443673b
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 0 deletions.
80 changes: 80 additions & 0 deletions System-Architecture/Supporting-Infrastructure/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
.. _System-Architecture-Supporting-Infrastructure:

#########################
Supporting Infrastructure
#########################

Storage Retention Policies
^^^^^^^^^^^^^^^^^^^^^^^^^^

Summit
------

.. list-table:: Retention Policies
:widths: auto
:header-rows: 1

* - Content
- Retention (days)
- Provider
- Server
* - AuxTel Raw Images (Butler)
- 90
- Ceph NFS
- yagan
* - AuxTel Quicklook Images (Butler)
- ?? (90)
- Ceph NFS
- yagan
* - Large File Annex
- 30
- Ceph S3
- chonchon
* - RubinTV Products
- ??
- Ceph S3
- chonchon
* - ComCam Raw Images (Butler)
- 90
- NFS
- comcam-archiver
* - ComCam Quicklook Images (Butler)
- ?? (90)
- NFS
- comcam-archiver
* - LSSTCam Raw Images (Butler)
- ??
- Ceph NFS
- chonchon
* - LSSTCam Quicklook Images (Butler)
- ??
- Ceph NFS
- chonchon
* - LSSTCam Raw Images (Butler)
- 30
- Ceph S3
- new cluster
* - LSSTCam Quicklook Images (Butler)
- ?? (30)
- Ceph S3
- new cluster
* - DREAM Products
- ??
- Ceph S3
- new cluster
* - EFD
- 30
- Ceph PVC
- yagan

Notes:
* Quicklook images are generated by Rapid Analysis.
* Butler retentions are shared, but independent application is in progress.
* The Ceph S3 on the new cluster entries are for future reference. System is currently not ready for use.
Open issues:
* All retention policies should be reviewed for relevance.
* Anything with ?? needs a decision.
* Is DREAM going to store any products when the new cluster is available?
* Since calibration products are absolved from deletion, they will accumulate over time. How do we decide when and/or if to purge?
* User data products in the butler are not covered by a retention policy. Should they?
* Ceph NFS for LSSTCam and ComCam will have to be shared so will greatly influence the retention duration
1 change: 1 addition & 0 deletions System-Architecture/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ These are generally grouped according to their association with a specific teles

Control-Packages/*
CSC-Overview/*
Supporting-Infrastructure/*

There exist multiple components for which documentation is not yet available including: the EFD, configuration management, deployment, and the scriptQueue.

Expand Down

0 comments on commit 443673b

Please sign in to comment.