Gen3 Monthly Release BDC Preprod 2025.01 preprod.gen3.biodatacatalyst.nhlbi.nih.gov 1736883397

[PlanXCyborg]

Applying version 2025.01 to preprod.gen3.biodatacatalyst.nhlbi.nih.gov

[github-actions]

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

• awshelper
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (IRSA Changes cloud-automation#2531)
  • "serviceAccountName": "batch-export-sa"... (IRSA Changes cloud-automation#2531)
  • name: BUCKET (IRSA Changes cloud-automation#2531)
  • valueFrom: (IRSA Changes cloud-automation#2531)
  • configMapKeyRef: (IRSA Changes cloud-automation#2531)
  • name: batch-export-g3auto (IRSA Changes cloud-automation#2531)
  • key: bucket_name (IRSA Changes cloud-automation#2531)
  • Remove the following: (IRSA Changes cloud-automation#2531)
  • "volumeMounts": [ (https://github.com/IRSA Changes cloud-automation#2531)
  • { (IRSA Changes cloud-automation#2531)
  • "name": "batch-export-creds-volume", (IRSA Changes cloud-automation#2531)
  • "readOnly": true, (IRSA Changes cloud-automation#2531)
  • "mountPath": "/batch-export-creds.json", (IRSA Changes cloud-automation#2531)
  • "subPath": "config.json" (IRSA Changes cloud-automation#2531)
  • } (IRSA Changes cloud-automation#2531)
  • ], (IRSA Changes cloud-automation#2531)
  • }, (IRSA Changes cloud-automation#2531)
  • "volumes": [ (https://github.com/IRSA Changes cloud-automation#2531)
  • { (IRSA Changes cloud-automation#2531)
  • "name": "batch-export-creds-volume", (IRSA Changes cloud-automation#2531)
  • "secret": { (IRSA Changes cloud-automation#2531)
  • "secretName": "batch-export-g3auto" (IRSA Changes cloud-automation#2531)
  • } (IRSA Changes cloud-automation#2531)
  • } (IRSA Changes cloud-automation#2531)
  • ], (IRSA Changes cloud-automation#2531)
  • Need to mount the configuration files in "requestor" and "metadata" directories to accommodate the new Dockerfile for these services. (adding changes for new al image changes cloud-automation#2661)
  • We are also using Poetry to manage virtual environments instead of python, so the command to run alembic needs to be modified. (adding changes for new al image changes cloud-automation#2661)
  • Elasticsearch 7 endpoint will be the default. If you want to use ES6, make sure to set "ES7" "false" in the manfist.json. (Making ES7 domain default cloud-automation#2655)
• fence
  • activation of the feature depends on a new configuration setting: ALLOW_NEW_USER_ON_LOGIN (Feat: only allow login of onboarded users fence#1184)
  • any env that has S3 buckets in fence config that has wildcard characters should consider adopting this release (Fix/bucket name fence#1193)
• guppy
  • To enable this feature in helm charts add "allowRefresh": {{ .Values.allowRefresh }} to gen3-helm/helm/guppy/templates/guppy_config.yaml and then in your values.yaml file, specify allowRefresh as the boolean true value. (Feature/refresh guppy#273)
• metadata-delete-expired-objects
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• portal
  • new config option dataDictionaryVersion that can be set to "new" or something else. If "new", then the new data dictionary component is loaded. This is a temporary solution until we have fully deprecated the old one. (Feat: enable old and new data dictionary for VA data-portal#1613)
  • The VLMD syntax in HEAL MDS has been updated (HP-1484 HP-1617 Feat/cde submission data-portal#1547)
• sower.container.image.download-indexd-manifest
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• sower.container.image.get-dbgap-metadata
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• sower.container.image.manifest-indexing
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• sower.container.image.manifest-merging
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• sower.container.image.metadata-manifest-ingestion
  • Manifest.json files MUST be changed to use the new "batch-export-sa" service account and the "BUCKET" environment variable must be set to the batch-export-g3auto secret with the "bucket_name" key. You also no longer need to mount the batch-export-g3auto secret as this is now deprecated (it is replaced by the batch-export-g3auto configmap). (Export Job to IRSA sower-jobs#53)
• sower.container.image.pelican-export
  • Our Helm charts utilize eternal secrets to create and manage our secrets. The new Peregrine-dbcreds secret is formatted differently and no longer uses json, so we need to edit the export job so the values can be read via environment variables. (updating job export to work with Helm pelican#89)
  • basing the pelican images off of a more secure AL base image. (making al base image updates to pelican export image pelican#88)

Breaking changes

• fence
  • remove role field from /admin/user POST endpoint (Feat: remove role from POST /admin/user endpoint fence#1202)
  • set User.is_admin to DEPRECATED (Feat: remove role from POST /admin/user endpoint fence#1202)
  • deprecation notices for future breaking changes where we'll remove a number of legacy endpoints (Feat: deprecate legacy unused /admin/ endpoints fence#1201)
  • some of the admin endpoints and methods now expect username instead of name. (Feat: add support for soft delete user fence#1189)