chore: prep 0.8.4 release (#2217)

* chore: update renku version * chore: add instructions on upgrading from 0.8.3 to 0.8.4 * chore: fix spelling errors Co-authored-by: Rok Roškar <[email protected]> Co-authored-by: Pamela Delgado <[email protected]>
SwissDataScienceCenter · Jul 26, 2021 · 7abd149 · 7abd149
1 parent 2a91eb2
commit 7abd149
Show file tree

Hide file tree

Showing 6 changed files with 142 additions and 3 deletions.
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -1,5 +1,92 @@
 .. _changelog:
 
+0.8.4
+-----
+
+This version of Renku introduces the ability to use persistent volumes for user sessions. This is optional and can be enabled in the values
+file for the helm chart. In addition to enabling this feature users have the ability to select the storage class used by the persistent 
+volumes. We strongly recommend that a storage class with a `Delete` reclaim policy is used, otherwise persistent volumes from all user
+sessions will keep accumulating. 
+
+Also, unlike previous versions, with 0.8.4 the amount of disk storage will be **strongly enforced**, 
+regardless of whether persistent volumes are used or not. With persistent volumes users will simply run out of space. However, 
+when persistent volumes are not used, going over the amount of storage that a user has requested when starting their session 
+will result in eviction of the k8s pod that runs the session and termination of the session. Therefore, admins are advised 
+to review and set proper options for disk sizes in the `notebooks.serverOptions` portion of the values file.
+
+Improvements
+~~~~~~~~~~~~~
+
+* **UI**: Add banner advertising new version to logged-in users and various improvements in the new canary deployment itself.
+* **Notbooks**: Ability to use persistent volumes for user sessions.
+
+Bug Fixes
+~~~~~~~~~
+
+-  **CI/CD:** CI action entrypoint typo
+   (`3858df0 <https://github.com/SwissDataScienceCenter/renku/commit/3858df02182abeab26e324914fd7bcae7e7226ff>`__)
+-  **Acceptance Tests:** flaky FreeTextDatasetSearchSpec
+   (`a872504 <https://github.com/SwissDataScienceCenter/renku/commit/a872504becb41c1a761cbe02525cae3ebdb6ebea>`__)
+-  **Acceptance Tests:** retry when EOF occurs on the Rest Client
+   (`#2211 <https://github.com/SwissDataScienceCenter/renku/issues/2211>`__)
+   (`e81a212 <https://github.com/SwissDataScienceCenter/renku/commit/e81a21229621463b4be4759f8c4b16714de097c4>`__)
+-  **Acceptance Tests:** Wait for the dataset search results
+   (`#2210 <https://github.com/SwissDataScienceCenter/renku/issues/2210>`__)
+   (`132ec8b <https://github.com/SwissDataScienceCenter/renku/commit/132ec8b813ad6777ae309699d1769cdf07380571>`__)
+
+Features
+~~~~~~~~
+
+-  **CI/CD:** parametrize rancher API endpoint
+   (`46a5155 <https://github.com/SwissDataScienceCenter/renku/commit/46a51551da48225156f7e6c3a526a310574e674f>`__)
+
+Individual components
+~~~~~~~~~~~~~~~~~~~~~
+
+For changes to individual components, please check:
+
+* renku-ui:
+  `1.0.0-beta4 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/1.0.0-beta4>`__
+  `0.11.14 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/0.11.14>`__
+* renku-notebooks:
+  `0.8.15 <https://github.com/SwissDataScienceCenter/renku-notebooks/releases/tag/0.8.15>`__
+
+Upgrading from 0.8.3
+~~~~~~~~~~~~~~~~~~~~
+
+When upgrading from 0.8.3 the following steps should be taken based on whether you would 
+like to use persistent volumes for user sessions or not:
+
+Use persistent volumes:
+1. Edit the `notebooks.userSessionPersistentVolumes` section of the
+[values.yaml file](https://github.com/SwissDataScienceCenter/renku/blob/master/helm-chart/renku/values.yaml#L527)
+changing the `enabled` flag to true and selecting a storage class to be used with every user session. It is strongly
+recommended to select a storage class with a `Delete` retention policy to avoid the accumulation of persistent volumes
+with every session launch.
+2. Review and modify (if needed) [the disk request options in the values.yaml file]
+(https://github.com/SwissDataScienceCenter/renku/blob/master/helm-chart/renku/values.yaml#L506).
+3. Review and modify (if needed) the [the server defaults in the values.yaml file]
+(https://github.com/SwissDataScienceCenter/renku/blob/master/helm-chart/renku/values.yaml#L479). These will be
+used if a specific server options is omitted in the request to create a session and should be compatible 
+with any matching values in the `serverOptions` section. It also allows an administrator to omit an option
+from the selection menu that is defined in the `serverOptions` section and have renku always use the default from the 
+`serverDefaults` section.
+
+Do not use persistent volumes:
+1. Review and modify (if needed) [the disk request options in the values.yaml file]
+(https://github.com/SwissDataScienceCenter/renku/blob/master/helm-chart/renku/values.yaml#L506). Please note that
+if a user consumes more disk space than they requested (or more than what is set in the `serverDefaults` of the 
+values file) then the user's session will be evicted. This is necessary because if
+a user consumes a lot of space on the node where their session is scheduled k8s starts to evict user sessions on that node
+regardless of whether they are using a lot of disk space or not. This sometimes results in the eviction of multiple
+sessions and not the session that is consuming the most storage resources.
+2. Review and modify (if needed) the [the server defaults in the values.yaml file]
+(https://github.com/SwissDataScienceCenter/renku/blob/master/helm-chart/renku/values.yaml#L479). These will be
+used if a specific server options is omitted in the request to create a session and should be compatible 
+with any matching values in the `serverOptions` section. It also allows an administrator to omit an option
+from the selection menu that is defined in the `serverOptions` section and have renku always use the default from the 
+`serverDefaults` section.
+
 0.8.3
 -----
 

diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
@@ -41,6 +41,7 @@ dockerhub
 dockerignore
 dotfiles
 dropdown
+entrypoint
 extraEnv
 extraEnvFrom
 filesystem
@@ -105,6 +106,7 @@ OpenID
 openssl
 papermill
 parameterizable
+parametrize
 parametrized
 pipenv
 Postgresql

diff --git a/helm-chart/README.rst b/helm-chart/README.rst
@@ -64,6 +64,27 @@ in the `values changelog file <https://github.com/SwissDataScienceCenter/renku/b
 For upgrades that require some steps other than modifying the values files to be
 executed, we add some instructions here.
 
+Upgrading to 0.8.4
+******************
+We have added add a new section called `serverDefaults` to the `values.yaml` for the notebook service. 
+The information in this new `serverDefaults` section is used for any server options that are not specified 
+explicitly when launching a session. This allows a renku admin to leave out a specific option from the 
+`serverOptions` section and apply the value specified in the `serverDefaults` section for all sessions. 
+Please note that the default values specified in the  `serverDefaults` should be available as one of the options 
+in `serverOptions` - if the specific option appears in both sections. The defaults in the `serverOptions` 
+section now only refer to the default selection that is shown to the user in the UI.
+
+This ability to use persistent volumes for user sesssions is also introduced with this release. This is optional and can be enabled in the values
+file for the helm chart. In addition to enabling this feature users have the ability to select the storage class used by the persistent 
+volumes. We strongly recommend that a storage class with a `Delete` reclaim policy is used, otherwise persistent volumes from all user
+sessions will keep accumulating. 
+
+Lastly, unlike previous versions, with 0.8.4 the amount of disk storage will be **strongly enforced**, 
+regardless of whether persistent volumes are used or not. With persistent volumes users will simply run out of space. However, 
+when persistent volumes are not used, going over the amount of storage that a user has requested when starting their session 
+will result in eviction of the k8s pod that runs the session and termination of the session. Therefore, admins are advised 
+to review and set proper options for disk sizes in the `notebooks.serverOptions` portion of the values file.
+
 Upgrading to 0.8.0
 ******************
 We bump the PostgreSQL version from 9.6 to 11 and the GitLab major version from 11 to 13.

diff --git a/helm-chart/renku/Chart.yaml b/helm-chart/renku/Chart.yaml
@@ -3,4 +3,4 @@ appVersion: '1.0'
 description: A Helm chart for the Renku platform
 name: renku
 icon: https://github.com/SwissDataScienceCenter/renku-sphinx-theme/raw/master/renku_sphinx_theme/static/favicon.png
-version: 0.8.3
+version: 0.8.4
diff --git a/helm-chart/renku/values.yaml b/helm-chart/renku/values.yaml
@@ -474,6 +474,15 @@ ui:
 
 # ## Configuration for the notebooks service
 notebooks:
+  ## serverDefaults specifies defaults used when server options parameters
+  ## are not specified in the post request to renku notebooks to start a session.
+  # serverDefaults:
+  #   defaultUrl: /lab
+  #   cpu_request: 0.5
+  #   mem_request: 1G
+  #   disk_request: 1G
+  #   gpu_request: 0
+  #   lfs_auto_fetch: false
   ## serverOptions allow for modifications to the notebook pod resource requests and the UI rendering
   # serverOptions:
   #   defaultUrl:
@@ -494,17 +503,30 @@ notebooks:
   #     type: enum
   #     default: 1G
   #     options: [1G, 2G]
-  #   gpu_request:
+  #   disk_request:
   #     order: 4
+  #     displayName: Amount of disk space requested
+  #     type: enum
+  #     default: "1G"
+  #     options: ["1G", "10G"]
+  #   gpu_request:
+  #     order: 5
   #     displayName: Number of GPUs
   #     type: enum # use "type: int" instead to get a slider
   #     default: 0
   #     options: [0, 0] # use "range: [x, y]" when type is int
   #   lfs_auto_fetch:
-  #     order: 5
+  #     order: 6
   #     displayName: Automatically fetch LFS data
   #     type: boolean
   #     default: false
+  ## Enable the use of persistent volumes for user sessions below. The storage class
+  ## can be omited in which case the default is used. However, we strongly recommend that you use a storage class
+  ## with a `Delete` reclaim policy because otherwise the persistent volumes will keep accumulating
+  ## regardless of how the user sessions are shut down and will require manual intervention to clean them up.
+  # userSessionPersistentVolumes:
+  #   enabled: false
+  #   storageClass:
   ## image registry host
   # imageRegistry:
   ## JupyterHub setup

diff --git a/helm-chart/values.yaml.changelog.md b/helm-chart/values.yaml.changelog.md
@@ -6,6 +6,13 @@ Please follow this convention when adding a new row
 * `<type: NEW|EDIT|DELETE> - *<resource name>*: <details>`
 
 ----
+## Upgrading to Renku 0.8.4
+* NEW *notebooks.serverDefaults* has been added with default values that will be
+used to create a session when specific server options are left out of the request to launch
+a session. See the [values.yaml file](https://github.com/SwissDataScienceCenter/renku/tree/master/helm-chart/renku/values.yaml) for more details.
+* NEW *notebooks.userSessionPersistentVolumes* has been added which enables the use of persistent volumes for user sessions. The use of persistent volumes is disabled by default however. To turn this feature on set the `enabled` flag and specify a storage class that should be used. We strongly recommend using a storage class with a `Delete` retain policy because otherwise the persistent volumes from the user sessions will keep accumulating and will require manual intervention for cleanup. 
+
+
 ## Upgrading to Renku 0.8.0 (breaking changes)
 * NEW/EDIT *postgresql.persistence.existingClaim* will most likely need to be modified in the course of upgrading your postgresql version. See [these instructions](https://github.com/SwissDataScienceCenter/renku/tree/master/helm-chart/utils/postgres_migrations/version_upgrades/README.md)
 * NEW/EDIT/DELETE *gitlab.image.tag* might have to be adjusted as we do a GitLab major version bump in with this release. See [these instructions](https://github.com/SwissDataScienceCenter/renku/tree/master/helm-chart#upgrading-to-080)