Merge pull request #1409 from ScilifelabDataCentre/dev

New release: 2.2.62
ScilifelabDataCentre · May 22, 2023 · 3c8a734 · 3c8a734
2 parents 3cc3608 + a906d52
commit 3c8a734
Show file tree

Hide file tree

Showing 27 changed files with 810 additions and 170 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -1,34 +1,12 @@
-<!--
-> **Before _submitting_ PR:**
->
-> - Fill in and tick fields
-> - _Remove all rows_ that are not relevant for the current PR
->   - Revelant option missing? Add it as an item and add a PR comment informing that the new option should be included into this template.
->
-> **Before _merging_ PR:**
->
-> _Tick all relevant items._
--->
-
-## **1. This PR contains the following changes...**
-
-_Add a summary of the changes and the related issue._
-
-## **2. The following additional changes are required for this to work**
-
-_Add information on additional changes required for the PR changes to work, both locally and in the deployments._
+## Before submitting this PR
 
-> E.g. Does the deployment setup need anything for this to work?
+1. **Description:** _Add a summary of the changes in this PR and the related issue._
+2. **Jira task / GitHub issue:** _Link to the github issue or add the Jira task ID here._
+3. **How to test:** _Add information on how someone could manually test this functionality. As detailed as possible._
+4. **Type of change:** [_Check the relevant boxes in the section below_](#what-type-of-changes-does-the-pr-contain)
+5. **Add docstrings and comments to code**, _even if_ you personally think it's obvious.
 
-## **3. The PR fixes the following GitHub issue / Jira task**
-
-<!-- Comment out the item which does not apply here.-->
-
-- [ ] GitHub issue (link):
-- [ ] Jira task (ID, `DDS-xxxx`):
-- [ ] The PR does not fix a specific GitHub issue or Jira task
-
-## **4. What _type of change(s)_ does the PR contain?**
+## What _type of change(s)_ does the PR contain?
 
 <!--
 - "Breaking": The change will cause existing functionality to not work as expected.
@@ -49,15 +27,9 @@ _Add information on additional changes required for the PR changes to work, both
 - [ ] Tests **(only)**
 - [ ] Workflow
 
-## **5. Checklist**
-
-<!-- Comment out the items which do not apply here.-->
-
-### **Always**
+## Checklist
 
-<!-- Always go through the following items. -->
-
-- [Changelog](../CHANGELOG.md)
+- [Sprintlog](../SPRINTLOG.md)
   - [ ] Added
   - [ ] Not needed (E.g. PR contains _only_ tests)
 - Rebase / Update / Merge _from_ base branch (the branch from which the current is forked)
@@ -67,35 +39,11 @@ _Add information on additional changes required for the PR changes to work, both
   - [ ] Merged
   - [ ] No blocking PRs
 - PR to `master` branch
-  - [ ] Yes: Go to the section [PR to master](#pr-to-master)
+  - [ ] Yes: Read [the release instructions](../doc/procedures/new_release.md)
+    - [ ] I have followed steps 1-7.
   - [ ] No
 
-### If PR consists of **code change(s)**
-
-<!-- If the PR contains code changes, the following need to be checked.-->
-
-- Self review
-  - [ ] Done
-- Comments, docstrings, etc
-  - [ ] Added / Updated
-- Documentation
-  - [ ] Updated
-  - [ ] Update not needed
-
-### If PR is to **master**
-
-<!-- Is your PR to the master branch? The following items need to be checked off. -->
-
-- [ ] I have followed steps 1-5 in [the release instructions](../doc/procedures/new_release.md)
-- [ ] I am bumping the major version (e.g. 1.x.x to 2.x.x)
-- [ ] I have made the corresponding changes to the CLI version
-
-**Is this version _backward compatible?_**
-
-- [ ] Yes: The code works together with `dds_cli/master` branch
-- [ ] No: The code **does not** entirely / at all work together with the `dds_cli/master` branch. _Please add detailed and clear information about the broken features_
-
-## **6. Actions / Scans**
+## Actions / Scans
 
 <!-- Go through all checkboxes. All actions must pass before merging is allowed.-->
 

diff --git a/.github/workflows/docker-compose-tests.yml b/.github/workflows/docker-compose-tests.yml
@@ -19,7 +19,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v3
 
       - name: Run tests against database container
         run: docker-compose -f docker-compose.yml -f tests/docker-compose-test.yml up --build --exit-code-from backend

diff --git a/.github/workflows/prettier.yml b/.github/workflows/prettier.yml
@@ -9,9 +9,9 @@ jobs:
   Prettier:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
-      - uses: actions/setup-node@v2
+      - uses: actions/setup-node@v3
 
       - name: Install Prettier
         run: npm install -g prettier

diff --git a/.github/workflows/python-black.yml b/.github/workflows/python-black.yml
@@ -12,5 +12,5 @@ jobs:
   lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - uses: psf/black@stable
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -0,0 +1,21 @@
+Changelog
+==========
+
+.. _2.2.62:
+
+2.2.62 - 2023-03-20
+~~~~~~~~~~~~~~~~~~~~
+
+- Added this version changelog. 
+- Fixed bugs:
+    - Percentage calculation in cronjob for monitoring unit usage has been fixed; Warning email will be sent to Data Centre when a units data usage on DDS reaches 80% of their allocated quota.
+    - User is redirected to same page and message when attempting to reset their password, independent on if the email is registered to an active user or not.
+    - Non-latin1 encodable characters are not allowed in passwords.
+- The _default_ settings for the Argon2 password hashing function have been changed to increase the complexity and security.
+
+.. _earlier-versions:
+
+Earlier versions
+~~~~~~~~~~~~~~~~~
+
+Please see `the release page on GitHub <https://github.com/ScilifelabDataCentre/dds_web/releases>`_ for detailed information about the changes in each release.
diff --git a/CHANGELOG.md → SPRINTLOG.md b/CHANGELOG.md → SPRINTLOG.md
@@ -2,7 +2,7 @@
 
 Please add a _short_ line describing the PR you make, if the PR implements a specific feature or functionality, or refactor. Not needed if you add very small and unnoticable changes. Not needed when PR includes _only_ tests for already existing feature.
 
-## Sprint (2022-02-09 - 2022-02-23)
+## 2022-02-09 - 2022-02-23
 
 - Secure operations that require cryptographic keys are protected for each user with the user's password ([#889](https://github.com/ScilifelabDataCentre/dds_web/pull/889))
 - Implemented the functionality to add project to the invites of a new user as outlined in [issue 887](https://github.com/scilifelabdatacentre/dds_web/issues/887) ([PR888](https://github.com/ScilifelabDataCentre/dds_web/pull/888)).
@@ -20,7 +20,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Changed `is_sensitive` to `non_sensitive` and set to default False ([#913](https://github.com/ScilifelabDataCentre/dds_web/pull/913))
 - Rearrangement and clean up of the token ([910](https://github.com/ScilifelabDataCentre/dds_web/pull/910))
 
-## Sprint (2022-02-23 - 2022-03-09)
+## 2022-02-23 - 2022-03-09
 
 - Add landing page after password reset ([#931](https://github.com/ScilifelabDataCentre/dds_web/pull/931))
 - Add endpoint for health check (intended for readinessProbe) ([#933](https://github.com/ScilifelabDataCentre/dds_web/pull/933))
@@ -42,7 +42,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Fix regex for listing and deleting files [#1029](https://github.com/scilifelabdatacentre/dds_web/issues/1029)
 - Hides the "Size" and "total_size" variables according to the role and project status ([#1032](https://github.com/ScilifelabDataCentre/dds_web/pull/1032)).
 
-## Sprint (2022-03-09 - 2022-03-23)
+## 2022-03-09 - 2022-03-23
 
 - Introduce a separate error message if someone tried to add an unit user to projects individually. ([#1039](https://github.com/ScilifelabDataCentre/dds_web/pull/1039))
 - Catch KeyNotFoundError when user tries to give access to a project they themselves do not have access to ([#1045](https://github.com/ScilifelabDataCentre/dds_web/pull/1045))
@@ -63,7 +63,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Catch OperationalError if there is a database malfunction in `files.py` ([#1089](https://github.com/ScilifelabDataCentre/dds_web/pull/1089))
 - Switched the validation for the principal investigator from string to email ([#1084](https://github.com/ScilifelabDataCentre/dds_web/pull/1084)).
 
-## Sprint (2022-03-23 - 2022-04-06)
+## 2022-03-23 - 2022-04-06
 
 - Add link in navbar to the installation documentation ([#1112](https://github.com/ScilifelabDataCentre/dds_web/pull/1112))
 - Change from apscheduler to flask-apscheduler - solves the app context issue ([#1109](https://github.com/ScilifelabDataCentre/dds_web/pull/1109))
@@ -73,7 +73,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Cronjob: Scheduled task for changing project status from Expired to Archived ([#1115](https://github.com/ScilifelabDataCentre/dds_web/pull/1115))
 - Add a Flask command for finding and deleting "lost files" (files that exist only in db or s3) ([#1124](https://github.com/ScilifelabDataCentre/dds_web/pull/1124))
 
-## Sprint (2022-04-06 - 2022-04-20)
+## 2022-04-06 - 2022-04-20
 
 - New endpoint for adding a message of the day to the database ([#1136](https://github.com/ScilifelabDataCentre/dds_web/pull/1136))
 - Patch: Custom error for PI email validation ([#1146](https://github.com/ScilifelabDataCentre/dds_web/pull/1146))
@@ -82,13 +82,13 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Checkbox in registration form and policy to agree to ([#1151](https://github.com/ScilifelabDataCentre/dds_web/pull/1151))
 - Patch: Add checks for valid public_id when creating new unit to avoid bucket name errors ([#1154](https://github.com/ScilifelabDataCentre/dds_web/pull/1154))
 
-## Sprint (2022-04-20 - 2022-05-04)
+## 2022-04-20 - 2022-05-04
 
 - Renamed `api/units.py` to `api/superadmin_only.py` to reflect that it's only Super Admin accessible resources ([#1159](https://github.com/ScilifelabDataCentre/dds_web/pull/1159))
 - Add unit tests for the "set_available_to_expired" and "set_expired_to_archived" functions ([#1158](https://github.com/ScilifelabDataCentre/dds_web/pull/1158))
 - DC Branding: SciLifeLab logo + "Data Delivery System" in nav bar & DC branding in footer ([#1160](https://github.com/ScilifelabDataCentre/dds_web/pull/1160))
 
-## Sprint (2022-05-04 - 2022-05-18)
+## 2022-05-04 - 2022-05-18
 
 - `adr-tools` to document architecture decisions ([#1161](https://github.com/ScilifelabDataCentre/dds_web/pull/1161))
 - Bug: API returning float again and CLI `--size` flag works again ([#1162](https://github.com/ScilifelabDataCentre/dds_web/pull/1162))
@@ -100,20 +100,20 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - New `add-missing-buckets` argument option to the `lost-files` flask command ([#1174](https://github.com/ScilifelabDataCentre/dds_web/pull/1174))
 - Bug: Corrected `lost-files` logic and message ([#1176](https://github.com/ScilifelabDataCentre/dds_web/pull/1176))
 
-## Sprint (2022-05-18 - 2022-06-01)
+## 2022-05-18 - 2022-06-01
 
 - Allow all characters but unicode (e.g. emojis) in project description ([#1178](https://github.com/ScilifelabDataCentre/dds_web/pull/1178))
 - Cronjob: Scheduled task for monthly usage data ([#1181](https://github.com/ScilifelabDataCentre/dds_web/pull/1181))
 - New tests for `dds_web/__init__.py` ([#1185](https://github.com/ScilifelabDataCentre/dds_web/pull/1185))
 - New tests for `dds_web/utils.py` ([#1188](https://github.com/ScilifelabDataCentre/dds_web/pull/1188))
 - Removed FontAwesome from web ([#1192](https://github.com/ScilifelabDataCentre/dds_web/pull/1192))
 
-## Sprint (2022-06-01 - 2022-06-15)
+## 2022-06-01 - 2022-06-15
 
 - Change FontAwesome source link to own license ([#1194](https://github.com/ScilifelabDataCentre/dds_web/pull/1194))
 - Display MOTD on web ([#1196](https://github.com/ScilifelabDataCentre/dds_web/pull/1196))
 
-## Sprint (2022-06-15 - 2022-06-29)
+## 2022-06-15 - 2022-06-29
 
 - Get MOTD from API ([#1198](https://github.com/ScilifelabDataCentre/dds_web/pull/1198))
 - New endpoint for listing all users ([#1204](https://github.com/ScilifelabDataCentre/dds_web/pull/1204))
@@ -126,7 +126,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Raise AccessDeniedError with message when token specified but user not existent ([#1235](https://github.com/ScilifelabDataCentre/dds_web/pull/1235))
 - Display multiple MOTDS ([#1212](https://github.com/ScilifelabDataCentre/dds_web/pull/1212))
 
-## Sprint (2022-08-18 - 2022-09-02)
+## 2022-08-18 - 2022-09-02
 
 - Allow Super Admins to deactivate user 2FA via authenticator app ([#1247](https://github.com/ScilifelabDataCentre/dds_web/pull/1247))
 - Get troubleshooting document from Confluence ([#1244](https://github.com/ScilifelabDataCentre/dds_web/pull/1244))
@@ -135,7 +135,7 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Technical Overview moved to repository ([#1250](https://github.com/ScilifelabDataCentre/dds_web/pull/1253))
 - Troubleshooting document moved to repository and buttons added to web to link and download ([#1255](https://github.com/ScilifelabDataCentre/dds_web/pull/1255))
 
-## Sprint (2022-09-02 - 2022-09-16)
+## 2022-09-02 - 2022-09-16
 
 - Add storage usage information in the Units listing table for Super Admin ([#1264](https://github.com/ScilifelabDataCentre/dds_web/pull/1264))
 - New endpoint for setting project as busy / not busy ([#1266](https://github.com/ScilifelabDataCentre/dds_web/pull/1266))
@@ -145,19 +145,19 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Added trivy when publishing to dockerhub ([#1276](https://github.com/ScilifelabDataCentre/dds_web/pull/1276))
 - Bug fix: Cost value displayed by the --usage flag fixed ([#1274](https://github.com/ScilifelabDataCentre/dds_web/pull/1274))
 
-## Sprint (2022-09-16 - 2022-09-30)
+## 2022-09-16 - 2022-09-30
 
 - New endpoint: SendMOTD - send important information to users ([#1283](https://github.com/ScilifelabDataCentre/dds_web/pull/1283))
 - New table: `Maintenance`, for keeping track of DDS maintenance mode ([#1284](https://github.com/ScilifelabDataCentre/dds_web/pull/1284))
 - New endpoint: SetMaintenance - set maintenance mode to on or off ([#1286](https://github.com/ScilifelabDataCentre/dds_web/pull/1286))
 - New endpoint: AnyProjectsBusy - check if any projects are busy in DDS ([#1288](https://github.com/ScilifelabDataCentre/dds_web/pull/1288))
 
-## Sprint (2022-09-30 - 2022-10-14)
+## 2022-09-30 - 2022-10-14
 
 - Bug fix: Fix the Invite.projects database model ([#1290](https://github.com/ScilifelabDataCentre/dds_web/pull/1290))
 - New endpoint: ListInvites - list invites ([#1294](https://github.com/ScilifelabDataCentre/dds_web/pull/1294))
 
-## Sprint (2022-10-14 - 2022-10-28)
+## 2022-10-14 - 2022-10-28
 
 - Limit projects listing to active projects only; a `--show-all` flag can be used for listing all projects, active and inactive ([#1302](https://github.com/ScilifelabDataCentre/dds_web/pull/1302))
 - Return name of project creator from UserProjects ([#1303](https://github.com/ScilifelabDataCentre/dds_web/pull/1303))
@@ -167,30 +167,30 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Bug: Return instead of project creator if user has been deleted ([#1311](https://github.com/ScilifelabDataCentre/dds_web/pull/1311))
 - New endpoint: ProjectInfo - display project information ([#1310](https://github.com/ScilifelabDataCentre/dds_web/pull/1310))
 
-## Sprint (2022-11-11 - 2022-11-25)
+## 2022-11-11 - 2022-11-25
 
 - Link to "How do I get my user account?" from the login form ([#1318](https://github.com/ScilifelabDataCentre/dds_web/pull/1318))
 
-## Sprint (2022-11-25 - 2022-12-09)
+## 2022-11-25 - 2022-12-09
 
 - Changed support email ([#1324](https://github.com/ScilifelabDataCentre/dds_web/pull/1324))
 - Allow Super Admin login during maintenance ([#1333](https://github.com/ScilifelabDataCentre/dds_web/pull/1333))
 
-## Sprint (2022-12-09 - 2023-01-09) - Longer sprint due to Christmas
+## 2022-12-09 - 2023-01-09: Longer sprint due to Christmas
 
 - Dependency: Bump `certifi` due to CVE-2022-23491 ([#1337](https://github.com/ScilifelabDataCentre/dds_web/pull/1337))
 - Dependency: Bump `jwcrypto` due to CVE-2022-3102 ([#1339](https://github.com/ScilifelabDataCentre/dds_web/pull/1339))
 - Cronjob: Get number of units and users for reporting ([#1324](https://github.com/ScilifelabDataCentre/dds_web/pull/1335))
 - Add ability to change project information via ProjectInfo endpoint ([#1331](https://github.com/ScilifelabDataCentre/dds_web/pull/1331))
 - Fix the reporting file path ([1345](https://github.com/ScilifelabDataCentre/dds_web/pull/1345))
 
-## Sprint (2023-01-09 - 2023-01-20)
+## 2023-01-09 - 2023-01-20
 
 - Refactoring: Move flask commands to own module `commands.py` ([#1351](https://github.com/ScilifelabDataCentre/dds_web/pull/1351))
 - Workflow: Scan with Snyk on PR and schedule ([#1349](https://github.com/ScilifelabDataCentre/dds_web/pull/1349))
 - Flask command (cronjob): Monitor unit usage and warn if above level ([#1350](https://github.com/ScilifelabDataCentre/dds_web/pull/1350))
 
-## Sprint (2023-01-20 - 2023-02-03)
+## 2023-01-20 - 2023-02-03
 
 - Workflow: Do not publish to DockerHub anymore ([#1357](https://github.com/ScilifelabDataCentre/dds_web/pull/1357))
 - Refactoring: move cronjobs previously handled by APScheduler to flask commands ([#1355](https://github.com/ScilifelabDataCentre/dds_web/pull/1355))
@@ -201,11 +201,11 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Bug: Add value to `Unit.warning_level` for existing units ([#1378](https://github.com/ScilifelabDataCentre/dds_web/pull/1379))
 - Workflow: Add option to run trivy on dev and master branches manually ([#1380](https://github.com/ScilifelabDataCentre/dds_web/pull/1380))
 
-## Sprint (2023-02-03 - 2023-02-17)
+## 2023-02-03 - 2023-02-17
 
 - Workflow: Scan with yamllint ([#1385](https://github.com/ScilifelabDataCentre/dds_web/pull/1385))
 
-## Sprint (2023-02-17 - 2023-03-03)
+## 2023-02-17 - 2023-03-03
 
 - Move Rotating log file maximum size and backup count to config variables ([#1388](https://github.com/ScilifelabDataCentre/dds_web/pull/1388))
 - Workflow: Scan branch with trivy ([#1377](https://github.com/ScilifelabDataCentre/dds_web/pull/1377))
@@ -220,3 +220,24 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Workflow: Schedule trivy scan for both dev images and latest release ([#1392](https://github.com/ScilifelabDataCentre/dds_web/pull/1392))
 - Improve logging of delete-invites flask command ([#1386](https://github.com/ScilifelabDataCentre/dds_web/pull/1386))
 - Workflow: Schedule trivy scan for dev and latest separately ([#1395](https://github.com/ScilifelabDataCentre/dds_web/pull/1395))
+
+# 2023-03-03 - 2023-03-17
+
+- PR template restructured ([#1403](https://github.com/ScilifelabDataCentre/dds_web/pull/1403))
+- Only allow latin1-encodable usernames and passwords ([#1402](https://github.com/ScilifelabDataCentre/dds_web/pull/1402))
+- Bug: Corrected calculation of used storage space in `monitor_usage` command ([#1404](https://github.com/ScilifelabDataCentre/dds_web/pull/1404))
+- Config: Define Argon2 settings in `config.py` and use same settings (as default) during password-hashing as in key-derivation for private key access ([#1406](https://github.com/ScilifelabDataCentre/dds_web/pull/1406))
+- Bug: Display same message during password reset independent on if the email address is registered to an account or not ([#1408](https://github.com/ScilifelabDataCentre/dds_web/pull/1408))
+
+# 2023-03-17 - 2023-03-31
+
+_Nothing merged in CLI during this sprint_
+
+# 2023-03-31 - 2023-04-14
+
+_Nothing merged in CLI during this sprint_
+
+# 2023-04-14 - 2023-04-28
+
+- Documentation: Minor update of Technical Overview ((#1411)[https://github.com/ScilifelabDataCentre/dds_web/pull/1411])
+- Documentation: Account roles and their permissions ((#1412)[https://github.com/ScilifelabDataCentre/dds_web/pull/1412])