Merge pull request #1285 from ScilifelabDataCentre/dev

New release v2.0.0

.github/pull_request_template.md

@@ -1,40 +1,54 @@
-# Description
+> **Before submitting the PR, please go through the sections below and fill in what you can. If there are any items that are irrelevant for the current PR, remove the row. If a relevant option is missing, please add it as an item and add a PR comment informing that the new option should be included into this template.**
 
-Please include the following in this section
+> **All _relevant_ items should be ticked before the PR is merged**
 
-- [ ] Summary of the changes and the related issue
-- [ ] Relevant motivation and context
-- [ ] Any dependencies that are required for this change
+# Description
 
-Fixes # (issue)
+- [ ] Add a summary of the changes and the related issue
+- [ ] Add motivation and context regarding why the change is needed
+- [ ] List / describe any dependencies or other changes required for this change
+- [ ] Fixes [link to issue / Jira issue ID]
 
 ## Type of change
 
-- [ ] Bug fix (non-breaking change which fixes an issue)
-- [ ] New feature (non-breaking change which adds functionality)
-- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
-- [ ] This change requires a documentation update
+- [ ] Documentation
+- [ ] Workflow
+- [ ] Security Alert fix
+- [ ] Bug fix (non-breaking)
+- [ ] New feature (non-breaking)
+- [ ] Breaking change (breaking, will cause existing functionality to not work as expected)
 
 # Checklist:
 
-Please delete options that are not relevant.
-
-- [ ] Any dependent changes have been merged and published in downstream modules
-- [ ] Rebase/merge the branch which this PR is made to
-- [ ] Changes to the database schema: A new migration is included in the PR
-- [ ] Product Owner / Scrum Master: This PR is made to the `master` branch and I have updated the [version](../dds_web/version.py)
-- [ ] I am bumping the major version (e.g. 1.x.x to 2.x.x) and I have made the corresponding changes to the CLI version
-
-## Formatting and documentation
-
-- [ ] I have added a row in the [changelog](../CHANGELOG.md)
-- [ ] The code follows the style guidelines of this project: Black / Prettier formatting
-- [ ] I have performed a self-review of my code
-- [ ] I have commented my code, particularly in hard-to-understand areas
-- [ ] I have made corresponding changes to the documentation
-- [ ] My changes generate no new warnings
-
-## Tests
-
-- [ ] I have added tests that prove my fix is effective or that my feature works
-- [ ] New and existing unit tests pass locally with my changes
+## General
+
+- [ ] [Changelog](../CHANGELOG.md): New row added
+- [ ] Database schema has changed
+  - [ ] A new migration is included in the PR
+  - [ ] The change does not require a migration
+- [ ] Code change
+  - [ ] Self-review of code done
+  - [ ] Comments added, particularly in hard-to-understand areas
+  - [ ] Documentation is updated
+
+## Repository / Releases
+
+- [ ] Blocking PRs have been merged
+- [ ] Rebase / update of branch done
+- [ ] Product Owner / Scrum Master
+  - [ ] The [version](../dds_web/version.py) is updated (PR to `master` branch)
+  - [ ] I am bumping the major version (e.g. 1.x.x to 2.x.x)
+    - [ ] I have made the corresponding changes to the CLI version
+
+## Checks
+
+- [ ] Formatting: Black & Prettier checks pass
+- [ ] CodeQL passes
+- [ ] Tests
+  - [ ] I have added tests for the new code
+  - [ ] The tests pass
+- [ ] Trivy / Snyk:
+  - [ ] There are no new security alerts
+  - [ ] This PR fixes new security alerts
+  - [ ] Security alerts have been dismissed
+  - [ ] PR will be merged with new security alerts; This is why: _Please add a short description here_

.github/workflows/publish_and_trivyscan.yml

@@ -0,0 +1,65 @@
+---
+name: Publish Docker Image and run Trivy Security Scan
+on:
+  pull_request:
+  push:
+    branches:
+      - master
+      - dev
+  release:
+    types: [published]
+jobs:
+  push_to_registry:
+    if: github.repository == 'ScilifelabDataCentre/dds_web'
+    name: Push Docker image to Docker Hub
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the repo
+        uses: actions/checkout@v2
+      - name: Log in to Docker Hub
+        uses: docker/login-action@v2
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - name: Log in to Github Container Repository
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Docker metadata
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: |
+            scilifelabdatacentre/dds-backend
+            ghcr.io/scilifelabdatacentre/dds-backend
+      - name: Ensure lowercase name
+        run: echo IMAGE_REPOSITORY=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]') >> $GITHUB_ENV
+      - name: Build for scan
+        uses: docker/build-push-action@v3
+        with:
+          file: Dockerfiles/backend.Dockerfile
+          context: .
+          push: false
+          tags: ghcr.io/${{ env.IMAGE_REPOSITORY }}:sha-${{ github.sha }}
+      - name: Run Trivy vulnerability scanner
+        uses: aquasecurity/[email protected]
+        with:
+          image-ref: "ghcr.io/${{ env.IMAGE_REPOSITORY }}:sha-${{ github.sha }}"
+          format: "sarif"
+          output: "trivy-results.sarif"
+          severity: "CRITICAL,HIGH"
+      - name: Upload Trivy scan results to Github Security tab
+        uses: github/codeql-action/upload-sarif@v2
+        with:
+          sarif_file: "trivy-results.sarif"
+          category: trivy-build
+      - name: Publish image
+        uses: docker/build-push-action@v3
+        with:
+          file: Dockerfiles/backend.Dockerfile
+          context: .
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

.github/workflows/trivy.yml

@@ -0,0 +1,32 @@
+name: trivy
+on:
+  schedule:
+    # - cron: "*/5 * * * *"
+    - cron: "0 7,13 * * *"
+jobs:
+  scan:
+    permissions:
+      contents: read
+      security-events: write
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Ensure lowercase name
+        run: echo REPOSITORY_OWNER=$(echo ${{ github.repository_owner }} | tr "[:upper:]" "[:lower:]") >> $GITHUB_ENV
+
+      - name: Run Trivy vulnerability scanner
+        uses: aquasecurity/[email protected]
+        with:
+          image-ref: "ghcr.io/${{ env.REPOSITORY_OWNER }}/dds-backend:dev"
+          format: "sarif"
+          output: "trivy-results.sarif"
+          severity: "CRITICAL,HIGH"
+
+      - name: Upload Trivy scan results to GitHub Security tab
+        uses: github/codeql-action/upload-sarif@v2
+        with:
+          sarif_file: "trivy-results.sarif"
+          category: trivy

.gitpod.yml

@@ -17,7 +17,7 @@ tasks:
     command: >
       gp await-port 5000 &&
       echo -e "\033[1;31mUse the dds cli in this terminal window\033[0m\n\033[0;33me.g.: dds auth login\033[0m" &&
-      docker exec -it dds_cli bash
+      docker exec -it dds_cli sh
 
 ports:
   - port: 5000 # backend

CHANGELOG.md

@@ -140,3 +140,14 @@ Please add a _short_ line describing the PR you make, if the PR implements a spe
 - Add storage usage information in the Units listing table for Super Admin ([#1264](https://github.com/ScilifelabDataCentre/dds_web/pull/1264))
 - New endpoint for setting project as busy / not busy ([#1266](https://github.com/ScilifelabDataCentre/dds_web/pull/1266))
 - Check for if project busy before status change ([#1266](https://github.com/ScilifelabDataCentre/dds_web/pull/1266))
+- Bug fix: Default timestamps fixed ([#1271](https://github.com/ScilifelabDataCentre/dds_web/pull/1271))
+- Change docker image to alpine ([#1272](https://github.com/ScilifelabDataCentre/dds_web/pull/1272))
+- Added trivy when publishing to dockerhub ([#1276](https://github.com/ScilifelabDataCentre/dds_web/pull/1276))
+- Bug fix: Cost value displayed by the --usage flag fixed ([#1274](https://github.com/ScilifelabDataCentre/dds_web/pull/1274))
+
+## Sprint (2022-09-16 - 2022-09-30)
+
+- New endpoint: SendMOTD - send important information to users ([#1283](https://github.com/ScilifelabDataCentre/dds_web/pull/1283))
+- New table: `Maintenance`, for keeping track of DDS maintenance mode ([#1284](https://github.com/ScilifelabDataCentre/dds_web/pull/1284))
+- New endpoint: SetMaintenance - set maintenance mode to on or off ([#1286](https://github.com/ScilifelabDataCentre/dds_web/pull/1286))
+- New endpoint: AnyProjectsBusy - check if any projects are busy in DDS ([#1288](https://github.com/ScilifelabDataCentre/dds_web/pull/1288))

CONTRIBUTING.md

@@ -1,5 +1,51 @@
-**In progress**
+If you would like to contribute to this project...
 
-1. Steps for creating good issues or pull requests.
-2. Links to external documentation, mailing lists, or a code of conduct.
-3. Community and behavioral expectations.
+**Thank you, any suggestions and / or contributions are very appreciated**
+
+If you would like to suggest a change, please go to the issues tab and fill out and issue. If you would like to contribute with code, follow the steps below:
+
+1. Fork this repository
+2. Make a **draft** pull request. There is a PR template that will guide you through what information we want, checks that need to pass and items you need to tick before we will review your PR.
+3. When you have followed the guide in the template and all items are filled in, ticked etc, mark your PR as _ready for review_
+
+We will get back to you as soon as possible with possible changes, comments, suggestions etc.
+
+All information about this project can be found in this repository (e.g. at https://github.com/ScilifelabDataCentre/dds_web/tree/dev/doc). If there is some information missing, don't hestitate to inform us by _creating an issue_.
+
+Contributions, suggestions or comments regarding the corresponding CLI should be directed [here](https://github.com/ScilifelabDataCentre/dds_web).
+
+## Code of Conduct
+
+### Acceptable behavior
+
+Examples of behavior that contributes to creating a positive environment include:
+
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+- The use of sexualized language or imagery and unwelcome sexual attention or advances
+- Trolling, insulting/derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or electronic address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a professional setting
+
+### Our responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+### Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project maintainer using any of the private contact addresses. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+---
+
+_This was adapted from https://github.com/dec0dOS/amazing-github-template/blob/main/docs/CODE_OF_CONDUCT.md_

Dockerfiles/backend.Dockerfile

@@ -3,10 +3,21 @@
 #############################
 
 # Set official image -- parent image
-FROM python:latest as base
+FROM python:3.10-alpine as base
 
-# Install some necessary systems packages
-RUN apt-get update && apt-get upgrade -y
+# Update and upgrade
+RUN apk update && apk upgrade
+
+# Install required dependencies...
+# ...Some for build
+RUN apk add g++ gcc musl-dev libffi-dev
+
+# ...Some for requirements
+RUN apk add jpeg-dev zlib-dev libjpeg
+
+# Set time zone
+RUN apk add tzdata
+ENV TZ="UCT"
 
 # Copy the content to a code folder in container
 COPY ./requirements.txt /code/requirements.txt
@@ -25,7 +36,7 @@ ENV PYTHONPATH /code
 ###################
 FROM base as test
 RUN pip3 install -r /code/tests/requirements-test.txt
-RUN apt-get install -y mariadb-client
+RUN apk add mariadb-client
 
 ###################
 ## BUILD FRONTEND

Dockerfiles/cli.Dockerfile

@@ -1,6 +1,12 @@
-FROM python:latest as base
+# Set official image
+FROM python:3.10-alpine as base
 
-RUN apt-get update && apt-get upgrade -y && apt-get install git
+# Update and upgrade
+RUN apk update && apk upgrade
+
+# Install dependencies for build and requirements 
+RUN apk add g++ gcc musl-dev libffi-dev
+RUN apk add --no-cache git
 RUN git clone https://github.com/ScilifelabDataCentre/dds_cli /code
 WORKDIR /code
 

README.md

@@ -117,7 +117,7 @@ Requires that dds_cli is checked out in `../dds_cli` (otherwise adapt the volume
 2. Inject into the `dds_cli` container:
 
    ```bash
-   docker exec -it dds_cli /bin/bash
+   docker exec -it dds_cli /bin/sh
    ```
 
 Then you can freely use the dds cli component against the local development setup in the active CLI.
@@ -237,7 +237,7 @@ docker-compose -f docker-compose.yml -f tests/docker-compose-test-interactive.ym
 Then in a new terminal, shell into the container and run pytest:
 
 ```bash
-docker exec -it dds_backend /bin/bash
+docker exec -it dds_backend /bin/sh
 ```
 
 ```bash