Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(deployment): Preview loculus docs pages #2452

Merged
merged 9 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .github/workflows/docs-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: docs-image
on:
pull_request:
push:
branches:
- main
workflow_dispatch:
inputs:
build_arm:
type: boolean
description: "Build for ARM as well"
default: false
required: false
workflow_call:
inputs:
build_arm:
type: boolean
description: "Build for ARM as well"
default: false
required: false
env:
DOCKER_IMAGE_NAME: ghcr.io/loculus-project/docs
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
BUILD_ARM: ${{ github.event.inputs.build_arm || inputs.build_arm || github.ref == 'refs/heads/main' }}
sha: ${{ github.event.pull_request.head.sha || github.sha }}
concurrency:
group: ci-${{ github.ref == 'refs/heads/main' && github.run_id || github.ref }}-docs-${{github.event.inputs.build_arm}}
cancel-in-progress: true
jobs:
docs-docker:
name: Build Docs Docker Image
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
packages: write
checks: read
steps:
- name: Shorten sha
run: echo "sha=${sha::7}" >> $GITHUB_ENV
- uses: actions/checkout@v4
- name: Generate files hash
id: files-hash
run: |
DIR_HASH=$(echo -n ${{ hashFiles('docs/**', '.github/workflows/docs-image.yml') }})
echo "DIR_HASH=$DIR_HASH${{ env.BUILD_ARM == 'true' && '-arm' || '' }}" >> $GITHUB_ENV
- name: Setup Docker metadata
id: dockerMetadata
uses: docker/metadata-action@v5
with:
images: ${{ env.DOCKER_IMAGE_NAME }}
tags: |
type=raw,value=${{ env.DIR_HASH }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=${{ env.BRANCH_NAME }}
type=raw,value=commit-${{ env.sha }}
type=raw,value=${{ env.BRANCH_NAME }}-arm,enable=${{ env.BUILD_ARM }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Check if image exists
id: check-image
run: |
EXISTS=$(docker manifest inspect ${{ env.DOCKER_IMAGE_NAME }}:${{ env.DIR_HASH }} > /dev/null 2>&1 && echo "true" || echo "false")
echo "CACHE_HIT=$EXISTS" >> $GITHUB_ENV
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Get node version build arg
if: env.CACHE_HIT == 'false'
run: |
NODE_VERSION=$(cat docs/.nvmrc | tr -cd [:digit:].)
echo "NODE_VERSION=$NODE_VERSION" >> $GITHUB_ENV
- name: Build and push image
if: env.CACHE_HIT == 'false'
uses: docker/build-push-action@v6
with:
context: ./docs
push: true
tags: ${{ steps.dockerMetadata.outputs.tags }}
cache-from: type=gha,scope=docs-${{ github.ref }}
cache-to: type=gha,mode=max,scope=docs-${{ github.ref }}
platforms: ${{ env.BUILD_ARM == 'true' && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
build-args: NODE_VERSION=${{ env.NODE_VERSION }}
- name: Retag and push existing image if cache hit
if: env.CACHE_HIT == 'true'
run: |
TAGS=(${{ steps.dockerMetadata.outputs.tags }})
for TAG in "${TAGS[@]}"; do
docker buildx imagetools create --tag $TAG ${{ env.DOCKER_IMAGE_NAME }}:${{ env.DIR_HASH }}
done
17 changes: 17 additions & 0 deletions docs/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM node:20-alpine
WORKDIR /app

COPY package*.json ./

RUN npm ci

COPY . .

RUN npm run build
RUN mkdir -p server && mv dist server/loculus

RUN npm install -g http-server

EXPOSE 8080

CMD ["http-server", "server", "-p", "8080"]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/src/content/docs/introduction/system-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Loculus has a modular architecture and consists of several sub-services:
- **Website:** The frontend application of Loculus accesses the APIs of the backend server and LAPIS. It uses the backend server for everything related to data submission and LAPIS for searching and downloading released data. For logins and registrations, users are redirected to Keycloak.
- **Preprocessing pipeline(s):** A preprocessing pipeline fetches [unprocessed/user-submitted data](../glossary#unprocessed-data) from the backend server, processes them (which usually includes cleaning, alignment and adding annotations), and sends [processed data](../glossary#processed-data) back to the backend server. The pipeline contains [organism](../glossary#organism)-specific logic, thus, there is a separate pipeline for each organism. We maintain a customizeable preprocessing pipeline that uses [Nextclade](https://github.com/nextstrain/nextclade) for alignment, quality checks and annotations but it is easy to write a new one by following the [preprocessing pipeline specifications](https://github.com/loculus-project/loculus/blob/main/preprocessing/specification.md).

![Architecture overview](../../../../../backend/docs/plantuml/architectureOverview.svg)
![Architecture overview](./architectureOverview.svg)

## Motivation

Expand Down
72 changes: 72 additions & 0 deletions kubernetes/loculus/templates/docs-preview.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{{- $dockerTag := include "loculus.dockerTag" .Values }}
{{- $docsHost := printf "docs%s%s" .Values.subdomainSeparator .Values.host }}
{{- if .Values.previewDocs }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: loculus-docs
annotations:
argocd.argoproj.io/sync-options: Replace=true
reloader.stakater.com/auto: "true"
spec:
replicas: 1
selector:
matchLabels:
app: loculus
component: docs
template:
metadata:
labels:
app: loculus
component: docs
spec:
containers:
- name: docs
image: "ghcr.io/loculus-project/docs:{{ $dockerTag }}"
imagePullPolicy: Always
resources:
requests:
memory: "200Mi"
{{ include "loculus.standardCPURequest" $ }}
limits:
memory: "1Gi"
ports:
- containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
name: loculus-docs
spec:
selector:
app: loculus
component: docs
ports:
- protocol: TCP
port: 80
targetPort: 8080

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: loculus-docs-ingress
spec:
rules:
- host: "{{ $docsHost }}"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: loculus-docs
port:
number: 80
tls:
- hosts:
- "{{ $docsHost }}"
---
{{- end }}
2 changes: 1 addition & 1 deletion kubernetes/loculus/values_e2e_and_dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ secrets:
createTestAccounts: true
backendExtraArgs:
- "--loculus.debug-mode=true"
reduceResourceRequest: true
reduceResourceRequest: true
Loading
Loading