Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release version 0.7.29. #1431

Merged
merged 25 commits into from
Aug 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a8a5ec5
Prepare for next development cycle for version 0.7.29 [full build]
Jun 29, 2023
21bca35
Migrate Pluto from Netflix Zuul to Spring Cloud Gateway.
ewelinagr Jun 19, 2023
3a13349
Fix returning static content on html requests by the gateway.
ewelinagr Jul 5, 2023
2dbb27f
Fix logout endpoint in Pluto.
ewelinagr Jul 5, 2023
624223f
Spring Cloud Gateway setup improvements.
ewelinagr Jul 5, 2023
7048192
Fix workspace page styling - remove field width limit.
ewelinagr Jul 6, 2023
b3d37d3
Merge pull request #1422 from thehyve/workspace-page-styling
ewelinagr Jul 7, 2023
d354f8d
Upgrade gradle to 7.6.1. and add saturn dependency updates.
ewelinagr Jul 6, 2023
f7b43c2
Merge pull request #1423 from thehyve/saturn-updates
ewelinagr Aug 1, 2023
149166f
Merge pull request #1420 from thehyve/spring-cloud-gateway
ewelinagr Aug 1, 2023
21205ef
Fix Saturn docker image after Gradle update.
ewelinagr Aug 2, 2023
5e533a9
Fix missing xargs library installation in Dockerfile.
ewelinagr Aug 2, 2023
567f97a
Fix Keycloak returning secure header related error.
ewelinagr Aug 8, 2023
276f322
Fix "Connection prematurely closed BEFORE response" error.
ewelinagr Aug 8, 2023
31266eb
Add "admin" to possible roles.
ewelinagr Aug 8, 2023
bf7b484
increase checkout time to prevent pipeline run errors
frankyhollywood Aug 9, 2023
a5093fa
Merge pull request #1428 from thehyve/default-roles
ewelinagr Aug 10, 2023
daff336
Merge pull request #1427 from thehyve/spring-cloud-gateway-fixes
ewelinagr Aug 10, 2023
f45d23c
Fix folder upload issue (#1426)
frankyhollywood Aug 10, 2023
425d899
Increase checkpoint timeout for postgres k8s
frankyhollywood Aug 16, 2023
2283e22
Updated kubernets postgres configuration
frankyhollywood Aug 22, 2023
569840f
Merge pull request #1429 from thehyve/postgres-checkout
frankyhollywood Aug 22, 2023
190ce51
Improve styling of permission card not always fully visible.
ewelinagr Aug 24, 2023
46aed1d
Merge pull request #1430 from thehyve/permission-card-styling
ewelinagr Aug 25, 2023
5e66302
Prepare release 0.7.29.
ewelinagr Aug 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2500,7 +2500,7 @@ Create DNS records for the ``keycloak.example.com``, ``fairspace.example.com`` a
# List available fairspace chart versions
~/bin/helm/helm search repo --versions fairspace/fairspace
# Fetch the fairspace chart
~/bin/helm/helm pull fairspace/fairspace --version 0.7.28
~/bin/helm/helm pull fairspace/fairspace --version 0.7.29
----

===== Deploy Keycloak
Expand Down Expand Up @@ -2586,7 +2586,7 @@ Create a new deployment (called _release_ in helm terminology) and
install the Fairspace chart:
[source, shell]
----
~/bin/helm/helm install fairspace-new fairspace/fairspace --version 0.7.28 --namespace fairspace-new \
~/bin/helm/helm install fairspace-new fairspace/fairspace --version 0.7.29 --namespace fairspace-new \
-f /path/to/values.yaml --set-file saturn.vocabulary=/path/to/vocabulary.ttl --set-file saturn.views=/path/to/views.yaml
----
You can pass values files with ``-f`` and provide a file for a specified
Expand Down Expand Up @@ -2723,7 +2723,7 @@ jupyterhub:
To update a deployment using a new chart:
[source, shell]
----
~/bin/helm/helm upgrade fairspace-new fairspace-0.7.28.tgz
~/bin/helm/helm upgrade fairspace-new fairspace-0.7.29.tgz
----
With ``helm upgrade`` you can also pass new values files with ``-f``
and pass files with ``--set-file`` as for ``helm install``.
Expand Down Expand Up @@ -3004,9 +3004,8 @@ Pluto keeps track of user sessions, handles the OIDC authentication flow (author
JWT tokens (validation, refreshing) and redirects to back-end applications.

Pluto is a link:https://spring.io/projects/spring-boot[Java Spring Boot] application,
using link:https://github.com/Netflix/zuul[Netflix Zuul] as an embedded reverse proxy
and link:https://bitbucket.org/connect2id/nimbus-jose-jwt[Nimbus JOSE + JWT library] to connect to Keycloak,
follow the authentication flow and to handle JWT tokens.
using link:https://github.com/spring-cloud/spring-cloud-gateway[Spring Cloud Gateway] to route APIs
and, together with link:https://bitbucket.org/connect2id/nimbus-jose-jwt[Nimbus JOSE + JWT library], to connect to Keycloak, follow the authentication flow, handle web sessions and JWT tokens.

=== Saturn

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.7.28
0.7.29
46 changes: 27 additions & 19 deletions charts/fairspace/templates/project/configmap-pluto.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ data:
- {{ template "fairspace.url" . }}
{{ if .Values.pluto.externalDomains }}{{ toYaml .Values.pluto.externalDomains | indent 8 }}{{ end }}
downstreamServiceHealthUrl: {{ .Values.pluto.backends.saturn | default "http://localhost:8090"}}/api/health/
max-file-size: {{ .Values.pluto.maxFileSize }}
staticHtmlLocation: /opt/mercury
oauth2:
base-url: {{ .Values.external.keycloak.baseUrl }}
realm: {{ .Values.external.keycloak.realm }}
Expand Down Expand Up @@ -41,25 +43,31 @@ data:
clientSecret: '{{ .Values.external.keycloak.clientSecret }}'

spring:
zipkin:
enabled: {{ .Values.tracing.enabled }}
base-url: {{ .Values.tracing.baseUrl }}
servlet:
multipart:
max-file-size: {{ .Values.pluto.maxFileSize }}
max-request-size: {{ .Values.pluto.maxRequestSize }}

zuul:
retryable: false
host:
connect-timeout-millis: {{ .Values.pluto.connectTimeoutMillis }}
socket-timeout-millis: {{ .Values.pluto.socketTimeoutMillis }}
routes:
web:
resources:
static-locations: classpath:static,file:/opt/mercury
cloud:
gateway:
routes:
{{ if .Values.pluto.backends.storageRoutes }}
{{ toYaml .Values.pluto.backends.storageRoutes | indent 8 }}
{{ end }}
saturn:
path: /api/**
url: {{ .Values.pluto.backends.saturn | default "http://localhost:8090"}}
strip-prefix: false
add-proxy-headers: false
- id: saturn-post-request-size
uri: {{ .Values.pluto.backends.saturn | default "http://localhost:8090/api/" }}
predicates:
- Method=POST
- Path=/api/**
filters:
- name: RequestSize
args:
maxSize: {{ .Values.pluto.maxFileSize }}
- id: saturn
uri: {{ .Values.pluto.backends.saturn | default "http://localhost:8090"}}
predicates:
- Path=/api/**
default-filters:
- RemoveRequestHeader=Pragma X-Frame-Options X-Content-Type-Options X-XSS-Protection X-Permitted-Cross-Domain-Policies Origin
# Including the ORIGIN header would trigger CORS filtering downstream, but Pluto is already doing the filtering.
httpclient:
connect-timeout: {{ .Values.pluto.connectTimeoutMillis }}
response-timeout: {{ .Values.pluto.responseTimeoutMillis }}
1 change: 1 addition & 0 deletions charts/fairspace/templates/project/stateful-set.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ spec:
containers:
- name: {{ .Release.Name }}-postgres-database
image: postgres:15-alpine
args: ["-c", "checkpoint_timeout=43200"]
volumeMounts:
- name: postgres
mountPath: /var/lib/postgresql/data
Expand Down
5 changes: 2 additions & 3 deletions charts/fairspace/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -167,11 +167,10 @@ pluto:
terminationGracePeriodSeconds: 10
keycloak:
redirectAfterLogoutUrl:
# Zuul timeouts when connecting to upstream services
socketTimeoutMillis: 600000 # 10 minutes
# Gateway timeouts when connecting to upstream services
responseTimeoutMillis: 600000 # 10 minutes
connectTimeoutMillis: 2000
maxFileSize: 1GB # max total size of file(s) that can be uploaded
maxRequestSize: 1GB # max total size of the request (should be > maxFileSize)
backends:
storageRoutes:

Expand Down
1 change: 1 addition & 0 deletions local-development/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ services:
volumes:
- fairspace-postgres-data:/var/lib/postgresql/data
restart: unless-stopped
command: ["postgres", "-c", "checkpoint_timeout=43200"]
logging:
driver: ${DOCKER_LOGGING_DRIVER:-journald}
options:
Expand Down
3 changes: 1 addition & 2 deletions local-development/fairspace/local-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ pluto:
image: pluto-local
tag: latest
pullPolicy: Never
socketTimeoutMillis: 600000 # 10 minutes
responseTimeoutMillis: 600000 # 10 minutes
connectTimeoutMillis: 2000
maxFileSize: 1GB
maxRequestSize: 1GB
2 changes: 1 addition & 1 deletion projects/mercury/configmap-server.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ data:
config.json: |-
{
"urls": {
"download": "{{ template "workspace.url" . }}/zuul/api/webdav",
"download": "{{ template "workspace.url" . }}/api/webdav",
"files": "{{ template "storage.url" . }}/api/webdav"
{{ if ( and .Values.services .Values.services.jupyterhub ) }}, "jupyterhub": "{{ .Values.services.jupyterhub }}"{{ end }}
{{ if ( and .Values.services .Values.services.cbioportal ) }}, "cbioportal": "{{ .Values.services.cbioportal }}"{{ end }}
Expand Down
6 changes: 1 addition & 5 deletions projects/mercury/src/file/FileAPI.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,6 @@ class FileAPI {
this.remoteURL = remoteURL;
}

uploadClient() {
return createClient('/zuul' + this.remoteURL);
}

client() {
return createClient(this.remoteURL);
}
Expand Down Expand Up @@ -169,7 +165,7 @@ class FileAPI {
onUploadProgress,
data: formData
};
return this.uploadClient()
return this.client()
.customRequest(destinationPath, requestOptions)
.catch(e => {
if (e && e.response) {
Expand Down
42 changes: 13 additions & 29 deletions projects/mercury/src/file/FileBrowser.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,19 +107,12 @@ export const FileBrowser = (props: FileBrowserProperties) => {
const [overwriteFileCandidateNames, setOverwriteFileCandidateNames] = useState([]);
const [overwriteFolderCandidateNames, setOverwriteFolderCandidateNames] = useState([]);
const [currentUpload, setCurrentUpload] = useState({});
const [isFolderUpload, setIsFolderUpload] = useState();

const {
getRootProps,
getInputProps,
isDragActive,
isDragAccept,
isDragReject,
open
} = useDropzone({
const useFairspaceDropzone = () => useDropzone({
noClick: true,
noKeyboard: true,
multiple: true,
useFsAccessApi: false,
onDropAccepted: (droppedFiles) => {
const newUpload = {
id: generateUuid(),
Expand All @@ -144,6 +137,9 @@ export const FileBrowser = (props: FileBrowserProperties) => {
}
});

const fileDropzoneProperties = useFairspaceDropzone();
const folderDropzoneProperties = useFairspaceDropzone();

// Deselect all files on history changes
useEffect(() => {
const historyListener = history.listen(() => {
Expand All @@ -156,13 +152,6 @@ export const FileBrowser = (props: FileBrowserProperties) => {
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [history]);

// A hook to make sure that isFolderUpload state is changed before opening the upload dialog
useEffect(() => {
if (isFolderUpload !== undefined) {
open();
}
}, [isFolderUpload, open]);

const isParentCollectionDeleted = openedCollection.dateDeleted != null;
const parentCollectionDeletedRef = useRef(isParentCollectionDeleted);
useEffect(() => {
Expand Down Expand Up @@ -191,19 +180,11 @@ export const FileBrowser = (props: FileBrowserProperties) => {
}

const uploadFolder = () => {
if (isFolderUpload) {
open();
} else {
setIsFolderUpload(true);
}
folderDropzoneProperties.open();
};

const uploadFile = () => {
if (!isFolderUpload) {
open();
} else {
setIsFolderUpload(false);
}
fileDropzoneProperties.open();
};

// A highlighting of a path means only this path would be selected/checked
Expand Down Expand Up @@ -300,10 +281,13 @@ export const FileBrowser = (props: FileBrowserProperties) => {
return (
<div data-testid="files-view" className={classes.container}>
<div
{...getRootProps()}
className={`${classes.dropzone} ${isDragActive && classes.activeStyle} ${isDragAccept && classes.acceptStyle} ${isDragReject && classes.rejectStyle}`}
{...fileDropzoneProperties.getRootProps()}
className={`${classes.dropzone} ${fileDropzoneProperties.isDragActive && classes.activeStyle} ${fileDropzoneProperties.isDragAccept && classes.acceptStyle} ${fileDropzoneProperties.isDragReject && classes.rejectStyle}`}
>
<input {...getInputProps()} {...(isFolderUpload && {webkitdirectory: ""})} />
{/* Since dropzone doesn't support a native folder picker we need to create an input element for both folders and files
https://github.com/react-dropzone/react-dropzone/discussions/1157 */}
<input {...fileDropzoneProperties.getInputProps({webkitdirectory: undefined})} />
<input {...folderDropzoneProperties.getInputProps({webkitdirectory: ""})} />
<FileList
selectionEnabled={openedCollection.canRead}
files={files.map(item => ({...item, selected: selection.isSelected(item.filename)}))}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ export const LinkedDataValuesList = (props: LinkedDataValuesListProps) => {
key={index}
>
<Grid item xs={property.isEditable ? 10 : 12} className={classes.values}>
<div style={{overflow: "hidden", textOverflow: "ellipsis", width: "20rem"}}>
<div style={{overflow: "hidden", textOverflow: "ellipsis"}}>
{
columnDefinition.id === LABEL_URI
? <Typography variant="h6">{columnDefinition.getValue(entry, index)}</Typography>
Expand Down
5 changes: 4 additions & 1 deletion projects/mercury/src/permissions/PermissionCard.js
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,7 @@ export const PermissionCard = (props: PermissionCardProperties) => {
<ListItemText
primary={camelCaseToWords(collection.accessMode)}
secondary={descriptionForAccessMode(collection.accessMode)}
style={{whiteSpace: "normal"}}
/>
);
const showMultipleAccessModes = () => (
Expand All @@ -241,6 +242,7 @@ export const PermissionCard = (props: PermissionCardProperties) => {
<ListItemText
primary={camelCaseToWords(mode)}
secondary={descriptionForAccessMode(mode)}
style={{whiteSpace: "normal"}}
/>
</MenuItem>
))}
Expand All @@ -249,6 +251,7 @@ export const PermissionCard = (props: PermissionCardProperties) => {
.map(unavailableMode => (
<MenuItem key={unavailableMode} value={unavailableMode} disabled>
<ListItemText
style={{whiteSpace: "normal"}}
primary={camelCaseToWords(unavailableMode)}
secondary={descriptionForAccessMode(unavailableMode)}
/>
Expand Down Expand Up @@ -351,7 +354,7 @@ export const PermissionCard = (props: PermissionCardProperties) => {
/>
<Collapse in={expanded} timeout="auto" unmountOnExit>
<CardContent style={{paddingTop: 0}}>
<div>
<div style={{overflowX: "auto"}}>
<List>
<ListItem disableGutters>
{renderAccessMode()}
Expand Down
7 changes: 1 addition & 6 deletions projects/mercury/src/setupProxy.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ module.exports = (app) => {
);

// Return 401 Unauthorized for API requests
keycloak.redirectToLogin = (request) => !(request.baseUrl.startsWith('/api/') || request.baseUrl.startsWith('/zuul/'));
keycloak.redirectToLogin = (request) => !(request.baseUrl.startsWith('/api/'));

app.use(keycloak.middleware({logout: '/logout'}));
app.use('/dev', keycloak.protect());
Expand All @@ -41,11 +41,6 @@ module.exports = (app) => {
onProxyReq: addToken
}));

app.use(createProxyMiddleware('/zuul', {
target: 'http://localhost:8080/',
onProxyReq: addToken
}));

app.use(createProxyMiddleware('/actuator/health', {
target: 'http://localhost:8080/'
}));
Expand Down
28 changes: 13 additions & 15 deletions projects/pluto/build.gradle
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
buildscript {
ext {
mockitoVersion = '3.12.0'
mockitoVersion = '5.3.1'
}
}

plugins {
id 'java'
id 'org.springframework.boot' version '2.7.10'
id "io.freefair.lombok" version "6.1.0"
id 'org.springframework.boot' version '3.1.0'
id "io.freefair.lombok" version '8.0.1'
id 'application'
id 'jacoco'
//id 'com.github.spotbugs' version '1.6.10'
id "org.owasp.dependencycheck" version '8.2.1'
id 'io.spring.dependency-management' version '1.1.0'
}

apply plugin: 'io.spring.dependency-management'

group 'nl.fairspace'
version System.getenv('VERSION') ?: '0.0.1-SNAPSHOT'
Expand All @@ -37,31 +36,30 @@ repositories {
}

ext {
springCloudVersion = '2021.0.2'
jacksonVersion = '2.14.2'
springCloudVersion = '2022.0.3'
jacksonVersion = '2.15.2'
}

jacoco.toolVersion = "0.8.7"

dependencies {
implementation("com.fasterxml.jackson.core:jackson-annotations:${jacksonVersion}")
implementation('javax.validation:validation-api:2.0.1.Final')
implementation('org.springframework.boot:spring-boot-starter-web')
implementation('org.springframework.cloud:spring-cloud-starter-netflix-zuul:2.2.10.RELEASE')
implementation('org.springframework.boot:spring-boot-starter-actuator')
implementation('org.springframework.cloud:spring-cloud-gateway-mvc')
implementation('org.springframework.session:spring-session-core')
implementation('org.springframework.cloud:spring-cloud-starter-sleuth')
implementation('org.springframework.cloud:spring-cloud-sleuth-zipkin')
implementation 'io.micrometer:micrometer-tracing-bridge-brave'
implementation("org.springframework.cloud:spring-cloud-starter-gateway")
implementation('org.springframework.boot:spring-boot-actuator')

implementation 'com.nimbusds:nimbus-jose-jwt:9.31'
implementation 'com.nimbusds:oauth2-oidc-sdk:10.7'
implementation 'com.nimbusds:oauth2-oidc-sdk:10.9.1'

testImplementation 'junit:junit:4.12'
testImplementation 'junit:junit:4.13.2'
testImplementation "org.mockito:mockito-core:${mockitoVersion}"
testImplementation 'org.springframework.boot:spring-boot-starter-test'
testImplementation 'org.springframework.cloud:spring-cloud-contract-wiremock'
testImplementation 'org.apache.commons:commons-lang3:3.12.0'
testImplementation 'commons-io:commons-io:2.8.0'
testImplementation 'commons-io:commons-io:2.12.0'

annotationProcessor "org.springframework.boot:spring-boot-configuration-processor"
}
Expand Down
Binary file modified projects/pluto/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
3 changes: 2 additions & 1 deletion projects/pluto/gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.2-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.1-bin.zip
networkTimeout=10000
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Loading