Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#264 Execution of additional SBG workflows #12

Merged
merged 11 commits into from
Mar 5, 2024

Conversation

LucaCinquini
Copy link
Collaborator

Purpose

  • The purpose of this PR is to improve the SPS functionality to execute SBG workflows.

Proposed Changes

  • [ADD] The generic CWL DAG can now use job parameters from either a JSON formatted string, or a URL referencing a YAML or JSON file
  • [ADD] A new DAG that is specific to collecting the input parameters for the SBG L1 to L2 end-to-end workflow
  • [CHANGE] The generic docker-cwl pod is now using a fixed version of the sps-docker-cwl image

Issues

Testing

  • The generic cwl-dag was used to successfully execute the SBG pre-process workflow, either passing the URL of the YAML parameters file, or the same parameter values specified in a JSON formatted string
  • The new sbg-l1-to-l2-e2e-cwl-dag was successfully triggered, although eventually it crashed because of the Pod disk becoming full

@drewm-jpl drewm-jpl merged commit b733337 into develop Mar 5, 2024
1 of 2 checks passed
@drewm-jpl drewm-jpl deleted the 264-execution-of-additional-sbg-workflows branch March 5, 2024 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants