Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Dockerfile for easier deployment and consistent builds #105

Merged
merged 6 commits into from
Aug 21, 2024

Conversation

olliecheng
Copy link
Collaborator

@olliecheng olliecheng commented Aug 20, 2024

Hi,

This PR contains an updated Docker build process which I hope will become a supported and well-maintained deployment method for JAFFA.

It supports GitHub Actions for auto deployment, includes a slimmed down final run stage, and includes a fixed set of binaries and runtimes which mirror the settings recommended in the JAFFA wiki. This PR also includes a patch for the bpipe bug in ssadedin/bpipe#290, which is a randomly occurring issue which can present in both containers and native runs.

Improvements compared to existing approaches:

  • It runs the most recent version of JAFFA, 2.4! Hurrah!
  • Sensible binding defaults:
    • JAFFA files are all located at /JAFFA, so you can find e.g. JAFFA_assembly.groovy at the location /JAFFA/JAFFA_assembly.groovy
    • Reference files are expected at /ref
  • An entrypoint which points to bpipe (so you can execute the command bpipe run <params> with docker run docker://ghcr.io/olliecheng/jaffa:latest <params>
  • A GitHub Action for builds and uploads. An automated build and deployment can be started using the corresponding button in Actions (see https://github.com/olliecheng/JAFFA/actions)
  • Separate build and run stages are used, reducing the filesize of the final image
  • Bundled R 4.4.0 and OpenJDK 11 runtimes (this matches the default of the WEHI HPC)
  • Dependencies are installed using install_linux64.sh instead of a package manager. This should ease maintainability in two ways. Firstly, this means that the Docker image uses the same fixed dependency versions as the native install method, eliminating any potential differences arising as a result of version mismatches. Secondly, this prevents any update to package manager repositories affecting key dependencies.
  • No wrapper around bpipe run <groovy>. When running JAFFA through bioconda or its auto-built Docker image, the Conda distribution obscures the bpipe call around a wrapper which may be confusing to users: see Please provide a conda wrapper for jaffal, as for other jaffa functionality #74.

The main goal is to produce an image which can be easily updated in the future if JAFFA changes, with as minimal change to the Dockerfile as possible.

In my testing, I've observed that the output from the native and the Docker JAFFAL pipelines are identical when testing using the instructions and data provided in the wiki. For direct, assembly, and hybrid modes, I think that the output is unlikely to be identical due to randomness in bowtie2. However, the first 9 columns of JAFFA_results.csv are identical.

You can try out my build of version 2.4 at docker://ghcr.io/olliecheng/jaffa:latest.

@olliecheng
Copy link
Collaborator Author

olliecheng commented Aug 20, 2024

I am also interested in writing documentation to the JAFFA wiki explaining how to use the Docker image, alongside the regular method. That way, users won't need to read the Dockerfile to understand where JAFFA expects the reference files to be located or where to find the pipeline files.

@nadiadavidson nadiadavidson merged commit bab028b into Oshlack:master Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants