Skip to content

Commit

Permalink
Fix dockerfile location, README and adapt scripts to new docker locat…
Browse files Browse the repository at this point in the history
…ion(s)
  • Loading branch information
brainstorm committed Apr 18, 2024
1 parent d3c028e commit 13b49f5
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 11 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile.aspera
file: docker/Dockerfile.aspera
push: true
tags: |
ghcr.io/umccr/aspera:latest
Expand All @@ -38,7 +38,7 @@ jobs:
uses: docker/build-push-action@v5
with:
context: .
file: deploy/Dockerfile.egacrypt
file: docker/Dockerfile.egacrypt
push: true
tags: |
ghcr.io/umccr/egacrypt:latest
Expand Down
11 changes: 4 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ Get it done on the spot with an AWS instance (a.k.a: to be automated with AWS Ba
$ sudo yum install docker && gpasswd -a ec2-user docker # Note: Only needed if your AWS instance doesn't have docker yet (docker ps -a works?)
# Note2: Logout and login for the docker group add to be successful
$ git clone https://github.com/umccr/ega-submit && cd ega-submit
$ docker build . -f docker/Dockerfile.aspera -t aspera
$ docker build . -f docker/Dockerfile.egacrypt -t egacrypt
$ vim ega-files.txt <--- the input file list, consisting of a full s3:// url to the object, one per line
$ export EGA_BOX=ega-box-1578
$ export EGA_PASSWORD=<PASSWORD_HERE>
Expand All @@ -21,11 +19,6 @@ The serial script will (very slowly and screaming to be parallelized) download e
Please note that the serial script downloads, encrypts and **deletes** files. This is by design, so that a small AWS SPOT box with almost no disk space
can be used on the span of days, without risk of saturating the (majorly unknown) ingress limit specifications for EGA, therefore avoiding timeouts.

# Building

1. docker build . -f docker/Dockerfile.aspera -t aspera
2. docker build . -f docker/Dockerfile.egacrypt -t egacrypt

# Running

By using [EGA cryptor][ega-cryptor] and Aspera Connect:
Expand All @@ -34,4 +27,8 @@ By using [EGA cryptor][ega-cryptor] and Aspera Connect:
2. Run, for instance: `$ ./bin/submit.sh data/htsnexus_test_NA12878.bam ega-box-NNNN pass`.
3. `data/enc` contains encrypted data that have been sent to EBI's EGA server.

# Similar third party solutions

* https://github.com/DKFZ-ODCF/ega-cluster-cryptor

[ega-cryptor]: https://ega-archive.org/submission/tools/egacryptor
4 changes: 2 additions & 2 deletions bin/submit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ USERNAME=$2
PASSWORD=$3

# Encrypt
docker run -v $PWD/data:/data egacrypt -f -i /data/$SAMPLE_NAME -o /data/enc/$SAMPLE_NAME
docker run -v $PWD/data:/data ghcr.io/umccr/egacrypt -f -i /data/$SAMPLE_NAME -o /data/enc/$SAMPLE_NAME

# Submit
docker run -v $PWD/data/enc:/data --env ASPERA_SCP_PASS=$PASSWORD aspera -P33001 -O33001 -QT -L- /data/$SAMPLE_NAME $USERNAME@$HOSTNAME:/.
docker run -v $PWD/data/enc:/data --env ASPERA_SCP_PASS=$PASSWORD ghcr.io/umccr/aspera -P33001 -O33001 -QT -L- /data/$SAMPLE_NAME $USERNAME@$HOSTNAME:/.

0 comments on commit 13b49f5

Please sign in to comment.