diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 49ee377..4b9822c 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -27,7 +27,7 @@ jobs: uses: docker/build-push-action@v5 with: context: . - file: deploy/Dockerfile.aspera + file: docker/Dockerfile.aspera push: true tags: | ghcr.io/umccr/aspera:latest @@ -38,7 +38,7 @@ jobs: uses: docker/build-push-action@v5 with: context: . - file: deploy/Dockerfile.egacrypt + file: docker/Dockerfile.egacrypt push: true tags: | ghcr.io/umccr/egacrypt:latest diff --git a/README.md b/README.md index f0b0d2b..04f7fd9 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,6 @@ Get it done on the spot with an AWS instance (a.k.a: to be automated with AWS Ba $ sudo yum install docker && gpasswd -a ec2-user docker # Note: Only needed if your AWS instance doesn't have docker yet (docker ps -a works?) # Note2: Logout and login for the docker group add to be successful $ git clone https://github.com/umccr/ega-submit && cd ega-submit -$ docker build . -f docker/Dockerfile.aspera -t aspera -$ docker build . -f docker/Dockerfile.egacrypt -t egacrypt $ vim ega-files.txt <--- the input file list, consisting of a full s3:// url to the object, one per line $ export EGA_BOX=ega-box-1578 $ export EGA_PASSWORD= @@ -21,11 +19,6 @@ The serial script will (very slowly and screaming to be parallelized) download e Please note that the serial script downloads, encrypts and **deletes** files. This is by design, so that a small AWS SPOT box with almost no disk space can be used on the span of days, without risk of saturating the (majorly unknown) ingress limit specifications for EGA, therefore avoiding timeouts. -# Building - -1. docker build . -f docker/Dockerfile.aspera -t aspera -2. docker build . -f docker/Dockerfile.egacrypt -t egacrypt - # Running By using [EGA cryptor][ega-cryptor] and Aspera Connect: @@ -34,4 +27,8 @@ By using [EGA cryptor][ega-cryptor] and Aspera Connect: 2. Run, for instance: `$ ./bin/submit.sh data/htsnexus_test_NA12878.bam ega-box-NNNN pass`. 3. `data/enc` contains encrypted data that have been sent to EBI's EGA server. +# Similar third party solutions + +* https://github.com/DKFZ-ODCF/ega-cluster-cryptor + [ega-cryptor]: https://ega-archive.org/submission/tools/egacryptor diff --git a/bin/submit.sh b/bin/submit.sh index 24860db..4ac75c2 100755 --- a/bin/submit.sh +++ b/bin/submit.sh @@ -11,7 +11,7 @@ USERNAME=$2 PASSWORD=$3 # Encrypt -docker run -v $PWD/data:/data egacrypt -f -i /data/$SAMPLE_NAME -o /data/enc/$SAMPLE_NAME +docker run -v $PWD/data:/data ghcr.io/umccr/egacrypt -f -i /data/$SAMPLE_NAME -o /data/enc/$SAMPLE_NAME # Submit -docker run -v $PWD/data/enc:/data --env ASPERA_SCP_PASS=$PASSWORD aspera -P33001 -O33001 -QT -L- /data/$SAMPLE_NAME $USERNAME@$HOSTNAME:/. +docker run -v $PWD/data/enc:/data --env ASPERA_SCP_PASS=$PASSWORD ghcr.io/umccr/aspera -P33001 -O33001 -QT -L- /data/$SAMPLE_NAME $USERNAME@$HOSTNAME:/.