GREAT enrichment analysiss script generalisable?

[chrisclarkson]

Hi,
I've just taken a look at your run_GREAT_submission.sh script- it looks really useful.
I had a play with it- trying to get it to run on my own bed file:
I made an INFOFILE as your script seems to require and a shuffled version of the input bed file.
The script seems to process the data correctly (print the first 3 columns of each bed file)
However, when I get the output, it just results in a web page that says:

GREAT encountered a user error.

GREAT encountered an error while retrieving data from http://www.broadinstitute.org/~rmgpccl/run_GREAT_analysis_32566/regions_29134.bed.

The script seems to also require your ChromImpute.sh script but I have commented out this line as I simply want to upload my own data to the GREAT website and download the results?

Is it possible to make this script generalisable for all bed files?

[cboix]

Hi,
The submission script temporarily puts the BED file of interest into an outward facing personal directory.
This directory is tailored to my configuration through the Broad Institute cluster. By dropping my file in the /web/personal/$USER directory, I can make it public facing and GREAT can see it.

The relevant code requesting the output is here, which stitches together a url request to the GREAT webpage:

    REQ_BASE="http://bejerano.stanford.edu/great/public/cgi-bin/greatStart.php"
    REQ_OUT="outputType=batch&requestSpecies=hg19"
    REQ_FILE="requestURL=http://www.broadinstitute.org/~${USER}/${TMP_FILE}"
    if [[ "$BKGFILE" != "NONE" ]]; then
        REQ_BKG="bgURL=http://www.broadinstitute.org/~${USER}/${TMP_BKG}"
        FULLREQUEST="${REQ_BASE}?${REQ_OUT}&${REQ_FILE}&${REQ_BKG}"
    else
        FULLREQUEST="${REQ_BASE}?${REQ_OUT}&${REQ_FILE}"
    fi

The code is not currently working for you because you are not using the same cluster system:
You are telling GREAT to look here: http://www.broadinstitute.org/~rmgpccl/run_GREAT_analysis_32566/regions_29134.bed
But you haven't created this file.

If you adjust the webpage + where you put your BED file, the rest should work, unless the GREAT API has changed recently.

[chrisclarkson]

Hi,
Thanks for such a quick reply!
So I tried fiddling with the path to the file REQ_FILE="requestURL=http://www.broadinstitute.org/~${USER}/${TMP_FILE}"
I did the following (for the TMP and BKG file):
Initially I had tried making the PUBROOT variable equal to ./web/ and this worked in creating the web and sub directories (I do not have permission on my cluster to access the root so I have to use my own local directory.)
This however did not work so I tried setting PUBROOT to ./
and subsequently I played with line 80:
REQ_FILE="requestURL=http://www.broadinstitute.org/${TMP_FILE}"
This results in:

GREAT encountered an error while retrieving data from http://www.broadinstitute.org/run_GREAT_analysis_3253/regions_26723.bed.

I also tried specifying the full path but got the same error:

GREAT encountered an error while retrieving data from http://www.broadinstitute.org/~/EPIMAP_ANALYSIS/great_enrichment/run_GREAT_analysis_17015/regions_5513.bed

I also just tried omitting the http:..org part and then the requestURL= consecutively and these resulted in their own distinct errors.

Hence- Is there something special about the /web/ dir on your cluster that allows you to do this or am I missing something?

[cboix]

Hi,
The files we put on the /web/personal/$USER directory on our cluster can be accessed through the http://www.broadinstitute.org/$user page. Because GREAT needs to get the files from a server like that, I put them up temporarily, run the script, and then take them down when the script is done. If you want to replicate this, you would need a similar solution through your own cluster.