Picard running out of memory

[alyazeeditalal]

I am running TEfinder to find insertions in a large bam file 18GB however the analysis pipeline stops at the Picard step with a message Runtime.totalMemory()=2097152000. I am running the analysis in HPC and installed all dependencies using conda. I had to change picard command in the TEfinder file pipline because I can run Picard from the shell with out having to run java first, by only typing "picard".

picard -Xmx2000m FilterSamReads I=${workingdir}/${outname}Alignments.bam O=${currdir}/${line}_DiscordantPairs.bam
READ_LIST_FILE=${currdir}/${line}_ReadID.txt FILTER=includeReadList WRITE_READS_FILES=false
echo -e $(date) " Filtering original alignment based on discordant reads IDs is complete for "${line}"\n" ####

The analysis stops at the picard step
INFO 2021-12-10 15:42:43 FilterSamReads 2 SAMRecords written to Motif:LTR_retrotransposon_161_DiscordantPairs.bam
[Fri Dec 10 15:42:43 GMT 2021] picard.sam.FilterSamReads done. Elapsed time: 6.93 minutes.
Runtime.totalMemory()=2097152000

I tried to downsample the bam file to a smaller size bam by selecting only 25% coverage. Though, exactly the same issue where the pipeline stopped at the Picard step.

What are your suggestions?

Kind regards,

[VistaSohrab]

Picard is unfortunately memory intensive. I'm wondering whether increasing the max heap memory from the default value of 2000 megabytes or 2 gigabytes (-Xmx2000m) to a higher value depending on the total requested memory in the HPC would help.