Memory problem? #11
Assignees
Labels
Comments
|
Use ntCard to count the number of distinct 32-mers in the data set. With that value in hand, we can predict the memory usage of ABySS. |
|
@kokyriakidis Reconfigure from source with this command and then re-compile: By the way, Trans-ABySS is actually not using the Bloom filter de bruijn graph implementation of ABySS. |
|
The easiest way to install ABySS with a different value of |
|
So let me ask you something. Let’s say I compile abyss with maxk=512 in a machine and in an other machine I compile abyss with kmax=90. If I run the same commands with k=32 and with the same input files, am I gonna have different memory usage results? |
|
Yes, |
|
Ok I will try it and I will tell you my results. Keep it open. One more question. If I iterate from 32 to 90 by 2 and in a second machine (with the same maxk) iterate from 32 to 90 by 1, will I have the same result at k=40 for example? Or the number of previous iterations have an impact on the result? |
|
If you assemble at k=40 on different machines:
However, the merged assembly of 32-90:1 would not be the same as the merged assembly of 32-90:2. |
|
So what's the best procedure? With step 1? |
|
Ok this is the kmer frequency The command was: How do I interpret it? |
|
@kokyriakidis |
|
I had asked @kokyriakidis to report the output of ntCard to predict the memory usage of ABySS. There are three billion distinct k-mers. The memory usage of ABySS is… (8 + maxk / 4) · F₀ @benvvalk I added an entry to ABySS's FAQ: https://github.com/bcgsc/abyss/wiki/ABySS-Users-FAQ#6-how-much-memory-does-abyss-use |
|
Here is the situation. Abyss is installed in my university's cluster. I compiled it again locally in my given partition, with default maxk (128), so according to the output of ntCards I need 120gb of RAM. The job is gonna run on a 300gb RAM machine. How can I specify transabyss to use the locally compiled abyss and not just randomly try to find it and maybe use the abyss version installed in the cluster? |
|
@kmnip ? My guess is that it uses the version of ABySS found first in the |
|
Yes, it is just that simple!
For example:
```
ABYSS_PATH=/path/to/my/abyss/bin/directory/
export PATH=${ABYSS_PATH}:${PATH}
transabyss -k 32 --se read_1.fq read_2.fq ...
```
…On Wed, Oct 3, 2018 at 11:09 AM Shaun Jackman ***@***.***> wrote:
@kmnip <https://github.com/kmnip> ? My guess is that it uses the version
of ABySS found first in the PATH environment variable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AFJULmnZmToBIovTqzOdJAiQ9lNWebmvks5uhP1wgaJpZM4XEhyJ>
.
|
|
Hi, after running transabyss through the local compiled abyss it produced some output files but it shows up an error at the end make: *** No rule to make target `k32.fastq.gz-1.adj'. Stop. What is that and how can I fix it? |
|
Can you please report the exact command that you use and your entire log
file?
…On Fri, Oct 5, 2018 at 7:39 AM kokyriakidis ***@***.***> wrote:
Hi, after running transabyss through the local compiled abyss it produced
some output files but it shows up an error at the end
make: *** No rule to make target `k32.fastq.gz-1.adj'. Stop.
What is that and how can I fix it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AFJULrCcdTYwOqOpfqDn8XIBakV8h2ZIks5uh28vgaJpZM4XEhyJ>
.
|
|
Ok this is the command: These are the output files: This is the log: |
|
@kokyriakidis Thank you. I see what the problem is now! You were using Trans-ABySS version 1.5.5 with ABySS version is 2.1.1, but Trans-ABySS 1.5.x does not support ABySS 2.x.x. To resolve the error that you got earlier, you just need to upgrade your Trans-ABySS to the latest version, clean the directory, and re-run. |
|
@kmnip Thank you! I will try it and I will report! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I tried transabyss (k32) on my university cluster on 40gb RNASeq data, following your short tutorial, and it is stuck because it used all 384gb of the node and also swaped! I think it is really strange to require so much memory! Do you have any thoughts?
The text was updated successfully, but these errors were encountered: