-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sequence to exported GFF3 #281
Add sequence to exported GFF3 #281
Conversation
packages/apollo-collaboration-server/src/features/features.service.ts
Outdated
Show resolved
Hide resolved
packages/apollo-collaboration-server/src/features/features.service.ts
Outdated
Show resolved
Hide resolved
Has this been tested on exporting very large data? Looking at the code, it looks like it's loading all the sequences into memory, which we want to avoid because we want to keep server memory usage low, and it will be slower. I think for the pipeline(
this.refSeqChunksModel
.find({ refSeq: refSeqDocs }) // get chunks for all refSeqs in a single query
.sort({ n: 1 }) // make sure the sort accounts for having all refSeqs
.cursor(),
new Transform({
construct(callback) {
this.printFasta = true;
this.printSeqName = true;
this.remainingLastLine = ''
callback();
},
transform(chunk, encoding, callback) {
// basically the same logic that's already there, but using `this.push` instead of `sequenceStream.push`
callback();
},
flush(callback) {
if (this.remainingLastLine) this.push(remainingLastLine)
},
}),
(error) => {
if (error) {
this.logger.error('GFF3 export failed')
this.logger.error(error)
}
},
); The node docs on streams might be helpful here: https://nodejs.org/docs/latest-v18.x/api/stream.html |
dd856e1
to
b09dd8d
Compare
b09dd8d
to
699a5e8
Compare
FASTA is now included in exported GFF3. Please, merge this only after Bug_251_Download_not_working has been merged.