Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(website): Add new form to submit individual sequences #3710

Open
wants to merge 67 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 63 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
3214444
Add routing stub
fhennig Feb 18, 2025
e1563d3
Add TODO markers
fhennig Feb 18, 2025
d66f56e
Pull out the form into seperate component
fhennig Feb 18, 2025
1a80b44
WIP
fhennig Feb 18, 2025
e1db27d
Finished refactoring of edit form
fhennig Feb 18, 2025
a9a5552
fmt; refactoring some names
fhennig Feb 18, 2025
598d070
extract more
fhennig Feb 18, 2025
afcb0ec
Started the switch component
fhennig Feb 18, 2025
aee6b2c
Progress
fhennig Feb 18, 2025
c57ac05
comment
fhennig Feb 19, 2025
fe0021d
seems to work
fhennig Feb 19, 2025
f06363e
Make it possible to switch with URL parameter
fhennig Feb 19, 2025
801908a
default to bulk again
fhennig Feb 19, 2025
0146432
fix missing prop
fhennig Feb 19, 2025
5f22ac7
make title respect singular/plural
fhennig Feb 19, 2025
3716d36
Fix error: td cannot be inside of div
fhennig Feb 20, 2025
78a8ce3
Remove outdated todo
fhennig Feb 20, 2025
9a47ae3
Initialize form with segment names
fhennig Feb 20, 2025
dcf4208
Add a mini doc string
fhennig Feb 20, 2025
697be71
Add test for SubmitRouteUtils
fhennig Feb 20, 2025
b78e6a0
Add some comments
fhennig Feb 20, 2025
5878934
Add some comments
fhennig Feb 20, 2025
e915d73
Remove outdated todo
fhennig Feb 20, 2025
d4f12ac
Design progress
fhennig Feb 20, 2025
08377ad
Shuffled around spacing to make verything align much nicer
fhennig Feb 20, 2025
b11675b
make link positioning responsive
fhennig Feb 20, 2025
ca3a2f3
change how display works for Revise page
fhennig Feb 20, 2025
2a9cc7a
Change mention of data use terms in the ackknowledgement
fhennig Feb 20, 2025
c7a3c8e
Improve error messages
fhennig Feb 20, 2025
5bf470a
fix
fhennig Feb 20, 2025
689b7cd
test stuff
fhennig Feb 20, 2025
ebc8028
another test
fhennig Feb 20, 2025
3f7e96a
test
fhennig Feb 20, 2025
d94d57e
WIP
fhennig Feb 20, 2025
5be016c
refacotred
fhennig Feb 20, 2025
c6ea0fd
test fix
fhennig Feb 20, 2025
bfe250b
fixed other tests
fhennig Feb 20, 2025
21e47fa
Refactoring
fhennig Feb 20, 2025
853f926
Expand submission form testing
fhennig Feb 20, 2025
eb53024
chore: add some setup to integration-tests
fhennig Feb 20, 2025
566b598
Add documentation
fhennig Feb 21, 2025
c04b2f0
Make form the default and add integration test
fhennig Feb 21, 2025
bb2d170
foo
fhennig Feb 21, 2025
e9fb7ca
Test fix
fhennig Feb 21, 2025
cda2d1e
Remove broken (and not really necessary) test
fhennig Feb 21, 2025
fe771d6
make 'form' the default input mode
fhennig Feb 21, 2025
0a4bc51
fix website tests
fhennig Feb 21, 2025
6843cfd
Default to bulk
fhennig Feb 24, 2025
84d0e5c
adapt integration test
fhennig Feb 24, 2025
6560d1a
Use papaparse instead of string interpolation for TSV building
fhennig Feb 24, 2025
aa452a7
Add tests to check TSV contents
fhennig Feb 24, 2025
6733382
Let users enter a Submission ID
fhennig Feb 24, 2025
ee69df9
Better submission ID handling
fhennig Feb 24, 2025
7da24e4
remove justify-end
fhennig Feb 24, 2025
5a8dacc
Fix website tests
fhennig Feb 24, 2025
9c23688
Use tabs
fhennig Feb 24, 2025
e61361a
use default input mode from portal
fhennig Feb 24, 2025
b30b532
add Submission ID to integration test (should fix)
fhennig Feb 25, 2025
df1b947
update docs
fhennig Feb 25, 2025
912e2a3
use const
fhennig Feb 25, 2025
2b6b48f
Add waiting for header to test
fhennig Feb 25, 2025
a1a1d45
formatting
fhennig Feb 25, 2025
167f747
Increase timeout
fhennig Feb 25, 2025
e7b5ad0
Update docs/src/content/docs/for-users/submit-sequences.md
fhennig Feb 25, 2025
93f60fa
Update docs/src/content/docs/for-users/submit-sequences.md
fhennig Feb 25, 2025
1788f4a
Fix and expand docs
fhennig Feb 26, 2025
aca1895
Add missing step to submission instructions
fhennig Feb 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 24 additions & 10 deletions docs/src/content/docs/for-users/submit-sequences.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ title: Submit sequences

You can only upload sequences if you already have an account and [are part of a submitting group](../create-manage-groups/).

You can submit sequences either through uploading files or entering sequence metadata on the website directly.
Submission for the website form is only recommended for uploading a single sequence, for multiple sequences you should use bulk submission with file upload.

## Submit a single sequence through the form

1. Log into your account, and then click 'Submit' in the top-right corner of the website.
2. Select the organism that you'd like to submit sequences for.
3. Click on 'Submit single sequence'.
4. Enter the metadata information about your sequence, as well as the unaligned nucleotide sequence(s).
5. If Terms of Use are enabled for your Loculus instance: Select the Terms of Use that you would like for your data.
6. Select 'Submit sequences' at the bottom of the page.

## Submit sequences via file upload

Before you begin this process, you should ensure your data is in the correct format. Each sequence should have a unique submissionID that can be used to associate it with its corresponding metadata entry.

Loculus expects:
Expand All @@ -13,23 +27,23 @@ Loculus expects:

![Metadata template.](../../../assets/MetadataTemplate.png)

## Multi-segmented Pathogens
### Multi-segmented Pathogens

Loculus expects multi-segmented pathogens to have one unique submissionID per **isolate** (pathogen sample containing all segments). However, `fasta` files should still have a separate entry/record per segment. Therefore, each record id should include the unique submissionID of the isolate and the segment name, for example: `submissionID + '_' + segmentName`. The metadata is uploaded per isolate, i.e. there will be only one row for each `submissionID` and segmented metadata parameters need to be uploaded individually, i.e. under `length_{segmentName}` etc.

## Website
### Website

Uploading sequences via the website is a easy way to submit sequences without having to worry about any code.
Uploading sequences via the website is an easy way to submit sequences without having to worry about any code.

1. Log into your account, and then click 'Submit' in the top-right corner of the website
2. Select the organism that you'd like to submit sequences for
3. Drag-and-drop a `fasta` file with the sequences and a metadata file with the associated metadata into the box on the website, or click the 'Upload a file' link within the boxes to open a file-selection box
4. If Terms of Use are enabled for your Loculus instance: Select the Terms of Use that you would like for your data
5. Select 'Submit sequences' at the bottom of the page
1. Log into your account, and then click 'Submit' in the top-right corner of the website.
2. Select the organism that you'd like to submit sequences for.
3. Drag-and-drop a `fasta` file with the sequences and a metadata file with the associated metadata into the box on the website, or click the 'Upload a file' link within the boxes to open a file-selection box.
4. If Terms of Use are enabled for your Loculus instance: Select the Terms of Use that you would like for your data.
5. Select 'Submit sequences' at the bottom of the page.

The data will now be processed, and you will have to approve your submission before it is finalized. You can see how to do this [here](../approve-submissions/).

## API
### API

It is possible to upload sequences through an HTTP API. We also plan to release a command-line interface soon.

Expand Down Expand Up @@ -63,6 +77,6 @@ Further information can be found in the API documentation of the instance.

As with the website, data will now be processed, and you will have to approve your submission before it is finalized. You can see how to do this [here](../approve-submissions/).

## Compressing files
### Compressing files

For both sequence and metadata files, compression is supported. The supported formats are: `zip`, `gz` (gzip), `zst` (ZStandard) and `xz` (LZMA). (Note that Excel file uploads with `xz` compression are currently not supported.)
19 changes: 18 additions & 1 deletion integration-tests/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Integration tests

These are tests of the full Loculus system, following sequences through submission to preprocessing to release.
These are tests of the full Loculus system, following sequences through submission to preprocessing to release in the browser, using Playwright.

## Principles

Expand All @@ -13,3 +13,20 @@ Here are some current guiding principles for these tests:
There are some fixtures to help with the development of tests:
- `pageWithACreatedUser` creates a user account and logs into it
- `pageWithGroup` inherits from `pageWithACreatedUser` and in addition creates a group for the user

## Running the tests

### Initial Setup

Install dependencies:

npm ci
npx playwright install --with-deps

Set up the cluster to test:

./start-server.sh

Run the tests:

npx playwright test
16 changes: 8 additions & 8 deletions integration-tests/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion integration-tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"description": "",
"devDependencies": {
"@playwright/test": "^1.48.2",
"@types/node": "^22.8.3",
"@types/node": "^22.13.4",
"@types/uuid": "^10.0.0",
"uuid": "^11.0.2"
}
Expand Down
50 changes: 49 additions & 1 deletion integration-tests/tests/specs/features/submission-flow.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,15 @@ import { test } from '../../fixtures/group.fixture';
import { join } from 'path';

test.describe('Submission flow', () => {
test('basic submission flow works', async ({ pageWithGroup }) => {
test('submission page shows group creation button when not in a group', async ({ pageWithACreatedUser }) => {
test.setTimeout(10000);
const page = pageWithACreatedUser;
await page.getByRole('link', { name: 'Loculus' }).click();
await page.getByRole('link', { name: 'Submit' }).click();
await page.getByRole('link', { name: 'create a submitting group' }).click();
});

test('basic file upload submission flow works', async ({ pageWithGroup }) => {
test.setTimeout(90000);
const page = pageWithGroup;

Expand All @@ -23,6 +31,9 @@ test.describe('Submission flow', () => {
await page.getByLabel('I confirm I have not and will').check();

await page.getByRole('button', { name: 'Submit sequences' }).click();

await expect(page.getByRole('heading', { name: 'Review current submissions' })).toBeVisible();

await page.getByRole('button', { name: 'Release 1 valid sequence' }).click();
await page.getByRole('button', { name: 'Release', exact: true }).click();
await page.getByRole('link', { name: 'Released Sequences' }).click();
Expand All @@ -33,6 +44,43 @@ test.describe('Submission flow', () => {
await page.getByRole('cell', { name: 'Pakistan' }).click();
await page.waitForSelector('text="test_NIHPAK-19"');
await page.waitForSelector('text="NC_005302.1"'); // reference
});

test('basic form submission flow works', async ({ pageWithGroup }) => {
test.setTimeout(90000);
const page = pageWithGroup;

await page.getByRole('link', { name: 'Loculus' }).click();
await page.getByRole('link', { name: 'Submit' }).click();
await page.getByRole('link', { name: 'Crimean-Congo Hemorrhagic Fever Virus' }).click();
await page.getByRole('link', { name: 'Submit Upload New Sequences' }).click();
await page.getByRole('link', { name: 'Submit single sequence' }).click();

await page.getByLabel(/Submission ID/).fill('XF499');
await page.getByLabel(/Collection country/).fill('Colombia');
await page.getByLabel(/Collection date/).fill('2012-12-12');
await page.getByLabel("L:", { exact: true }).fill("CCACATTGACACAGANAGCTCCAGTAGTGGTTCTCTGTCCTTATTAAACCATGGACTTCTTAAGAAACCTTGACTGGACTCAGGTGATTGCTAGTCAGTATGTGACCAATCCCAGGTTTAATATCTCTGATTACTTCGAGATTGTTCGACAGCCTGGTGACGGGAACTGTTTCTACCACAGTATAGCTGAGTTAACCATGCCCAACAAAACAGATCACTCATACCATAACATCAAACATCTGACTGAGGTGGCAGCACGGAAGTATTATCAGGAGGAGCCGGAGGCTAAGCTCATTGGCCTGAGTCTGGAAGACTATCTTAAGAGGATGCTATCTGACAACGAATGGGGATCGACTCTTGAGGCATCTATGTTGGCTAAGGAAATGGGTATTACTATCATCATTTGGACTGTTGCAGCCAGTGACGAAGTGGAAGCAGGCATAAAGTTTGGTGATGGTGATGTGTTTACAGCCGTGAATCTTCTGCACTCCGGACAGACACACTTTGATGCCCTCAGAATACTGCCNCANTTTGAGGCTGACACAAGAGAGNCCTTNAGTCTGGTAGACAANNTNATAGCTGTGGACCANNTGACCTCNTCTTCAAGTGATGAANTGCAGGACTANGAAGANCTTGCTTTAGCACTTACNAGNGCGGAAGAACCATNTAGACGGTCTAGCNTGGATGAGGTNACCCTNTCTAAGAAACAAGCAGAGNTATTGAGGCAGAAGGCATCTCAGTTGTCNAAACTGGTTAATAAAAGTCAGAACATACCGACTAGAGTTGGCAGGGTTCTGGACTGTATGTTTAACTGCAAACTATGTGTTGAAATATCAGCTGACACTCTAATTCTGCGACCAGAATCTAAAGAAAGAATTGG");
await page.getByLabel("M:", { exact: true }).fill("GTGGATTGAGCATCTTAATTGCAGCATACTTGTCAACATCATGCATATATCATTGATGTATGCAGTTTTCTGCTTGCAGCTGTGCGGTCTAGGGAAAACTAACGGACTACACAATGGGACTGAACACAATAAGACACACGTTATGACAACGCCTGATGACAGTCAGAGCCCTGAACCGCCAGTGAGCACAGCCCTGCCTGTCACACCGGACCCTTCCACTGTCACACCTACAACACCAGCCAGCGGATTAGAAGGCTCAGGAGAGGTTCACACATCCTCTCCAATCACCACCAAGGGTTTGTCTCTGCCGGGGGCTACATCTGAGCTCCCTGCGACTACTAGCATAGTCACTTCAGGTGCAAGTGATGCCGATTCTAGCACACAGGCAGCCAGAGACACCCCTAAACCATCAGTCCGCACGAGTCTGCCCAACAGCCCTAGCACACCATCCACACCACAAGGCACACACCATCCCGTGAGGAGTCTGCTTTCAGTCACGAGCCCTAAGCCAGAAGAAACACCAACACCGTCAAAATCAAGCAAAGATAGCTCAGCAACCAACAGTCCTCACCCAGCCGCCAGCAGACCAACAACCCCTCCCACAACAGCCCAGAGACCCGCTGAAAACAACAGCCACAACACCACCGAACAGCTTGAGTCCTTAACACAATTAGCAACTTCAGGTTCAATGATCTCTCCAACACAGACAGTCCTCCCAAAGAGTGTTACTTCTATAGCCATTCAAGACATTCATCCCAGCCCAACAAATAGGTCTAAAAGAAACCTTGATATGGAAATAATCT");
await page.getByLabel("S:", { exact: true }).fill("GTGTTCTCTTGAGTGTTGGCAAAATGGAAAACAAAATCGAGGTGAACAACAAAGATGAGATGAACAAATGGTTTGAGGAGTTCAAGAAAGGAAATGGACTTGTGGACACTTTCACAAACTCNTATTCCTTTTGTGAAAGCGTNCCAAATCTGGACAGNTTTGTNTTCCAGATGGCNAGTGCCACTGATGATGCACAAAANGANTCCATCTACGCATCTGCNCTGGTGGANGCAACCAAATTTTGTGCACCTATATACGAGTGTGCTTGGGCTAGCTCCACTGGCATTGTTAAAAAGGGACTGGAGTGGTTCGAGAAAAATGCAGGAACCATTAAATCCTGGGATGAGAGTTATACTGAGCTTAAAGTTGAAGTTCCCAAAATAGAACAACTCTCCAACTACCAGCAGGCTGCTCTCAAATGGAGAAAAGACATAGGCTTCCGTGTCAATGCAAATACGGCAGCTTTGAGTAACAAAGTCCTAGCAGAGTACAAAGTTCCTGGCGAGATTGTAATGTCTGTCAAAGAGATGTTGTCAGATATGATTAGAAGNAGGAACCTGATTCTCAACAGAGGTGGTGATGAGAACCCACGCGGCCCAGTTAGCCGTGAACATGTGGAGTGGTGC");

await page.getByLabel('I confirm that the data').check();
await page.getByLabel('I confirm I have not and will').check();

await page.getByRole('button', { name: 'Submit sequences' }).click();

await expect(page.getByRole('heading', { name: 'Review current submissions' })).toBeVisible();

await page.getByRole('button', { name: 'Release 1 valid sequence' }).click();
await page.getByRole('button', { name: 'Release', exact: true }).click();
await page.getByRole('link', { name: 'Released Sequences' }).click();

await page.waitForTimeout(35000);
await page.reload();

await page.getByRole('cell', { name: 'Colombia' }).click();
await page.waitForSelector('text="2012-12-12"');
await page.waitForSelector('text="NC_005301.3"'); // reference
await page.waitForSelector('text="NC_005300.2"');
await page.waitForSelector('text="NC_005302.1"');
});
});
6 changes: 6 additions & 0 deletions integration-tests/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"compilerOptions": {
"esModuleInterop": true,
"moduleResolution": "node"
}
}
8 changes: 3 additions & 5 deletions website/src/components/Edit/DataRow.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,15 @@ export const EditableDataRow: FC<EditableRowProps> = ({ label, inputField, row,

return (
<tr className='table-fixed w-full'>
<div>
<td className={`w-1/4 relative ${colorClassName}`} data-tooltip-id={'field-tooltip' + row.key}>
{`${label ?? row.key}:`}
</td>
<td className={`w-1/4 relative ${colorClassName}`} data-tooltip-id={'field-tooltip' + row.key}>
<label htmlFor={row.key}>{`${label ?? row.key}:`}</label>
<Tooltip
id={'field-tooltip' + row.key}
place='bottom-start'
content={content}
className='absolute z-50 top-full left-0 mt-1'
/>
</div>
</td>
<td className='pr-3 text-right '>
<ErrorAndWarningIcons row={row} />
</td>
Expand Down
8 changes: 1 addition & 7 deletions website/src/components/Edit/EditPage.spec.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -76,20 +76,14 @@ describe('EditPage', () => {
expectTextInSequenceData.originalMetadata(defaultReviewData.originalData.metadata);
});

test('should show original data and processed data', () => {
test('should show original data', () => {
renderEditPage();

expect(screen.getByText(/Original Data/i)).toBeInTheDocument();
expectTextInSequenceData.originalMetadata(defaultReviewData.originalData.metadata);

expect(screen.getAllByText(/Unaligned nucleotide sequences/i)[0]).toBeInTheDocument();
expectTextInSequenceData.original(defaultReviewData.originalData.unalignedNucleotideSequences);

expect(screen.getByText('processedInsertionSequenceName:')).toBeInTheDocument();
expect(screen.getByText('nucleotideInsertion1,nucleotideInsertion2')).toBeInTheDocument();

expect(screen.getByText('processedInsertionGeneName:')).toBeInTheDocument();
expect(screen.getByText('aminoAcidInsertion1,aminoAcidInsertion2')).toBeInTheDocument();
});

test('should show error and warning tooltips', () => {
Expand Down
Loading
Loading