Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

corrected session_id by replacing invalid characters with underscores… #1227

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/neuroconv/tools/data_transfers/_dandi.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ def automatic_dandi_upload(
with NWBHDF5IO(path=organized_nwbfile, mode="r") as io:
nwbfile = io.read()
session_id = nwbfile.session_id

# Replace invalid characters with underscores
session_id = session_id.replace("/", "_").replace("\\", "_").replace(":", "_")

dandi_stem = organized_nwbfile.stem
dandi_stem_split = dandi_stem.split("_")
dandi_stem_split.insert(1, f"ses-{session_id}")
Expand Down
44 changes: 44 additions & 0 deletions tests/test_minimal/test_tools/dandi_transfer_tools.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of creating more than one file given that dandi organize is meant to work across a set of many files. Let's do 4-5-6 or something like that.

Also, for the last bit of the test I think it is better to just be explicit instead of relying on a loop to make the error obvious and transparent.

What do you think? @alessandratrapani @anoushkajain

Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,47 @@ def test_automatic_dandi_upload_non_parallel_non_threaded(tmp_path):
number_of_jobs=1,
number_of_threads=1,
)


@pytest.mark.skipif(
not HAVE_DANDI_KEY,
reason="You must set your DANDI_API_KEY to run this test!",
)
def test_automatic_dandi_upload_invalid_session_id_chars(tmp_path):
"""Test that invalid characters in session_id are replaced with underscores during DANDI upload."""
nwb_folder_path = tmp_path / "test_nwb"
nwb_folder_path.mkdir()

# Create metadata with session_id containing invalid characters
metadata = get_default_nwbfile_metadata()
invalid_chars_session_id = f"test/invalid\\chars:session-id-{datetime.now().strftime('%Y%m%d%H%M%S')}"
expected_session_id = invalid_chars_session_id.replace("/", "_").replace("\\", "_").replace(":", "_")

metadata["NWBFile"].update(
session_start_time=datetime.now().astimezone(),
session_id=invalid_chars_session_id,
)
metadata.update(Subject=dict(subject_id="foo", species="Mus musculus", age="P1D", sex="U"))

# Create NWB file
nwb_file_path = nwb_folder_path / "test_invalid_chars.nwb"
with NWBHDF5IO(path=nwb_file_path, mode="w") as io:
io.write(make_nwbfile_from_metadata(metadata=metadata))

# Upload to DANDI
organized_nwbfiles = automatic_dandi_upload(
dandiset_id="200560",
nwb_folder_path=nwb_folder_path,
staging=True,
cleanup=False, # Don't clean up so we can check the renamed file
)

# Verify that at least one file was organized
assert len(organized_nwbfiles) > 0

# Check that the organized file name contains the sanitized session_id
for file_path in organized_nwbfiles:
file_path = Path(file_path)
assert (
"ses-" + expected_session_id in file_path.stem
), f"Expected sanitized session ID not found in {file_path.stem}"
Loading