[ENH] BEP044 - Stim-BIDS #2022

neuromechanist · 2024-12-22T02:29:37Z

This PR addresses #153 by introducing specifications for handling and standardizing stimulus files and their annotations within the BIDS specifications. The changes focus on improving the organization, referencing, and metadata of stimulus files to enhance consistency, reusability, and efficiency.

Here are the relevant links and documents:

The issue: BEP044: Within-stimuli conditions #153
Google Doc
Example PR: [WIP] An example for the stimuli-structure bids-examples#433
HTML render of the stimuli page : https://bids-specification--2022.org.readthedocs.build/en/2022/modality-specific-files/stimuli.html

Known issues:

The validator fails because extensions in /src/schema/rules/files/raw require datatype. Stimuli might be a special data type that can only be present at the root of the dataset. So, the datatype field is missing for now.
There are some style errors by the remark validators

cc: @bids-standard/bep044 and @monique2208

Implement the standardization of stimulus files and their annotations within the BIDS specifications. * **Add new file `src/modality-specific-files/stimuli.md`** - Describe the specifications for the stimuli directory. - Include guidelines for storing stimulus files and their annotations. - Define what goes into `stimuli.tsv/json`, `annotations.tsv/json`, and `stim-<label>.json`. - Use the same style as other modality-specific docs to design the tables, variables, and examples. * **Modify `src/modality-specific-files/task-events.md`** - Add a section detailing the standardization of stimulus files and their annotations within the BIDS specifications. - Include examples of how to use the `stim_file` and `stim_id` columns in `events.tsv` files. - Provide guidelines for storing stimulus files in the `/stimuli` directory. - Expand the definition of the `stim_file` column to include `stim_id`. * **Modify `src/schema/objects/columns.yaml`** - Update the definition of the `stim_file` column to ensure consistency in stimulus file references. - Add the `stim_id` column definition for `events.tsv` files. * **Modify `src/schema/rules/checks/events.yaml`** - Add a check for missing stimulus files declared in `events.tsv`. - Add a check for missing `stim_id` references in `events.tsv`. * **Modify `src/schema/rules/sidecars/events.yaml`** - Specify the `StimulusPresentation` metadata field for `events.tsv` files. - Include the `stim_id` column in the metadata field specifications. * **Modify `src/schema/objects/entities.yaml`** - Add entities described in the document with proper requirement levels and descriptions. * **Modify `src/schema/objects/suffixes.yaml`** - Add suffixes for `{audio, image, video, audiovideo}`. - Include the file extensions and descriptions for each suffix. * **Add new file `src/schema/rules/sidecars/stimulus.yaml`** - Define sidecar tables for `stimuli.tsv/json`, `annotations.tsv/json`, and `stim-<label>.json`. - Use the same style as other modality-specific docs to design the tables. --- For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/neuromechanist/bids-specification?shareId=XXXX-XXXX-XXXX-XXXX).

Add stimuli specifications to BIDS

for more information, see https://pre-commit.ci

src/modality-specific-files/stimuli.md

VisLab · 2024-12-31T17:47:50Z

src/modality-specific-files/stimuli.md


-The `stimuli.json` file provides detailed descriptions of the columns in the `stimuli.tsv` file. There can be extra entries in the `stimuli.json` in addition to the columns in the `stimuli.tsv` to provide more details about the stimulus.
+In cases where the stimulus is not shared, the `stimuli.tsv` file can be used to provide metadata about the stimuli, including the license, copyright, URL, and description. This is simialr to the use of `stim-<label>_<suffix>.json` files for individual stimuli files. In the case of conflict between the metadata in the `stimuli.tsv` and `stim-<label>_<suffix>.json` files, the metadata in the `stim-<label>_<suffix>.json` file takes precedence.


Again, here _. It should be consistent. The _ appears in several places as does without the underbar. I'm not going to mark them further --- just need to be consistent.

src/modality-specific-files/stimuli.md

for more information, see https://pre-commit.ci

…/2022

Remi-Gau · 2025-01-09T18:35:54Z

will do a bit of clean up to get less red in CI and maybe see if we can get the HTML version of the BEP to render

Remi-Gau · 2025-01-09T19:37:11Z

HTML: stimuli page
https://bids-specification--2022.org.readthedocs.build/en/2022/modality-specific-files/stimuli.html

yarikoptic

initial quick pass with little recommendations etc

yarikoptic · 2025-01-09T21:43:18Z

src/modality-specific-files/stimuli.md

+| suffix      | extensions                      | description                  |
+| ----------- | ------------------------------- | ---------------------------- |
+| audio       | `.wav`, `.mp3`, `.aac`, `.ogg`  | Audio-only stimulus files    |
+| image       | `.jpg`, `.png`, `.svg`          | Static visual stimulus files |


we do include .webm for video below and there is an increasing use of the new https://en.wikipedia.org/wiki/WebP format as "the best" of both jpg and png since provides composition of both words

supports lossy and lossless compression

supports transparency (alpha channel), not only for lossless like in png

supports animation

so I would expect studies to start using .

But may be it is premature since ATM I found no single .webp file among openneuro datasets.

yarikoptic · 2025-01-09T22:07:40Z

src/modality-specific-files/stimuli.md

+```JSON
+{
+    "License": "CC-BY-4.0",
+    "Copyright": "Lab 2023",


we might want to clarify what to state here and whether it should include year , and overall format

may be we should follow https://reuse.software/tutorial/#step-2 and SPDX (reuse uses it too) for license definitions.

yarikoptic · 2025-01-09T22:13:01Z

src/modality-specific-files/task-events.md

+
+The `stim_id` in the events file links to corresponding files:
+
+```Text


needs MACROS___make_filetree_example until someone smart implements parsing... ref: #2014 (comment)

yarikoptic · 2025-01-09T22:17:40Z

src/schema/rules/files/raw/audio.yaml

+    - audio
+  extensions:
+    - .wav
+    - .WAV


I do not think we should allow/encourage mixed casing allowed! Let's stick to lower case . we have .json and .tsv and no .JSON and .TSV . I do not see why these files need to be different

yarikoptic · 2025-01-09T22:20:31Z

src/schema/objects/suffixes.yaml

+    The JSON file associated with each media file should contain information such as License (RECOMMENDED), Copyright
+    (RECOMMENDED), URL (OPTIONAL), and Description (OPTIONAL) to describe the origin and the nature of the media.


I do not think it is advisable to describe all the fields and whether they are required or not -- this duplicates information formalized elsewhere so would be duplication and thus prone to get out of date etc.

Suggested change

The JSON file associated with each media file should contain information such as License (RECOMMENDED), Copyright

(RECOMMENDED), URL (OPTIONAL), and Description (OPTIONAL) to describe the origin and the nature of the media.

The JSON file associated with each media file should contain information to describe the origin and the nature of the media.

here and similarly below for others

yarikoptic · 2025-01-09T22:23:44Z

src/modality-specific-files/stimuli.md

+    }
+  }) }}
+
+Note: The presence of `stimuli.tsv` file indicates that the content of the `/stimuli` directory follows this BIDS specification for stimulus organization. This structure is planned to become mandatory in BIDS 2.0.


Is this a first time we have something like this? I believe we do not even have similar conditioning for derivaties/ yet... I just wonder if validator is "ready" or what needs tobe done? WDYT @effigies ?

Remi-Gau · 2025-01-10T09:35:44Z

mkdocs.yml

@@ -20,6 +20,7 @@ nav:
          - Near-Infrared Spectroscopy: modality-specific-files/near-infrared-spectroscopy.md
          - Motion: modality-specific-files/motion.md
          - Magnetic Resonance Spectroscopy: modality-specific-files/magnetic-resonance-spectroscopy.md
+          - Stimuli: modality-specific-files/stimuli.md


I put it there for now as the file is in modality specific folder, but I think this stimuli BEP should be 'modality agnostic".

See also [Potential BUG] Task events is modality-specific? #1751

@oesteban

I agree @Remi-Gau

+1.
IMHO, the same goes for (task) events. Hopefully, if stimuli moves to the modality agnostic section, the events will also move.

Remi-Gau · 2025-01-10T09:38:38Z

src/schema/objects/entities.yaml

+    This entity is used to indicate which component of a complex
+    representation is being stored. For MRI data, it indicates which component
+    of the complex signal is represented in voxel data. For stimulus files, it can
+    be used to distinguish different parts of a single stimulus, such as chapters
+    in an audiobook or segments of a long movie (for example, `part-1`, `part-2`,
+    `part-epilog`, `part-chapter1`).


we probably do not want to erase all the details from the previous definition

can we have several definition for an entity : mean different things depending on the datatype or suffix... ?

Remi-Gau · 2025-01-10T09:40:01Z

tools/schemacode/src/bidsschematools/tests/test_render_text.py

very hacky ways to make the tests pass

neuromechanist added 12 commits December 14, 2024 14:15

edit stimuli description

91df78b

Update task events to include stim-bids spec

0167a41

Add missing entities

3c2357b

handling GH Action errors

b7b19df

yaml-link issues

aa570ac

remark validation

162cd1c

Move extensions to their own place.

1ef76bc

Revise the extensions

e84cac4

YAML lint

1dd9083

Merge pull request #2 from neuromechanist/add-stimuli-specifications

50ad1dd

Add stimuli specifications to BIDS

Merge branch 'bids-standard:master' into master

77f33b0

neuromechanist requested a review from erdalkaraca as a code owner December 22, 2024 02:29

neuromechanist and others added 4 commits December 30, 2024 16:02

Sidecar description is only needed of rhte individual stimulus files.

4978552

[pre-commit.ci] auto fixes from pre-commit.com hooks

80d3077

for more information, see https://pre-commit.ci

tabular information matching the spec.

a335ebf

Edit stimuli.md to reflect the spec more completely

f370c95

VisLab approved these changes Dec 31, 2024

View reviewed changes

neuromechanist and others added 4 commits December 31, 2024 12:24

Implement Kay's comments for clarity

330b69f

[pre-commit.ci] auto fixes from pre-commit.com hooks

ad24800

for more information, see https://pre-commit.ci

Revert adding BEP044 doc, implementing Kay's suggestions.

a5acd54

Merge remote-tracking branch 'upstream/master' into pr/neuromechanist…

238e645

…/2022

Remi-Gau added 4 commits January 9, 2025 19:36

sty

074f5fb

remark

f197bba

fix

02c6f31

fix

3ae9fe1

yarikoptic requested changes Jan 9, 2025

View reviewed changes

Remi-Gau reviewed Jan 10, 2025

View reviewed changes

yarikoptic mentioned this pull request Jan 13, 2025

BEP for audio/video capture of behaving subjects #1771

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] BEP044 - Stim-BIDS #2022

[ENH] BEP044 - Stim-BIDS #2022

neuromechanist commented Dec 22, 2024 •

edited by Remi-Gau

Loading

VisLab Dec 31, 2024

Remi-Gau commented Jan 9, 2025

Remi-Gau commented Jan 9, 2025

yarikoptic left a comment

yarikoptic Jan 9, 2025

yarikoptic Jan 9, 2025

yarikoptic Jan 9, 2025

yarikoptic Jan 9, 2025

yarikoptic Jan 9, 2025

yarikoptic Jan 9, 2025

Remi-Gau Jan 10, 2025

oesteban Jan 10, 2025

neuromechanist Jan 10, 2025 •

edited

Loading

Remi-Gau Jan 10, 2025

Remi-Gau Jan 10, 2025


		The `stimuli.json` file provides detailed descriptions of the columns in the `stimuli.tsv` file. There can be extra entries in the `stimuli.json` in addition to the columns in the `stimuli.tsv` to provide more details about the stimulus.
		In cases where the stimulus is not shared, the `stimuli.tsv` file can be used to provide metadata about the stimuli, including the license, copyright, URL, and description. This is simialr to the use of `stim-<label>_<suffix>.json` files for individual stimuli files. In the case of conflict between the metadata in the `stimuli.tsv` and `stim-<label>_<suffix>.json` files, the metadata in the `stim-<label>_<suffix>.json` file takes precedence.


		The `stim_id` in the events file links to corresponding files:

		```Text

		The JSON file associated with each media file should contain information such as License (RECOMMENDED), Copyright
		(RECOMMENDED), URL (OPTIONAL), and Description (OPTIONAL) to describe the origin and the nature of the media.

[ENH] BEP044 - Stim-BIDS #2022

Are you sure you want to change the base?

[ENH] BEP044 - Stim-BIDS #2022

Conversation

neuromechanist commented Dec 22, 2024 • edited by Remi-Gau Loading

Choose a reason for hiding this comment

Remi-Gau commented Jan 9, 2025

Remi-Gau commented Jan 9, 2025

yarikoptic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neuromechanist Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neuromechanist commented Dec 22, 2024 •

edited by Remi-Gau

Loading

neuromechanist Jan 10, 2025 •

edited

Loading