From cfc274fe279ee4caef12c32a4ac058b4669e950f Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 14:20:28 +0200 Subject: [PATCH 01/10] move definition of derivatives --- src/derivatives/introduction.md | 4 +--- src/schema/objects/common_principles.yaml | 6 ++++++ src/schema/rules/common_principles.yaml | 2 ++ 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 669719cdc1..2c9ef93438 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -6,9 +6,7 @@ outputs in subsequent processing. Standardizing derivatives is motivated by use cases where formalized machine-readable access to processed data enables higher-level processing. -The following sections cover additions to and divergences from "raw" BIDS. -Raw data are data that have been curated into BIDS from a non-BIDS source. -If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset. +The following sections cover additions to and divergences from raw BIDS. Examples: diff --git a/src/schema/objects/common_principles.yaml b/src/schema/objects/common_principles.yaml index 7e669efb52..03bab82d7e 100644 --- a/src/schema/objects/common_principles.yaml +++ b/src/schema/objects/common_principles.yaml @@ -46,6 +46,9 @@ dataset: description: | A set of neuroimaging and behavioral data acquired for a purpose of a particular study. A dataset consists of data acquired from one or more subjects, possibly from multiple sessions. +derivative: + display_name: derivative dataset + description: If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset. deprecated: display_name: DEPRECATED description: | @@ -97,6 +100,9 @@ modality: the technique is sufficiently uniform to define the modalities `eeg`, `meg` and `ieeg`. When applicable, the modality is indicated in the **suffix**. The modality may overlap with, but should not be confused with the **data type**. +raw: + display_name: raw dataset + description: A raw BIDS dataset is data that have been curated into BIDS from a non-BIDS source. run: display_name: Run description: | diff --git a/src/schema/rules/common_principles.yaml b/src/schema/rules/common_principles.yaml index 8fac3a813d..ea177ed9f3 100644 --- a/src/schema/rules/common_principles.yaml +++ b/src/schema/rules/common_principles.yaml @@ -16,3 +16,5 @@ - suffix - extension - deprecated +- raw +- derivative From ecbed3cabf7c476827569bc0c0e1f658f7ddb5de Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 14:52:32 +0200 Subject: [PATCH 02/10] update qmri page --- src/appendices/qmri.md | 71 ++++++++++++++++++++++++++---------------- 1 file changed, 45 insertions(+), 26 deletions(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index 31e3b3ef7d..f249fd3b1b 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -133,10 +133,13 @@ A guide for using macros can be found at Please visit the [file collections appendix](./file-collections.md#magnetic-resonance-imaging) to see the list of currently supported qMRI applications. -### Quantitative maps are derivatives +### Outputs are quantitative maps -Regardless of how they are obtained (pre- or post-generated), qMRI maps are stored in the `derivatives` directory. -For example a `T1map` can be generated from an `MP2RAGE` file collection using either options. +qMRI maps are stored differently depending on the process that generated them. +Pre-generated qMRI maps MUST be stored as part of raw BIDS datasets, +where as it MUST be stored in a derivative BIDS dataset if it were post-generated. + +See the example below of a `T1map` generated from an `MP2RAGE` file collection using either options. If the map is post-generated: @@ -151,15 +154,25 @@ A guide for using macros can be found at "qMRI-software-name": { "sub-01": { "anat": { - "sub-01_T1map.nii.gz": "", + "sub-01_T1map.nii.gz": "T1 map stored as part a derivative dataset", "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": "", + "sub-01_UNIT1.nii.gz": "UNI T1 stored as part a derivative dataset", "sub-01_UNIT1.json": "", - }, }, }, }, }, + "sub-01": { + "anat": { + "sub-01_inv-1_part-mag_MP2RAGE.nii.gz":"", + "sub-01_inv-1_part-phase_MP2RAGE.nii.gz":"", + "sub-01_inv-1_MP2RAGE.json":"", + "sub-01_inv-2_part-mag_MP2RAGE.nii.gz":"", + "sub-01_inv-2_part-phase_MP2RAGE.nii.gz":"", + "sub-01_inv-2_MP2RAGE.json":"", + }, + }, + }, } ) }} @@ -172,25 +185,30 @@ A guide for using macros can be found at {{ MACROS___make_filetree_example( { "ds-example": { - "derivatives": { - "Siemens": { - "sub-01": { - "anat": { - "sub-01_T1map.nii.gz": "", - "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": "", - "sub-01_UNIT1.json": "", - }, - }, - }, + "sub-01": { + "anat": { + "sub-01_inv-1_part-mag_MP2RAGE.nii.gz":"", + "sub-01_inv-1_part-phase_MP2RAGE.nii.gz":"", + "sub-01_inv-1_MP2RAGE.json":"", + "sub-01_inv-2_part-mag_MP2RAGE.nii.gz":"", + "sub-01_inv-2_part-phase_MP2RAGE.nii.gz":"", + "sub-01_inv-2_MP2RAGE.json":"", + "sub-01_T1map.nii.gz": "T1 map stored as part a raw dataset", + "sub-01_T1map.json": "", + "sub-01_UNIT1.nii.gz": "UNI T1 stored as part a raw dataset", + "sub-01_UNIT1.json": "", }, }, + } } ) }} -Note: Even though the process from which pre-generated qMRI maps are obtained (vendor pipelines) is not known, -vendors generally allow exporting of the corresponding input data. -It is RECOMMENDED to share them along with the vendor outputs, whenever possible for a qMRI method supported by BIDS. +!!! note "Sharing of vendor outputs" + + Even though the process from which pre-generated qMRI maps are obtained (vendor pipelines) is not known, + vendors generally allow exporting of the corresponding input data. + It is RECOMMENDED to share them along with the vendor outputs, + whenever possible for a qMRI method supported by BIDS. ### Example datasets @@ -326,7 +344,7 @@ A guide for using macros can be found at `dataset_description.json`: -```text +```json { "Name": "qMRLab Outputs", "BIDSVersion": "1.5.0", @@ -526,12 +544,13 @@ as an input to offline calculation of a `T1map` using a dictionary lookup approa provided by the stock sequence. Instead, the `magnitude` and `phase` images are exported. Please see the relevant discussion at [qMRLab issue #255](https://github.com/qMRLab/qMRLab/issues/255). -Therefore, the `UNIT1` image provided by the scanner is RECOMMENDED to be stored under the `anat` -raw dataset directory along with the `MP2RAGE` file collection and to be used as the primary input -for quantifying a `T1map`. +Therefore, the `UNIT1` image provided by the scanner +MUST be stored under the `anat` in a raw BIDS dataset +along with the `MP2RAGE` file collection +and to be used as the primary input for quantifying a `T1map`. -If an additional `UNIT1` image is calculated offline, then the output is to be stored in the -`derivatives` directory with necessary provenance information. +If an additional `UNIT1` image is calculated offline, +then the output is to be stored in a derivative BIDS dataset with necessary provenance information. ##### `NumberShots` metadata field From fbd354c5df36870d7b246822896a9789d2a0992c Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:11:11 +0200 Subject: [PATCH 03/10] add comment --- src/appendices/qmri.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index f249fd3b1b..3bc6bf3749 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -154,9 +154,9 @@ A guide for using macros can be found at "qMRI-software-name": { "sub-01": { "anat": { - "sub-01_T1map.nii.gz": "T1 map stored as part a derivative dataset", + "sub-01_T1map.nii.gz": " # --> T1 map stored as part a derivative dataset", "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": "UNI T1 stored as part a derivative dataset", + "sub-01_UNIT1.nii.gz": " # --> UNI T1 stored as part a derivative dataset", "sub-01_UNIT1.json": "", }, }, @@ -193,9 +193,9 @@ A guide for using macros can be found at "sub-01_inv-2_part-mag_MP2RAGE.nii.gz":"", "sub-01_inv-2_part-phase_MP2RAGE.nii.gz":"", "sub-01_inv-2_MP2RAGE.json":"", - "sub-01_T1map.nii.gz": "T1 map stored as part a raw dataset", + "sub-01_T1map.nii.gz": " # --> T1 map stored as part a raw dataset", "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": "UNI T1 stored as part a raw dataset", + "sub-01_UNIT1.nii.gz": " # --> UNI T1 stored as part a raw dataset", "sub-01_UNIT1.json": "", }, }, From 3c1d8aaba118a2bd6908c304b930afd05865d538 Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:16:53 +0200 Subject: [PATCH 04/10] rm MUST --- src/appendices/qmri.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index 3bc6bf3749..e4f32671e2 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -136,8 +136,8 @@ Please visit the [file collections appendix](./file-collections.md#magnetic-reso ### Outputs are quantitative maps qMRI maps are stored differently depending on the process that generated them. -Pre-generated qMRI maps MUST be stored as part of raw BIDS datasets, -where as it MUST be stored in a derivative BIDS dataset if it were post-generated. +Pre-generated qMRI maps MAY be stored as part of a raw BIDS dataset, +where as it MAY be stored in a derivative BIDS dataset if it were post-generated. See the example below of a `T1map` generated from an `MP2RAGE` file collection using either options. From ff1111fe678e0b369a2d6840d9bf2f97a24211c9 Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:39:24 +0200 Subject: [PATCH 05/10] Update src/appendices/qmri.md Co-authored-by: Chris Markiewicz --- src/appendices/qmri.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index e4f32671e2..b3de6caa21 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -550,7 +550,7 @@ along with the `MP2RAGE` file collection and to be used as the primary input for quantifying a `T1map`. If an additional `UNIT1` image is calculated offline, -then the output is to be stored in a derivative BIDS dataset with necessary provenance information. +then the output MUST be stored in a derivative BIDS dataset with necessary provenance information. ##### `NumberShots` metadata field From f9ac6f3e6403e0f5ddbad73b8ff9dbddd9d16677 Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:39:38 +0200 Subject: [PATCH 06/10] Update src/appendices/qmri.md Co-authored-by: Chris Markiewicz --- src/appendices/qmri.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index b3de6caa21..2e8faf3b09 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -137,7 +137,7 @@ Please visit the [file collections appendix](./file-collections.md#magnetic-reso qMRI maps are stored differently depending on the process that generated them. Pre-generated qMRI maps MAY be stored as part of a raw BIDS dataset, -where as it MAY be stored in a derivative BIDS dataset if it were post-generated. +where as they MUST be stored in a derivative BIDS dataset if they were post-generated. See the example below of a `T1map` generated from an `MP2RAGE` file collection using either options. From 3599642247369bc5e2c84db9e9aec2e8a4d29fa0 Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:39:47 +0200 Subject: [PATCH 07/10] Update src/appendices/qmri.md Co-authored-by: Chris Markiewicz --- src/appendices/qmri.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index 2e8faf3b09..f52c15e61f 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -545,7 +545,7 @@ provided by the stock sequence. Instead, the `magnitude` and `phase` images are see the relevant discussion at [qMRLab issue #255](https://github.com/qMRLab/qMRLab/issues/255). Therefore, the `UNIT1` image provided by the scanner -MUST be stored under the `anat` in a raw BIDS dataset +SHOULD be stored under the `anat` in a raw BIDS dataset along with the `MP2RAGE` file collection and to be used as the primary input for quantifying a `T1map`. From a53270bf8fef30a5c5ada0c1f985c4e2d839c2eb Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Mon, 6 May 2024 15:36:16 +0200 Subject: [PATCH 08/10] shorten comments --- src/appendices/qmri.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index f52c15e61f..522d2b039c 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -154,9 +154,9 @@ A guide for using macros can be found at "qMRI-software-name": { "sub-01": { "anat": { - "sub-01_T1map.nii.gz": " # --> T1 map stored as part a derivative dataset", + "sub-01_T1map.nii.gz": " # --> T1 map in a derivative dataset", "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": " # --> UNI T1 stored as part a derivative dataset", + "sub-01_UNIT1.nii.gz": " # --> UNI T1 in a derivative dataset", "sub-01_UNIT1.json": "", }, }, @@ -193,9 +193,9 @@ A guide for using macros can be found at "sub-01_inv-2_part-mag_MP2RAGE.nii.gz":"", "sub-01_inv-2_part-phase_MP2RAGE.nii.gz":"", "sub-01_inv-2_MP2RAGE.json":"", - "sub-01_T1map.nii.gz": " # --> T1 map stored as part a raw dataset", + "sub-01_T1map.nii.gz": " # --> T1 map in a raw dataset", "sub-01_T1map.json": "", - "sub-01_UNIT1.nii.gz": " # --> UNI T1 stored as part a raw dataset", + "sub-01_UNIT1.nii.gz": " # --> UNI T1 in a raw dataset", "sub-01_UNIT1.json": "", }, }, From a653a188c3284e0146892079e32255eccf2c9dd4 Mon Sep 17 00:00:00 2001 From: Remi Gau Date: Tue, 7 May 2024 09:27:01 +0200 Subject: [PATCH 09/10] revert move of def --- src/derivatives/introduction.md | 4 +++- src/schema/objects/common_principles.yaml | 6 ------ src/schema/rules/common_principles.yaml | 2 -- 3 files changed, 3 insertions(+), 9 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 2c9ef93438..669719cdc1 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -6,7 +6,9 @@ outputs in subsequent processing. Standardizing derivatives is motivated by use cases where formalized machine-readable access to processed data enables higher-level processing. -The following sections cover additions to and divergences from raw BIDS. +The following sections cover additions to and divergences from "raw" BIDS. +Raw data are data that have been curated into BIDS from a non-BIDS source. +If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset. Examples: diff --git a/src/schema/objects/common_principles.yaml b/src/schema/objects/common_principles.yaml index 03bab82d7e..7e669efb52 100644 --- a/src/schema/objects/common_principles.yaml +++ b/src/schema/objects/common_principles.yaml @@ -46,9 +46,6 @@ dataset: description: | A set of neuroimaging and behavioral data acquired for a purpose of a particular study. A dataset consists of data acquired from one or more subjects, possibly from multiple sessions. -derivative: - display_name: derivative dataset - description: If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset. deprecated: display_name: DEPRECATED description: | @@ -100,9 +97,6 @@ modality: the technique is sufficiently uniform to define the modalities `eeg`, `meg` and `ieeg`. When applicable, the modality is indicated in the **suffix**. The modality may overlap with, but should not be confused with the **data type**. -raw: - display_name: raw dataset - description: A raw BIDS dataset is data that have been curated into BIDS from a non-BIDS source. run: display_name: Run description: | diff --git a/src/schema/rules/common_principles.yaml b/src/schema/rules/common_principles.yaml index ea177ed9f3..8fac3a813d 100644 --- a/src/schema/rules/common_principles.yaml +++ b/src/schema/rules/common_principles.yaml @@ -16,5 +16,3 @@ - suffix - extension - deprecated -- raw -- derivative From 963467a2408021976c9f58c47cb36a5df72f56d2 Mon Sep 17 00:00:00 2001 From: Chris Markiewicz Date: Thu, 29 Aug 2024 13:42:02 -0400 Subject: [PATCH 10/10] Apply suggestions from code review Co-authored-by: Taylor Salo --- src/appendices/qmri.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/appendices/qmri.md b/src/appendices/qmri.md index 5a26097d0f..6e1e8edd7f 100644 --- a/src/appendices/qmri.md +++ b/src/appendices/qmri.md @@ -137,9 +137,9 @@ Please visit the [file collections appendix](./file-collections.md#magnetic-reso qMRI maps are stored differently depending on the process that generated them. Pre-generated qMRI maps MAY be stored as part of a raw BIDS dataset, -where as they MUST be stored in a derivative BIDS dataset if they were post-generated. +whereas they MUST be stored in a derivative BIDS dataset if they were post-generated. -See the example below of a `T1map` generated from an `MP2RAGE` file collection using either options. +See the example below of a `T1map` generated from an `MP2RAGE` file collection using either option. If the map is post-generated: