diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml index 5e8d39a..d03c83c 100644 --- a/.github/workflows/codespell.yml +++ b/.github/workflows/codespell.yml @@ -1,4 +1,3 @@ ---- name: Codespell on: diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index e3ab0dd..467714d 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -35,4 +35,4 @@ jobs: # The GH actions bot is used by default if you didn't specify the two fields. # You can swap them out with your own user credentials. user_name: github-actions[bot] - user_email: 41898282+github-actions[bot]@users.noreply.github.com \ No newline at end of file + user_email: 41898282+github-actions[bot]@users.noreply.github.com diff --git a/docs/guide/Project/Curation.md b/docs/guide/Project/Curation.md index 68b1aae..a2d5996 100644 --- a/docs/guide/Project/Curation.md +++ b/docs/guide/Project/Curation.md @@ -5,32 +5,32 @@ sidebar_position: 0 # Curation -*Curation* is the first step in creating a meta-analysis, and begins by *searching* for and *importing* studies into the project. Next, you will *review* studies for inclusion based on their relevancy to your research question. This involves **excluding** irrelevant studies, and **including** relevant ones. +_Curation_ is the first step in creating a meta-analysis, and begins by _searching_ for and _importing_ studies into the project. Next, you will _review_ studies for inclusion based on their relevancy to your research question. This involves **excluding** irrelevant studies, and **including** relevant ones. At the end of the process, you will be ready to create a [**Studyset**](/compose-docs/guide/glossary#studyset) of related studies that are amenable for neuroimaging meta-analysis ## Getting Started -The curation interface uses columns as a way to label studies based on their current status. -The left most column is the starting point and all studies imported into the project will begin there. -The right most column is the **inclusion column** and is the place where studies will go to be included in the meta-analysis. +The curation interface uses columns as a way to label studies based on their current status. +The left most column is the starting point and all studies imported into the project will begin there. +The right most column is the **inclusion column** and is the place where studies will go to be included in the meta-analysis. The goal is to get studies from the left most starting column, and narrow them down to a final subset of included studies in the right most column. If a study is not eligible for inclusion at any point, it should be marked as "excluded" before reaching the inclusion column. The curation step is complete when all studies have been categorized, either by being included or excluded. -When you first begin *Curation*, you will choose between different workflows, which vary in how rigorous or systematic the selection of studies will be. +When you first begin _Curation_, you will choose between different workflows, which vary in how rigorous or systematic the selection of studies will be. :::tip How specific to be? -Performing a systematic meta-analysis involves a substantial amount of effort. It is up to you, the researcher, how rigorous to be in this process. We reccomend thinking about your *goals* prior to starting this process so that you can have clear inclusion guidelines. If you are looking for an exploratory analysis, we reccomend following the tutorial for *automated meta-analysis*, which replaces manual curation with an automated selection of studies. Note that automated meta-analyses are not a replacement for a careful systematic meta-analysis. -::: +Performing a systematic meta-analysis involves a substantial amount of effort. It is up to you, the researcher, how rigorous to be in this process. We reccomend thinking about your _goals_ prior to starting this process so that you can have clear inclusion guidelines. If you are looking for an exploratory analysis, we reccomend following the tutorial for _automated meta-analysis_, which replaces manual curation with an automated selection of studies. Note that automated meta-analyses are not a replacement for a careful systematic meta-analysis. +::: ### PRISMA Workflow PRISMA stands for the **Preferred Reporting Items for Systematic Review and Meta-Analyses**. The -[PRISMA guidelines](http://www.prisma-statement.org/?AspxAutoDetectCookieSupport=1) are a set of rules for reporting a +[PRISMA guidelines](http://www.prisma-statement.org/?AspxAutoDetectCookieSupport=1) are a set of rules for reporting a systematic review, and are the gold standard for producing a proper, rigorous, and transparent meta-analysis. If you are trying to create a rigorous [PRISMA compliant manual meta-analysis](/compose-docs/tutorial/manual) you will want to select this option. @@ -42,14 +42,14 @@ After importing studies into neurosynth, they will be placed into the identifica :::info PRISMA Summary The **Identification** column is where all imported studies are deposited into initially. As alluded to by the name, the identification -column is where you identify all records yielded from your search. In this column, you wil exclude studies purely based on whether they are +column is where you identify all records yielded from your search. In this column, you wil exclude studies purely based on whether they are duplicates of existing studies. All other studies are promoted. -The **Screening** column consists of all records that have been screened for duplicates. In this column, you want to review the titles/abstracts of studies -and exclude purely based on whether they are irrelevant to your research question or domain. All other studies are promoted. +The **Screening** column consists of all records that have been screened for duplicates. In this column, you want to review the titles/abstracts of studies +and exclude purely based on whether they are irrelevant to your research question or domain. All other studies are promoted. -The **Eligibility** column consists of all records that have been screened for duplicates and irrelevant content. In this column, you want to -review the full text of studies and exclude based on whether the study described aligns with the standards of the meta-analysis itself. Reasons +The **Eligibility** column consists of all records that have been screened for duplicates and irrelevant content. In this column, you want to +review the full text of studies and exclude based on whether the study described aligns with the standards of the meta-analysis itself. Reasons for exclusion may include wrong setting, wrong patient population, wrong intervention, wrong paragdigm, etc. The **Included** column consists of all records that have passed previous levels of exclusion and can be considered for the next step of the project. @@ -65,17 +65,17 @@ This workflow is initalized with only two columns. As before, the left most colu ## Importing -To begin importing studies into your project clicking the **IMPORT STUDIES** button. +To begin importing studies into your project click the **IMPORT STUDIES** button. -You will be asked to choose a source, name your import, and address +You will be asked to choose an import method, source, name for your import, and finally address any potential duplicate studies. :::tip -We recommend giving each import a meaningful name. This will be useful when you come back +We recommend giving each import a meaningful name. This will be useful when you come back and want to know where a certain study was imported from. ::: -### Import from Neurostore +### Import Method 1: Import from Neurostore Neurostore indexes a large number of neuroimaging studies which are ready for meta-analysis. Studies in Neurostore have been pre-processed, including extracting peak activation coordinates from Tables in the text, and computing semantic features from the abstract/full text. Neurostore also indexes studies which other users have annotated and made available to others for re-use. @@ -86,61 +86,65 @@ After entering your search, click the bottom right button to import the searched Importing from Neurostore can save you a lot of time, as these studies are much more likely to be amenable to meta-analysis, and have pre-extracted coordinates. However, note that some manual annotation may still be required to verify the coordinate extraction, and choose the relevant Analysis (i.e. contrast) for final inclusion. ::: -### Import from PubMed +### Import Method 2: Import from PubMed -Use this option to import studies directly from PubMed. To start, you need to go to the [PubMed Site](https://pubmed.ncbi.nlm.nih.gov/) -and either enter in a search or navigate to a previously created collection. +Use this option to import studies directly from PubMed. To start, you need to go to the [PubMed Site](https://pubmed.ncbi.nlm.nih.gov/) +and either enter in a search or navigate to a previously created collection. To import the search or collection from PubMed into neurosynth-compose, you will need a text file containing a list of PMIDs. -You can obtain this by going to your collection/search and clicking the **Save** button. Set **Selection** to **All results** +You can obtain this by going to your collection/search and clicking the **Save** button. Set **Selection** to **All results** and set **Format** to **PMID**. Click **Create file** and your text file containing the PMIDs will be generated and downloaded. :::caution Neurosynth-Compose uses the PubMed API to import studies. As a result, the maximum number of PMIDs that can be imported at once is 1,500. -If your collection has more than 1,500 PMIDs, split the import into multiple files. +If your collection has more than 1,500 PMIDs, split the import into multiple files. You can select the same import name so that they are all grouped together. ::: -### Import from BibTex/RIS/endnote +### Import Method 3: Import from BibTex/RIS/endnote Use this option to import studies via a .bib, .RIS, or .enw file. This may be useful if you want to import from a citation manager like Zotero. -### Custom Studies +### Import Method 4: Custom Studies -If there is any record that cannot be easily imported using one of the methods listed above, you can also manually create a study. This may be necessary +If there is any record that cannot be easily imported using one of the methods listed above, you can also manually create a study. This may be necessary to include resources like unpublished studies. +### Sleuth files + +At the moment, it's not possible to add studies from a sleuth file into an _existing project_. To create a new project from a sleuth file, [read more here](/compose-docs/guide/glossary#studyset). + ### Duplicates -If duplicates are detected in your import, you will be asked how to re-concile them by choosing which of the studies to keep by choosing "KEEP THIS STUDY". Matching duplicates will be automatically marked for exclusion. Note that as a user, you can over-ride any of these selection at any time, and choose which studies to keep or exclude. +If duplicates are detected in your import, you will be asked how to reconcile them by choosing which of the studies to keep by choosing "KEEP THIS STUDY". Matching duplicates will be automatically marked for exclusion. Note that as a user, you can override any of these selections at any time, and choose which studies to keep or exclude. :::info -*Neurosynth Compose* does not delete studies. If a study is marked as a duplicate, it is marked as **excluded** but it is not discarded to ensure complete provenance. +_Neurosynth Compose does not delete studies_. If a study is marked as a duplicate, it is marked as **excluded** but it is not discarded to ensure complete provenance. ::: There are two potential ways that duplicates can occur: #### Duplicates are detected between the studies being imported and the studies already in the project -If any study being important has the same title, PMID, or DOI is the same as +If any study being important has the same title, PMID, or DOI is the same as one already in the project, you will be asked reconcile these duplicates. -Note that if you mark a study that is already promoted as a duplicate, it will be "demoted" back to the first column and marked as "duplicate". It is reccomended to mark as duplicate the incoming study to avoid this. +Note that if you mark a study that is already promoted as a duplicate, it will be "demoted" back to the first column and marked as "duplicate". It is reccomended to mark as duplicate the incoming study to avoid this. #### Duplicates are detected within the file you are importing -Although rare, it is possible to have duplicates within a given import. For example, if within a RIS file there are duplicate entries. In this case, you will be asked to select which study to keep. Studies marked as duplicates will still be imported but marked as excluded. +Although rare, it is possible to have duplicates within a given import. For example, if within a RIS file there are duplicate entries. In this case, you will be asked to select which study to keep. Studies marked as duplicates will still be imported but marked as excluded. ## Excluding and Promoting Studies -Once studies have been imported into the first column of the curation phase, they need to be reviewed for inclusion into your meta-analysis. +Once studies have been imported into the first column of the curation phase, they need to be reviewed for inclusion into your meta-analysis. All studies must be either excluded or moved to the inclusion column in order to progress to the extraction phase. -To begin, either click on the button at the top of the column or click on any study in the column. This will open up a page which will show the study -along with the following options: **PROMOTE, NEEDS REVIEW**, and **EXCLUDE**. There is also a button to **ADD TAGS** which will assign an informational tag to the study. +To begin, either click on the button at the top of the column or click on any study in the column. This will open up a page which will show the study +along with the following options: **PROMOTE, NEEDS REVIEW**, and **EXCLUDE**. There is also a button to **ADD TAGS** which will assign an informational tag to the study. This informational tag is a tool that allows you to mark studies with certain information - once an informational tag is created, you can filter by those tags. ### Exclude -To exclude a study, click the **EXCLUDE** button and select the exclusion reason. You can either choose from the preset exclusion reasons or you can begin +To exclude a study, click the **EXCLUDE** button and select the exclusion reason. You can either choose from the preset exclusion reasons or you can begin typing to create a new one. :::tip @@ -148,7 +152,6 @@ For the PRISMA workflow, the default exclusion reason wil depend on the phase yo Revisit the [PRISMA workflow](./Curation#prisma-workflow) to review reccomended exclusion criteria. ::: - ### Promote If a study meets inclusion critera (for the current phase), click **PROMOTE** to move the study forward to the next curation column. If it is moved into the right most inclusion column, then it will be included in the meta-analysis. @@ -157,9 +160,13 @@ If a study meets inclusion critera (for the current phase), click **PROMOTE** to For the first column (especially in a PRISMA workflow) it can be tedious to promote non-duplicates to the next column. If all duplicates have been resolved, you can exit the dialog and click **PROMOTE ALL UNCATEGORIZED STUDIES** to advance all non-duplicate studies to the next column. ::: +## Downloading included studies + +In order to download the studies that you have included in Curation, you can utilize our download feature. Go to curation and click "Download Included as CSV". Alternatively, you can click the dropdown and download the included studies in BibTeX format instead. + ## On to Extraction -When you have categorized all imported studies by either excluding them or moving them to the inclusion column, then you have +When you have categorized all imported studies by either excluding them or moving them to the inclusion column, then you have successfully completed the curation portion of the meta-analysis. -*Neurosynth Compose* will detect this and reveal a new button: **MOVE TO EXTRACTION PHASE**. This will move you back to the project page and get you started on the next major component to building your meta-analysis: [extraction](./Extraction). +_Neurosynth Compose_ will detect this and reveal a new button: **MOVE TO EXTRACTION PHASE**. This will move you back to the project page and get you started on the next major component to building your meta-analysis: [extraction](./Extraction). diff --git a/docs/guide/Project/index.mdx b/docs/guide/Project/index.mdx index 5fed978..2c8ac40 100644 --- a/docs/guide/Project/index.mdx +++ b/docs/guide/Project/index.mdx @@ -5,19 +5,172 @@ sidebar_position: 0 # Project +## Overview + A project organizes the the various steps needed to create a meta-analysis from start to finish. Within a project you will be able to: - 1. **[Curate](./Project/Curation)** studies of interest and select the ones to be included in the meta-analysis - 2. **[Extract](./Project/Extraction)** the relevant data such as activation coordinates and other meta-data - 3. **[Specify](./Project/Specification)** the algorithm and corrector you would like to use -In each project, you can define a define a single StudySet (i.e. a collection of related studies), and one or more MetaAnalysis specifications. +1. **[Curate](./Project/Curation)** studies of interest and select the ones to be included in the meta-analysis +2. **[Extract](./Project/Extraction)** the relevant data such as activation coordinates and other meta-data +3. **[Specify](./Project/Specification)** the algorithm and corrector you would like to use + +In each project, you can define a single studyset (i.e. a collection of related studies defined and used in the extraction phase), and one or more meta-analysis specifications. + +## Create a project + +There are two ways to create a project and do your meta-analysis. + +### Create a blank project + +The first method is to click on the "NEW PROJECT" button in the top navigation menu. This will create a new blank project. You will then be asked to get started on the [curation phase](/compose-docs/guide/Project/Curation) in order to progress. + +### Create a project from a sleuth file + +The second method of creating a project is to click on the dropdown button on the right of "NEW PROJECT". This will show an option: "Create project from sleuth file". Click on that option and you will be brought to a new page to begin creating a new project from your sleuth file. + +Please note that neurosynth-compose requires a specific format in order to proceed with the import. We require that each experiment within the sleuth file _references either a PubMed ID or a DOI_. The exact requirements are specified below: + +#### Sleuth file format requirements + +##### 1. File Reference. + +This reference specifies the coordinate space and must appear at the top of the file. Do not omit the first two forward slashes.
+ +``` +// Reference=MNI +``` + +##### 2. Identifier (DOI or PubMed Id) + +The next lines of the file represents an experiment. We expect an identifier: _this can either be a DOI, a PubMedId, or both_. Do not omit the first two forward slashes.
+ +``` +// DOI=10.1038/nmeth.1635 +or +// PubMedId=21706013 +or +// DOI=10.1038/nmeth.1635 +// PubMedId=21706013 +``` + +##### 3. Author, year & experiment name + +We expect a line containing the authors, year, and experiment name. The authors + year should be separated from the experiment name by a colon. There can be 1 or more lines representing multiple authors + year and experiment name. Do not omit the first two forward slashes. + +``` +// Smith et al., 1996: 1 Back vs 0 Back +or +// Smith et al., 1996: 1 Back vs 0 Back +// Graeff et al., 1995: 2 Back vs 1 Back +// Edwards et al., 2017: 2 Back vs 0 Back +``` + +##### 4.Subjects + +After the lines representing one or more authors + year and experiment name, add a line for the number of subjects in the experiment. + +``` +// Subjects=23 +``` + +##### 5. Coordinates + +Add the X, Y, and Z coordinates. Do not add the two forward slashes. The coordinates should be **delimited by tabs**. + +``` +-27 34 72 +-7 -8 -9 +10 -12 -62 +``` + +Your sleuth file should look something like this: + +``` +// Reference=MNI +// DOI=10.1016/1234567 +// PubMedId=67123237 +// Smith et al., 2019: Working Memory vs Baseline +// Subjects=23 +-2 8 9 +10 -12 -62 +21 -14 -2 +0 -9 16 + +// DOI=10.217/1234568 +// PubMedId=23782389 +// Roberts et al., 1995: 2 Back vs 1 Back +// Graeff et al., 2000: 1 Back vs 0 Back +// Edwards et al., 2017: 2 Back vs 0 Back +// Subjects=62 +82 12 0 +-27 34 72 +-7 -8 -9 +10 -12 -62 +``` + +#### Common Import errors + +`Either DOI or PMID is required. (Hint: is it in the right format?) Encountered error at <...>`
+This error indicates that neither a DOI nor PMID was found at the given line. Please check that the DOI or PubMedId given was correctly formulated. + +``` +// DOI=10.1038/nmeth.1635 +// PubMedId=21706013 +``` + +`Unexpected format. (Hint: Did you omit a colon or use a semi colon instead of a colon?) Encountered error at: <...>`
+This error indicates that neurosynth-compose expected to find a valid author + year and experiment name separated by a colon. Instead, it found a string that it did not recognize (or no experiment name was provided at all). Check that the line of the file indicated in the error message contains a correctly formulated author + year and experiment pairing like: + +``` +// Roberts et al., 1995: 2 Back vs 1 Back +``` + +or multiple + +``` +// Roberts et al., 1995: 2 Back vs 1 Back +// Graeff et al., 2000: 1 Back vs 0 Back +// Edwards et al., 2017: 2 Back vs 0 Back +``` + +`Expected valid DOI but did not find one. Encountered error at: <...>`
+This error indicates that the format of the file did not adhere to the expected requirements due to an invalid DOI. Check that the line of the file indicated in the error message contains a correctly formulated DOI like: `// DOI=10.1038/nmeth.1635` + +`At least one experiment name is required. Encountered error at: <...>`
+This error indicates that no experiments were found at the line given by the error message. Please check that the sleuth file has at least one author + year and experiment name, or that the format is correctly formatted like: `// Smith et al., 2019: Working Memory vs Baseline` + +`Expected valid PMID but did not find one. Encountered error at: <...>`
+This error indicates that the format of the file did not adhere to the expected requirements due to an invalid PubMedId. Check that the line of the file indicated in the error message contains a correctly formulated PubMedId like: `// PubMedId=21706013` + +`Encountered multiple DOIs: <...>`
+This error indicates that multiple DOIs were discovered for the same experiment while parsing the file. Check that the line of the file indicated is part of an experiment that only has one DOI. + +`Encountered multiple PubMed IDs: <...>`
+This error indicates that multiple PubMedIds were discovered for the same experiment while parsing the file. Check that the line of the file indicated is part of an experiment that only has one PubMedId. + +`Expected number of subjects. Encountered error at: <...>`
+This error indicates that the format of the file did not adhere to the expected requirements due to an invalid Subjects field. Check that the line of the file indicated in the error message contains a correctly formulated Subjects field like: `// Subjects=23`. Make sure that it is not separated by a new line. + +`Invalid coordinates: <...>`
+The coordinates given were not in a valid format. Check that the line of the file indicated in the error message contains correctly formulated, tab separated coordinates. They should look like: + +``` +82 12 0 +-27 34 72 +-7 -8 -9 +10 -12 -62 +10 -12 -62 +21 -14 -2 +0 -9 16 +``` + +`Unexpected format. Encountered error at: <...>`
+This error indicates that neurosynth-compose tried to extract the author data from the file but was unable to do so either because the author was not provided or the string was formed in such a way that was not recognized. Check that the line of the file indicated in the error message contains a correctly formulated string like: `// Roberts et al., 1995: 2 Back vs 1 Back` +## Opening a project -You can open a specific project by logging in, navigating to the -[My Projects](https://compose.neurosynth.org/projects) page, and selecting a project you've created. +You can open a specific project by logging in, navigating to the +[My Projects](https://compose.neurosynth.org/projects) page, and selecting a project you've created. -When you view a project for the first time, you'll notice that you'll default to the "Edit Project" tab. -The "View Meta-Analyses" tab will be disabled and will become enabled when you have created the first meta-analysis specification -for the project. \ No newline at end of file +When you view a project for the first time, you'll notice that you'll default to the "Edit Project" tab. Another tab, "View Meta-Analyses", will become visible when you have completed both the curation and extraction phases.