From 1f05d1320f9a343810a6d0cd8ce40d586f6e9419 Mon Sep 17 00:00:00 2001 From: Tiago Lubiana Date: Wed, 5 Jun 2024 09:53:40 -0300 Subject: [PATCH 1/3] Update imports documentation Related to #2388 * Clarifies the language on the contributing file * Reduces and adapts to UpdateImports to CL * Adds link to general OBO Training docs Of note, I haven't changed the "Refresh reports" bit as I lack experience on that front. --- docs/contributing.md | 21 +++-- docs/odk-workflows/UpdateImports.md | 129 +++++----------------------- 2 files changed, 36 insertions(+), 114 deletions(-) diff --git a/docs/contributing.md b/docs/contributing.md index ffdfb1b14..c0444c80a 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -24,12 +24,23 @@ If you want a new term added, or want edits to a current term, or spot any mista - Give a short summary of the pull request - that way we can find suitable reviewers much quicker. Say which terms you are adding or what kinds of changes you are proposing. - It is most of the time a good idea to use `squash merge` rather than `merge` for your pull request, to keep the git history short and useful. -## Pull requests that require imports to be refreshed +## Contributions that use terms from other ontologies not yet referenced in CL -If your pull request references foreign terms from an external ontology that are not yet present in the import module for that ontology (for example, you’re adding a logical definition that makes use of a GO term for the first time), imports needs to be refreshed for the foreign terms to be available to use. +(in jargon, PRs that need imports to be refreshed) -If you have the technical skills and/or the required computer resources (refreshing imports can be a memory-intensive task), you may refresh the imports yourself before submitting the pull request, by following the [appropriate procedure](odk-workflows/UpdateImports.md). +Pull requests to the Cell Ontology often include terms from other external ontologies, such as [UBERON](https://github.com/obophenotype/uberon) or the [Gene Ontology](https://github.com/geneontology/go-ontology). If these terms do not yet exist, they need to be proposed, created, and released by the external source. -If you can’t apply the imports refreshing procedure for any reason, you may instead opt for using “bare IRIs” when editing the ontology, everywhere you need a reference to a foreign term. Then, when submitting your pull request, label it with the tag `update-imports-required` to ask that a member of the tech support group refresh the imports before the pull request can be merged. +If the foreign term exists but is not yet present in the import module of CL (for example, you’re adding a logical definition that makes use of a GO term for the first time), it is necessary to make them available. This requires ''refreshing the imports'', a technical task. +The easiest way to add the imports is by using what is called a Protége-based declaration, or a “bare IRIs” approach. Details on how to do so are available in the [OBO Training documentation](https://oboacademy.github.io/obook/howto/update-import/?h=import#protege-based-declaration). When submitting your pull request, you should label it with the tag `update-imports-required` to ask a member of the tech support group to refresh the imports before the pull request can be merged. -People reviewing pull requests must 1) make sure that if a pull request is referencing bare IRIs, the request is tagged with `update-imports-required` (adding the label themselves if needed); and 2) make sure that imports have indeed been updated (either by the author of the pull request, or by someone from the tech support group if requested) before allowing the request to be merged. +If you have the technical skills and/or the required computer resources (refreshing imports can be a memory-intensive task), you may refresh the imports yourself before submitting the pull request by following the [appropriate procedure](odk-workflows/UpdateImports.md). This approach is generally preferred, as it streamlines updates and reviews, but either is acceptable. + +People reviewing pull requests must: +1. Make sure that if a pull request is referencing bare IRIs, the request is tagged with `update-imports-required` . +2. Make sure that imports have indeed been updated (either by the author of the pull request or by someone from the tech support group if requested) before allowing the request to be merged. + +Additional details on imports are available in the [OBO Training docs](https://oboacademy.github.io/obook/howto/update-import/) and the [CL-specific import documentation](odk-workflows/UpdateImports.md). + +### Why the Cell Ontology does not pull all terms by default? + +If the Cell Ontology pulled all terms by default (from UBERON or the Gene Ontology, for example), that would lead to a tremendous increase in ontology size and the resources needed to run it. Thus, it is necessary to import only a subset of terms from each foreign resource. diff --git a/docs/odk-workflows/UpdateImports.md b/docs/odk-workflows/UpdateImports.md index b0d2d8929..a11e8bb82 100644 --- a/docs/odk-workflows/UpdateImports.md +++ b/docs/odk-workflows/UpdateImports.md @@ -1,27 +1,27 @@ # Update Imports Workflow -This page discusses how to update the contents of your imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see [here](RepoManagement.md). +This page details the import workflows mostly used by CL editors. + +There are several different ways of importing terms, though, and the details of the different approaches not covered here (such as the "Base Module approach") are available at the general [OBO Training Update Imports Workflow with ODK](https://oboacademy.github.io/obook/howto/update-import/). -## Importing a new term -Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title "Using the Base Module approach". +## Importing a new term Importing a new term is split into two sub-phases: -1. Declaring the terms to be imported -2. Refreshing imports dynamically +1. Declaring the terms to be imported (always done by the author of the PR) +2. Refreshing imports dynamically (may be done by the author or _post-hoc_ by the tech team ) ### Declaring terms to be imported -There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be): +There are three ways to declare terms that are to be imported from an external ontology 1. Protégé-based declaration 2. Using term files -3. Using the custom import template +3. Using the custom import template (described only in the [OBO general docs](https://oboacademy.github.io/obook/howto/update-import/)) #### Protégé-based declaration -This workflow is to be avoided, but may be appropriate if the editor _does not have access to the ODK docker container_. -This approach also applies to ontologies that use base module import approach. +This workflow is the simplest, but will require an update by the tech team. 1. Open your ontology (edit file) in Protégé (5.5+). 1. Select 'owl:Thing' @@ -33,53 +33,31 @@ This approach also applies to ontologies that use base module import approach. Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh [here](#refresh-imports)), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology. +Make sure that if a pull request is using Protégé-based declarations and using bare IRIs, the request is tagged with `update-imports-required`. + #### Using term files -Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in `src/ontology/go_import.owl`, you will also have an associated term file `src/ontology/go_terms.txt`. You can add terms in there simply as a list: +The Cell Ontology has several term files associated with each ontology it imports, which can be found in the imports directory ([`cell-ontology/src/ontology/imports/`](https://github.com/obophenotype/cell-ontology/tree/master/src/ontology/imports)). + +For example, you may add a Gene Ontology term to the end of the list at `src/ontology/imports/go_terms.txt`, for example: ``` GO:0008150 GO:0008151 +GO:0004990 +GO:0070278 ``` -Now you can run the [refresh imports workflow](#refresh-imports)) and the two terms will be imported. - -#### Using the custom import template - -This workflow is appropriate if: - -1. You prefer to manage all your imported terms in a single file (rather than multiple files like in the "Using term files" workflow above). -2. You wish to augment your imported ontologies with additional information. This requires a cautionary discussion. - -To enable this workflow, you add the following to your ODK config file (`src/ontology/cl-odk.yaml`), and [update the repository](RepoManagement.md): - -``` -use_custom_import_module: TRUE -``` - -Now you can manage your imported terms directly in the custom external terms template, which is located at `src/templates/external_import.owl`. Note that this file is a [ROBOT template](http://robot.obolibrary.org/template), and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully. +Now you can run the [refresh imports workflow](#refresh-imports)) and the new terms will be imported. -The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import `APOLLO_SV:00000480`, and you wish to import `APOLLO_SV:00000532`, you simply add a row like this: - -``` -ID Entity Type -ID TYPE -APOLLO_SV:00000480 owl:Class -APOLLO_SV:00000532 owl:Class -``` - -When the imports are refreshed [see imports refresh workflow](#refresh-imports), the term(s) will simply be imported from the configured ontologies. - -Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the `ID` and `ENTITY` columns and (b) ensure that the ROBOT template is valid otherwise, [see here](http://robot.obolibrary.org/template). - -_WARNING_. Note that doing this is a _widespread antipattern_ (see related [issue](https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1443)). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the _custom imports module_ as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the [base file](https://github.com/INCATools/ontology-development-kit/blob/master/docs/ReleaseArtefacts.md#release-artefact-1-base-required). ### Refresh imports -If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example): +If you want to refresh the import yourself, and you have the ODK installed, you can do the following (using GO as an example): + +First, you navigate in your terminal to the ontology directory. -First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). ``` cd src/ontology ``` @@ -107,70 +85,3 @@ If you wish to skip refreshing the mirror, i.e. skip downloading the latest vers ``` sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B ``` - -## Using the Base Module approach - -Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. -The idea is to only import axioms from ontologies that _actually belong to it_. -A base file is a subset of the ontology that only contains those axioms that nominally -belong there. In other words, the base file does not contain any axioms that belong -to another ontology. An example would be this: - -Imagine this being the full Uberon ontology: - -``` -Axiom 1: BFO:123 SubClassOf BFO:124 -Axiom 1: UBERON:123 SubClassOf BFO:123 -Axiom 1: UBERON:124 SubClassOf UBERON 123 -``` - -The base file is the set of all axioms that are about UBERON terms: - -``` -Axiom 1: UBERON:123 SubClassOf BFO:123 -Axiom 1: UBERON:124 SubClassOf UBERON 123 -``` - -I.e. - -``` -Axiom 1: BFO:123 SubClassOf BFO:124 -``` - -Gets removed. - -The base file pipeline is a bit more complex than the normal pipelines, because -of the logical interactions between the imported ontologies. This is solved by _first -merging all mirrors into one huge file and then extracting one mega module from it. - -Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. -When we use the base pipelines, we - -1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) -2) We merge all base files into one big pile -3) Then we extract a single module `imports/merged_import.owl` - -The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml. - -To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if `use_base_merging: TRUE` is declared under `import_group` - -If your ontology uses Base Module approach, please use the following steps: - -First, add the term to be imported to the term file associated with it (see above "Using term files" section if this is not clear to you) - -Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). -``` -cd src/ontology -``` - -Then refresh imports by running - -``` -sh run.sh make imports/merged_import.owl -``` -Note: if your mirrors are updated, you can run `sh run.sh make no-mirror-refresh-merged` - -This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you. - -Lastly, restart Protégé, and the term should be imported in ready to be used. - From fe87a1841ddbd9a2e3fb1ee3841ef3fbc42f69ea Mon Sep 17 00:00:00 2001 From: Tiago Lubiana Date: Wed, 5 Jun 2024 10:09:11 -0300 Subject: [PATCH 2/3] Add links to the "Adding classes" doc As @anitacaron pointed in the issue, there is also documentation about imports in the added link. --- docs/contributing.md | 6 +- docs/odk-workflows/UpdateImports.md | 129 ++++++++++++++++++++++++---- 2 files changed, 115 insertions(+), 20 deletions(-) diff --git a/docs/contributing.md b/docs/contributing.md index c0444c80a..32c6526d9 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -39,8 +39,12 @@ People reviewing pull requests must: 1. Make sure that if a pull request is referencing bare IRIs, the request is tagged with `update-imports-required` . 2. Make sure that imports have indeed been updated (either by the author of the pull request or by someone from the tech support group if requested) before allowing the request to be merged. -Additional details on imports are available in the [OBO Training docs](https://oboacademy.github.io/obook/howto/update-import/) and the [CL-specific import documentation](odk-workflows/UpdateImports.md). +Additional details on imports are available in: +* the general CL guideline called ["Adding classes from another ontology"](https://obophenotype.github.io/cell-ontology/Adding_classes_from_another_ontology/). +* [OBO Training docs](https://oboacademy.github.io/obook/howto/update-import/) +* the [CL-specific ODK workflow documentation](odk-workflows/UpdateImports.md). ### Why the Cell Ontology does not pull all terms by default? If the Cell Ontology pulled all terms by default (from UBERON or the Gene Ontology, for example), that would lead to a tremendous increase in ontology size and the resources needed to run it. Thus, it is necessary to import only a subset of terms from each foreign resource. + diff --git a/docs/odk-workflows/UpdateImports.md b/docs/odk-workflows/UpdateImports.md index a11e8bb82..104acf9fe 100644 --- a/docs/odk-workflows/UpdateImports.md +++ b/docs/odk-workflows/UpdateImports.md @@ -1,27 +1,29 @@ # Update Imports Workflow -This page details the import workflows mostly used by CL editors. - -There are several different ways of importing terms, though, and the details of the different approaches not covered here (such as the "Base Module approach") are available at the general [OBO Training Update Imports Workflow with ODK](https://oboacademy.github.io/obook/howto/update-import/). +This page details how to update the imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see [here](RepoManagement.md). +There are some notes available on the current status of the imports in CL at ["Adding classes from another ontology"](https://obophenotype.github.io/cell-ontology/Adding_classes_from_another_ontology/). ## Importing a new term +Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title "Using the Base Module approach". + Importing a new term is split into two sub-phases: -1. Declaring the terms to be imported (always done by the author of the PR) -2. Refreshing imports dynamically (may be done by the author or _post-hoc_ by the tech team ) +1. Declaring the terms to be imported +2. Refreshing imports dynamically ### Declaring terms to be imported -There are three ways to declare terms that are to be imported from an external ontology +There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be): 1. Protégé-based declaration 2. Using term files -3. Using the custom import template (described only in the [OBO general docs](https://oboacademy.github.io/obook/howto/update-import/)) +3. Using the custom import template #### Protégé-based declaration -This workflow is the simplest, but will require an update by the tech team. +This workflow is to be avoided, but may be appropriate if the editor _does not have access to the ODK docker container_. +This approach also applies to ontologies that use base module import approach. 1. Open your ontology (edit file) in Protégé (5.5+). 1. Select 'owl:Thing' @@ -33,31 +35,53 @@ This workflow is the simplest, but will require an update by the tech team. Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh [here](#refresh-imports)), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology. -Make sure that if a pull request is using Protégé-based declarations and using bare IRIs, the request is tagged with `update-imports-required`. - #### Using term files -The Cell Ontology has several term files associated with each ontology it imports, which can be found in the imports directory ([`cell-ontology/src/ontology/imports/`](https://github.com/obophenotype/cell-ontology/tree/master/src/ontology/imports)). - -For example, you may add a Gene Ontology term to the end of the list at `src/ontology/imports/go_terms.txt`, for example: +Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in `src/ontology/go_import.owl`, you will also have an associated term file `src/ontology/go_terms.txt`. You can add terms in there simply as a list: ``` GO:0008150 GO:0008151 -GO:0004990 -GO:0070278 ``` -Now you can run the [refresh imports workflow](#refresh-imports)) and the new terms will be imported. +Now you can run the [refresh imports workflow](#refresh-imports)) and the two terms will be imported. +#### Using the custom import template -### Refresh imports +This workflow is appropriate if: + +1. You prefer to manage all your imported terms in a single file (rather than multiple files like in the "Using term files" workflow above). +2. You wish to augment your imported ontologies with additional information. This requires a cautionary discussion. + +To enable this workflow, you add the following to your ODK config file (`src/ontology/cl-odk.yaml`), and [update the repository](RepoManagement.md): + +``` +use_custom_import_module: TRUE +``` + +Now you can manage your imported terms directly in the custom external terms template, which is located at `src/templates/external_import.owl`. Note that this file is a [ROBOT template](http://robot.obolibrary.org/template), and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully. + +The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import `APOLLO_SV:00000480`, and you wish to import `APOLLO_SV:00000532`, you simply add a row like this: + +``` +ID Entity Type +ID TYPE +APOLLO_SV:00000480 owl:Class +APOLLO_SV:00000532 owl:Class +``` -If you want to refresh the import yourself, and you have the ODK installed, you can do the following (using GO as an example): +When the imports are refreshed [see imports refresh workflow](#refresh-imports), the term(s) will simply be imported from the configured ontologies. -First, you navigate in your terminal to the ontology directory. +Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the `ID` and `ENTITY` columns and (b) ensure that the ROBOT template is valid otherwise, [see here](http://robot.obolibrary.org/template). +_WARNING_. Note that doing this is a _widespread antipattern_ (see related [issue](https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1443)). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the _custom imports module_ as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the [base file](https://github.com/INCATools/ontology-development-kit/blob/master/docs/ReleaseArtefacts.md#release-artefact-1-base-required). + +### Refresh imports + +If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example): + +First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). ``` cd src/ontology ``` @@ -85,3 +109,70 @@ If you wish to skip refreshing the mirror, i.e. skip downloading the latest vers ``` sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B ``` + +## Using the Base Module approach + +Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. +The idea is to only import axioms from ontologies that _actually belong to it_. +A base file is a subset of the ontology that only contains those axioms that nominally +belong there. In other words, the base file does not contain any axioms that belong +to another ontology. An example would be this: + +Imagine this being the full Uberon ontology: + +``` +Axiom 1: BFO:123 SubClassOf BFO:124 +Axiom 1: UBERON:123 SubClassOf BFO:123 +Axiom 1: UBERON:124 SubClassOf UBERON 123 +``` + +The base file is the set of all axioms that are about UBERON terms: + +``` +Axiom 1: UBERON:123 SubClassOf BFO:123 +Axiom 1: UBERON:124 SubClassOf UBERON 123 +``` + +I.e. + +``` +Axiom 1: BFO:123 SubClassOf BFO:124 +``` + +Gets removed. + +The base file pipeline is a bit more complex than the normal pipelines, because +of the logical interactions between the imported ontologies. This is solved by _first +merging all mirrors into one huge file and then extracting one mega module from it. + +Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. +When we use the base pipelines, we + +1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) +2) We merge all base files into one big pile +3) Then we extract a single module `imports/merged_import.owl` + +The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml. + +To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if `use_base_merging: TRUE` is declared under `import_group` + +If your ontology uses Base Module approach, please use the following steps: + +First, add the term to be imported to the term file associated with it (see above "Using term files" section if this is not clear to you) + +Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). +``` +cd src/ontology +``` + +Then refresh imports by running + +``` +sh run.sh make imports/merged_import.owl +``` +Note: if your mirrors are updated, you can run `sh run.sh make no-mirror-refresh-merged` + +This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you. + +Lastly, restart Protégé, and the term should be imported in ready to be used. + From d8ca87f244261b4ef21b989b2c3e386e6b5c0ec1 Mon Sep 17 00:00:00 2001 From: Tiago Lubiana Date: Wed, 5 Jun 2024 10:16:31 -0300 Subject: [PATCH 3/3] Add back the changes in UpdateImports.md --- docs/odk-workflows/UpdateImports.md | 130 +++++----------------------- 1 file changed, 20 insertions(+), 110 deletions(-) diff --git a/docs/odk-workflows/UpdateImports.md b/docs/odk-workflows/UpdateImports.md index 104acf9fe..95b6d8e0e 100644 --- a/docs/odk-workflows/UpdateImports.md +++ b/docs/odk-workflows/UpdateImports.md @@ -1,29 +1,28 @@ # Update Imports Workflow -This page details how to update the imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see [here](RepoManagement.md). +This page details the import workflows mostly used by CL editors. -There are some notes available on the current status of the imports in CL at ["Adding classes from another ontology"](https://obophenotype.github.io/cell-ontology/Adding_classes_from_another_ontology/). +There are several different ways of importing terms, though, and the details of the different approaches not covered here (such as the "Base Module approach") are available at the general [OBO Training Update Imports Workflow with ODK](https://oboacademy.github.io/obook/howto/update-import/). -## Importing a new term +There are some also notes available on the current practice of imports in CL at ["Adding classes from another ontology"](https://obophenotype.github.io/cell-ontology/Adding_classes_from_another_ontology/). -Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title "Using the Base Module approach". +## Importing a new term Importing a new term is split into two sub-phases: -1. Declaring the terms to be imported -2. Refreshing imports dynamically +1. Declaring the terms to be imported (always done by the author of the PR) +2. Refreshing imports dynamically (may be done by the author or _post-hoc_ by the tech team ) ### Declaring terms to be imported -There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be): +There are three ways to declare terms that are to be imported from an external ontology 1. Protégé-based declaration 2. Using term files -3. Using the custom import template +3. Using the custom import template (described only in the [OBO general docs](https://oboacademy.github.io/obook/howto/update-import/)) #### Protégé-based declaration -This workflow is to be avoided, but may be appropriate if the editor _does not have access to the ODK docker container_. -This approach also applies to ontologies that use base module import approach. +This workflow is the simplest, but will require an update by the tech team. 1. Open your ontology (edit file) in Protégé (5.5+). 1. Select 'owl:Thing' @@ -35,53 +34,31 @@ This approach also applies to ontologies that use base module import approach. Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh [here](#refresh-imports)), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology. +Make sure that if a pull request is using Protégé-based declarations and using bare IRIs, the request is tagged with `update-imports-required`. + #### Using term files -Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in `src/ontology/go_import.owl`, you will also have an associated term file `src/ontology/go_terms.txt`. You can add terms in there simply as a list: +The Cell Ontology has several term files associated with each ontology it imports, which can be found in the imports directory ([`cell-ontology/src/ontology/imports/`](https://github.com/obophenotype/cell-ontology/tree/master/src/ontology/imports)). + +For example, you may add a Gene Ontology term to the end of the list at `src/ontology/imports/go_terms.txt`, for example: ``` GO:0008150 GO:0008151 +GO:0004990 +GO:0070278 ``` -Now you can run the [refresh imports workflow](#refresh-imports)) and the two terms will be imported. - -#### Using the custom import template - -This workflow is appropriate if: - -1. You prefer to manage all your imported terms in a single file (rather than multiple files like in the "Using term files" workflow above). -2. You wish to augment your imported ontologies with additional information. This requires a cautionary discussion. - -To enable this workflow, you add the following to your ODK config file (`src/ontology/cl-odk.yaml`), and [update the repository](RepoManagement.md): - -``` -use_custom_import_module: TRUE -``` - -Now you can manage your imported terms directly in the custom external terms template, which is located at `src/templates/external_import.owl`. Note that this file is a [ROBOT template](http://robot.obolibrary.org/template), and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully. - -The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import `APOLLO_SV:00000480`, and you wish to import `APOLLO_SV:00000532`, you simply add a row like this: - -``` -ID Entity Type -ID TYPE -APOLLO_SV:00000480 owl:Class -APOLLO_SV:00000532 owl:Class -``` - -When the imports are refreshed [see imports refresh workflow](#refresh-imports), the term(s) will simply be imported from the configured ontologies. +Now you can run the [refresh imports workflow](#refresh-imports)) and the new terms will be imported. -Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the `ID` and `ENTITY` columns and (b) ensure that the ROBOT template is valid otherwise, [see here](http://robot.obolibrary.org/template). - -_WARNING_. Note that doing this is a _widespread antipattern_ (see related [issue](https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1443)). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the _custom imports module_ as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the [base file](https://github.com/INCATools/ontology-development-kit/blob/master/docs/ReleaseArtefacts.md#release-artefact-1-base-required). ### Refresh imports -If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example): +If you want to refresh the import yourself, and you have the ODK installed, you can do the following (using GO as an example): + +First, you navigate in your terminal to the ontology directory. -First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). ``` cd src/ontology ``` @@ -109,70 +86,3 @@ If you wish to skip refreshing the mirror, i.e. skip downloading the latest vers ``` sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B ``` - -## Using the Base Module approach - -Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. -The idea is to only import axioms from ontologies that _actually belong to it_. -A base file is a subset of the ontology that only contains those axioms that nominally -belong there. In other words, the base file does not contain any axioms that belong -to another ontology. An example would be this: - -Imagine this being the full Uberon ontology: - -``` -Axiom 1: BFO:123 SubClassOf BFO:124 -Axiom 1: UBERON:123 SubClassOf BFO:123 -Axiom 1: UBERON:124 SubClassOf UBERON 123 -``` - -The base file is the set of all axioms that are about UBERON terms: - -``` -Axiom 1: UBERON:123 SubClassOf BFO:123 -Axiom 1: UBERON:124 SubClassOf UBERON 123 -``` - -I.e. - -``` -Axiom 1: BFO:123 SubClassOf BFO:124 -``` - -Gets removed. - -The base file pipeline is a bit more complex than the normal pipelines, because -of the logical interactions between the imported ontologies. This is solved by _first -merging all mirrors into one huge file and then extracting one mega module from it. - -Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. -When we use the base pipelines, we - -1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) -2) We merge all base files into one big pile -3) Then we extract a single module `imports/merged_import.owl` - -The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml. - -To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if `use_base_merging: TRUE` is declared under `import_group` - -If your ontology uses Base Module approach, please use the following steps: - -First, add the term to be imported to the term file associated with it (see above "Using term files" section if this is not clear to you) - -Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). -``` -cd src/ontology -``` - -Then refresh imports by running - -``` -sh run.sh make imports/merged_import.owl -``` -Note: if your mirrors are updated, you can run `sh run.sh make no-mirror-refresh-merged` - -This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you. - -Lastly, restart Protégé, and the term should be imported in ready to be used. -