Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TG2-AMENDMENT_ESTABLISHMENTMEANS_STANDARDIZED #269

Open
ArthurChapman opened this issue Feb 7, 2024 · 19 comments
Open

TG2-AMENDMENT_ESTABLISHMENTMEANS_STANDARDIZED #269

ArthurChapman opened this issue Feb 7, 2024 · 19 comments
Labels
Amendment Conformance CORE TG2 CORE tests OTHER Parameterized Test requires a parameter Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT TG2 VOCABULARY

Comments

@ArthurChapman
Copy link
Collaborator

ArthurChapman commented Feb 7, 2024

TestField Value
GUID 15d15927-7a22-43f8-88d6-298f5eb45c4c
Label AMENDMENT_ESTABLISHMENTMEANS_STANDARDIZED
Description Proposes an amendment to the value of dwc:establishmentMeans using the bdq:sourceAuthority.
TestType Amendment
Darwin Core Class dwc:Occurrence
Information Elements ActedUpon dwc:establishmentMeans
Information Elements Consulted
Expected Response EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL PREREQUISITES_NOT_MET if dwc:establishmentMeans is bdq:Empty; AMENDED the value of dwc:establishmentMeans if it can be unambiguously matched to a term in the bdq:sourceAuthority; otherwise NOT_AMENDED
Data Quality Dimension Conformance
Term-Actions ESTABLISHMENTMEANS_STANDARDIZED
Parameter(s) bdq:sourceAuthority
Source Authority bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}
Specification Last Updated 2024-02-08
Examples [dwc:establishmentMeans="vag.": Response.status=AMENDED, Response.result=dwc:establishmentMeans="vagrant", Response.comment="dwc:establishmentMeans contains an interpretable value in the bdq:sourceAuthority"]
[dwc:establishmentMeans="cultivated": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:establishmentMeans is not an interpretable value in the bdq:sourceAuthority"]
Source TG2
References
  • Darwin Core Maintenance Group (2021) Establishment Means Controlled Vocabulary List of Terms. Biodiversity Information Standards (TDWG). http://rs.tdwg.org/dwc/doc/em/
  • Groom et al. (2019) Improving Darwin Core for research and management of alien species. Biodiversity Information Science and Services 3: e38084. https://doi.org/10.3897/biss.3.38084
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes
@ArthurChapman ArthurChapman added TG2 Amendment OTHER Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT VOCABULARY Supplementary Tests supplementary to the core test suite. These are tests that the team regarded as not CORE. Conformance Parameterized Test requires a parameter labels Feb 7, 2024
@tucotuco
Copy link
Member

tucotuco commented Feb 7, 2024

The Notes are in error. "Unknown" is not a valid value for the vocabulary. The corresponding standard controlled value "uncertain " and the preferred label "uncertain (unknown, cryptogenic)". See https://dwc.tdwg.org/em/.

@ArthurChapman
Copy link
Collaborator Author

ArthurChapman commented Feb 7, 2024

Like #268, maybe this should also be regarded as CORE now that there is a suitable Vocabulary. Again it received a rating of 6.5.

@ArthurChapman
Copy link
Collaborator Author

ArthurChapman commented Feb 7, 2024

Thanks @tucotuco - correct. I was misreading and going by memory. I've deleted the Note.

@chicoreus
Copy link
Collaborator

@ArthurChapman I would concur, it makes good sense to include in core, some of the data quality needs elicited by TG3 do involve invasive species and the movement of species, so with vocabularies these tests do feel in scope for core.

@Tasilee Tasilee added CORE TG2 CORE tests and removed Supplementary Tests supplementary to the core test suite. These are tests that the team regarded as not CORE. labels Feb 9, 2024
@Tasilee
Copy link
Collaborator

Tasilee commented Feb 9, 2024

I've removed the Note as we seem to agree this is now CORE.

@ArthurChapman
Copy link
Collaborator Author

The four tests (#277, #278, #268, #269) should be CORE (I have discussed this with Lee). Some reasons are

  • Invasive species are subject of the Convention on Biological Diversity Article 8(h) (https://www.cbd.int/idb/2009/about/cbd) and with a strong use case in the Cop 6 Decision last year VI/23 (https://www.cbd.int/decision/cop/default.shtml?id=7197)
  • These terms are recent additions to Darwin Core and are aspirational (we want all adding data follow the Standard)
  • There are good, well thought out vocabularies and APIs available
  • Apart from the CBD Use Case, there are good Use Case arguments given in Groom et al. (reference above)
  • There is no reason under our current criteria for excluding these tests from CORE

@chicoreus
Copy link
Collaborator

We should be using the following value for the source authority:

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/]}

See: #268 (comment)

@Tasilee
Copy link
Collaborator

Tasilee commented Feb 24, 2024

Thanks @chicoreus - Changing Source Authority from

[bdq:sourceAuthority default = "Darwin Core establishmentMeans" {[https://dwc.tdwg.org/list/#dwc_establishmentMeans]} {dwc:establishmentMeans vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/]}

@tucotuco
Copy link
Member

I think https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts is OK as a source authority because it actually has an API, as long as it is understood that the actually vocabulary is maintained at https://dwc.tdwg.org/em/ and the GBIF API is expected to remain up to date with that.

@Tasilee
Copy link
Collaborator

Tasilee commented Mar 25, 2024

https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts added back in. Dunno why it was changed back.

@Tasilee
Copy link
Collaborator

Tasilee commented Apr 16, 2024

Changed Source Authority from

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "GBIF EstablishmentMeans Vocabulary" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans]} {"dwc:establishmentMeans vocabulary API" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

@tucotuco
Copy link
Member

Source Authority should be

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

@Tasilee
Copy link
Collaborator

Tasilee commented Apr 16, 2024

Changed Source Authority from

bdq:sourceAuthority default = "GBIF EstablishmentMeans Vocabulary" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans]} {"dwc:establishmentMeans vocabulary API" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

@chicoreus
Copy link
Collaborator

The first example should be corrected to match the controlled vocabulary, and a disamiguation case that can be resolved from the controlled vocabulary, thus "Native" instead of "native" and "Native (Indigenous)" instead of "indigenous":

[dwc:establishmentMeans="Native (Indigenous)": Response.status=AMENDED, Response.result=dwc:establishmentMeans="Native", Response.comment="dwc:establishmentMeans contains an interpretable value according to the bdq:sourceAuthority"]

@chicoreus
Copy link
Collaborator

Looking harder, this is actually a problem with the GBIF controlled vocabulary, it differs from the the TDWG vocabulary in case. The correct value is "native", not "Native". Implementors will have to be aware that the expected value needs to be an exact match to the TDWG vocabulary. This applies to several tests.

[dwc:establishmentMeans="Native (Indigenous)": Response.status=AMENDED, Response.result=dwc:establishmentMeans="native", Response.comment="dwc:establishmentMeans contains an interpretable value according to the bdq:sourceAuthority"]

@tucotuco
Copy link
Member

Since a ratified controlled vocabulary actually exists, we should urge GBIF to amend their implementation of the vocabulary.

@chicoreus
Copy link
Collaborator

See #275 (comment)

The GBIF API is only of partial help here, it does not provide the actual Controlled Values from the TDWG vocabulary, the values it has differ in case, implementations can use the GBIF API for its additional values, but need to align the case to match the actual Controlled Value from the TDWG vocabulary for the dwc: term values (GBIF API is sufficient for the dwciri term values).

@chicoreus
Copy link
Collaborator

@tucotuco I agree.

@Tasilee
Copy link
Collaborator

Tasilee commented May 14, 2024

GBIF vocabulary has now been aligned with Darwin Core. Thanks @timrobertson100

chicoreus added a commit to FilteredPush/rec_occur_qc that referenced this issue Jul 27, 2024
…hmentMeans amendment and validation along with default methods and unit tests.
chicoreus added a commit to FilteredPush/rec_occur_qc that referenced this issue Aug 21, 2024
…#276 and tdwg/bdq#278 to recognized variants of controlled vocabulary terms.
chicoreus added a commit to FilteredPush/rec_occur_qc that referenced this issue Aug 25, 2024
…ng matching method to pass. Probably needs to be extended to the other wider net cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Amendment Conformance CORE TG2 CORE tests OTHER Parameterized Test requires a parameter Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT TG2 VOCABULARY
Projects
None yet
Development

No branches or pull requests

4 participants