Skip to content

Commit

Permalink
#569: Add documentation on adding an endpoint
Browse files Browse the repository at this point in the history
  • Loading branch information
matthew-morris-cessda committed Jul 7, 2023
1 parent 337f0b8 commit a131139
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 2 deletions.
21 changes: 21 additions & 0 deletions OPERATIONS_DOCUMENTATION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
# Operations Documentation - CESSDA Data Catalogue

## Adding a new repository to the CESSDA Data Catalogue

Adding a repository to the Data Catalogue is a simple process. There is only one file that needs to be modified, which can be found at <https://github.com/cessda/cessda.cdc.aggregator.deploy/blob/main/charts/harvester/config/config.yaml>.

To create a new repository, create a new entry under `harvester.repos`. Feel free to use an existing repository configuration entry as a reference.

- `code` - Short name used in logs
- `name` - Friendly name shown in the user interface
- `url` - URL of the OAI-PMH endpoint of the repository
- `validationGate` - The CMV validation gate to use. See <https://cmv.cessda.eu/documentation/constraints.html> for validation gate definitions.
- `metadataPrefixes` - Specific harvester configurations

Multiple metadata prefixes and sets can be configured for the same repository.

- `metadataPrefix` - the metadata prefix to harvest
- `setSpec` - the set to harvest, can be omitted
- `validationProfile` - the CMV profile to validate against, see <https://cmv.cessda.eu/documentation/profiles.html>
- `ddiVersion` - the DDI version harvested, currently unused

See <https://github.com/cessda/cessda.metadata.harvester/blob/main/README.md> for the full definition of the configuration file.

## Managing the Elasticsearch (ES) indices

The Elasticsearch cluster used to run the Data Catalogue is available on the catalogues endpoint, i.e. at <https://datacatalogue.cessda.eu/es/>. This endpoint is password protected (the password can be found in 1Password).
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CESSDA Data Catalogue v3.1.1
# CESSDA Data Catalogue

The CESSDA Data Catalogue (CDC) can harvest any XML content provided by an OAI-PMH endpoint. It uses different sets of XPath mappings to adapt the different flavours of the XML payloads to a standard format, namely the [CESSDA Metadata Model](https://doi.org/10.5281/zenodo.4751455).

Expand All @@ -8,7 +8,7 @@ The results of the validation checks are sent to an ElasticSearch index that fee

## Project Structure

The CDC product is made up of several components, which can be grouped as Data Gathering, User Facing, public API and Management. There are also some repositories which are concerned with Documentation & Issue Tracking and QA & Deployment respectively.
The CDC product is made up of several components, which can be grouped as Data Gathering, User Facing, Public API and Management. There are also some repositories which are concerned with Documentation & Issue Tracking and QA & Deployment respectively.

### Data Gathering components

Expand Down

0 comments on commit a131139

Please sign in to comment.