New data mappings approach #686
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
replace paths
…m/wis2box into new-data-mappings-approach
tomkralidis
requested changes
Jun 3, 2024
tomkralidis
approved these changes
Jun 3, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To allow data-ingestion based on metadata_id, this PR re-structures the "validate_and_load"-functionality that matches input-path with data-plugin using data-mappings.
To enable backwards-compatibility I still allow matching on the topic-hierarchy but the logic in validate_and_load will attempt to match path with metadata_id first.
#584
Note that since the index in elasticsearch is no longer topic-based, this PR breaks the display of data in the wis2box-ui which should be fixed by a change in wis2box-api (see station_info-process).