Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch field country in file-download mapping overlaps with other type #1840

Open
defnull opened this issue Mar 3, 2021 · 1 comment

Comments

@defnull
Copy link

defnull commented Mar 3, 2021

For a freshly installed instance (following the docker-compose based deployment documentation) elasticsearch logs the following error messages multiple times a second:

elasticsearch_1  | [2021-03-03 14:47:26,337][WARN ][cluster.action.shard     ] [Holly] [events-stats-file-download-2021-03][4] received shard failed for target shard [[events-stats-file-download-2021-03][4], node[bqEa69qYTwCwxaNwyyihOA], [P], v[1040], s[STARTED], a[id=FTNHGGHDRbWo81cnWYWr3A]], indexUUID [7L3TMWTRSNm5bxWsKpdBZQ], message [master [{Holly}{bqEa69qYTwCwxaNwyyihOA}{172.18.0.2}{172.18.0.2:9300}] marked shard as started, but shard has not been created, mark shard as failed]
elasticsearch_1  | [2021-03-03 14:47:26,344][WARN ][indices.cluster          ] [Holly] [events-stats-file-download-2021-02] failed to add mapping [file-download], source [{"file-download":{"_all":{"enabled":false},"_source":{"enabled":false},"dynamic_templates":[{"date_fields":{"mapping":{"format":"strict_date_hour_minute_second","type":"date"},"match_mapping_type":"date"}}],"date_detection":false,"properties":{"bucket_id":{"type":"string","index":"not_analyzed"},"collection":{"type":"string","index":"not_analyzed"},"country":{"type":"string","index":"not_analyzed"},"file_id":{"type":"string","index":"not_analyzed"},"file_key":{"type":"string","index":"not_analyzed"},"timestamp":{"type":"date","format":"strict_date_hour_minute_second"},"unique_id":{"type":"string","index":"not_analyzed"},"visitor_id":{"type":"string","index":"not_analyzed"}}}}]
elasticsearch_1  | java.lang.IllegalArgumentException: Field [country] is defined as a field in mapping [file-download] but this name is already used for an object in other types

I'm not sure where this comes from (b2share or invenio) but I guess this is triggered by the invenio_stats.tasks.process_events celery task. Any ideas?

@defnull
Copy link
Author

defnull commented Mar 4, 2021

Hmm strange. The events-stats-file-download-* index contains a stats-file-download type with a complex country field mapping (containing geoname_id, iso_code and names sub-fields). I cannot find where this mapping is created or defined.

This conflicts with an index template called file-download-v1 which is installed by invenio-stats into the ES cluster. That template defines country as a simple string. Two types with conflict mappings cannot live in the same index, thus the error form ES.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant