Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate the performance impact of index.mapping.total_fields.limit for metricbeat #41705

Open
AndersonQ opened this issue Nov 20, 2024 · 1 comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@AndersonQ
Copy link
Member

The current index.mapping.total_fields.limit for metricbeat (and any beat using the libbeat setup command) is 12,500, which was just increased as 10,000 wan't enough anymore. As this setting can impact elasticsearch's performance, it's important to have actual performance measurements if we're to ever increase it again.

The impacts on performance need to be evaluated for:
- ingest
- search
- dashboards query / load time
- other searches
- if indexes with documents that actually use a large amount of the mapped fields VS documents that use only a small % of the mapped fields. (several different integrations VS just a few)

This is actually a elasticsearch performance investigation, not specifically for metricbeat. Any test might use metricbeat or any other tool capable of producing events similar to what metricbeat might produce.

@AndersonQ AndersonQ added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Nov 20, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

2 participants