Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Measure metrics latency #2323

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fraillt
Copy link
Contributor

@fraillt fraillt commented Nov 22, 2024

Tests how much time it takes to call .add(value, attribs) under various conditions.
It measures several things: average time, median time (p50), p95 and p99.
The way it works is pretty simple:

  • call .add(value, attribs) on several threads in the loop and measure each call
  • in main thread, keep collecting data (delta collect) with some delay between calls.
    Here's the results on my machine on main branch.
See results.
*** updates, using 4 threads ***
no attribs
	iter 26.43M
	avg 103.0ns
	p50 57.00ns
	p95 177.0ns
	p99 182.0ns
1 attrib
	iter 8.267M
	avg 422.0ns
	p50 362.0ns
	p95 743.0ns
	p99 1.017μs
9 attribs
	iter 4.919M
	avg 718.0ns
	p50 615.0ns
	p95 854.0ns
	p99 1.026μs
*** inserts, using 4 threads ***
1 attrib
	iter 435.1K
	avg 8.931μs
	p50 958.0ns
	p95 64.16μs
	p99 113.8μs
10 attribs
	iter 119.0K
	avg 33.03μs
	p50 4.959μs
	p95 133.9μs
	p99 205.0μs
*** mix mostly updates (100 attribute-sets), using 4 threads ***
10 attribs
	iter 379.1K
	avg 10.36μs
	p50 1.729μs
	p95 67.73μs
	p99 125.9μs
*** updates, using 36 threads ***
no attribs
	iter 40.32M
	avg 682.0ns
	p50 215.0ns
	p95 228.0ns
	p99 243.0ns
1 attrib
	iter 11.72M
	avg 2.991μs
	p50 797.0ns
	p95 1.539μs
	p99 2.264μs
9 attribs
	iter 8.819M
	avg 3.952μs
	p50 933.0ns
	p95 1.442μs
	p99 1.797μs
*** inserts, using 36 threads ***
1 attrib
	iter 279.6K
	avg 128.2μs
	p50 2.757μs
	p95 792.6μs
	p99 1.596ms
10 attribs
	iter 80.01K
	avg 448.5μs
	p50 130.5μs
	p95 1.858ms
	p99 3.044ms
*** mix mostly updates (100 attribute-sets), using 36 threads ***
10 attribs
	iter 1.249M
	avg 28.57μs
	p50 945.0ns
	p95 13.10μs
	p99 871.9μs

Merge requirement checklist

  • CONTRIBUTING guidelines followed
  • Unit tests added/updated (if applicable)
  • Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
  • Changes in public API reviewed (if applicable)

@fraillt fraillt requested a review from a team as a code owner November 22, 2024 10:15
@fraillt fraillt mentioned this pull request Nov 22, 2024
4 tasks
Copy link

codecov bot commented Nov 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.5%. Comparing base (465fcc2) to head (e0df038).

Additional details and impacted files
@@          Coverage Diff          @@
##            main   #2323   +/-   ##
=====================================
  Coverage   79.5%   79.5%           
=====================================
  Files        123     123           
  Lines      21258   21258           
=====================================
+ Hits       16905   16907    +2     
+ Misses      4353    4351    -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

// collect threads in short intervals
let mut total_count = 0;
while threads.iter().any(|t| !t.is_finished()) {
// collect agressively so we could measure inserts properly,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to understand what this is simulating? Metric collection interval is by default 30 sec or 60 sec in the spec. What is the scenario for collecting every 500 microsec?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants