Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flaky test][exporter/elasticsearch] TestExporterMetrics flaky on Windows #35924

Closed
songy23 opened this issue Oct 22, 2024 · 3 comments · Fixed by #36255
Closed

[Flaky test][exporter/elasticsearch] TestExporterMetrics flaky on Windows #35924

songy23 opened this issue Oct 22, 2024 · 3 comments · Fixed by #36255
Assignees
@carsonip
Labels
exporter/elasticsearch flaky test a test is flaky os:windows

Comments

@songy23
Copy link
Member

songy23 commented Oct 22, 2024

Component(s)

exporter/elasticsearch

Describe the issue you're reporting

E.g. https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11453791269/job/31866855448


        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,2 +1,2 @@
        	            	-(map[string]interface {}) (len=5) {
        	            	+(map[string]interface {}) (len=6) {
        	            	  (string) (len=10) "@timestamp": (string) (len=30) "1970-01-01T01:00:00.000000000Z",
        	            	@@ -8,16 +8,3 @@
        	            	  (string) (len=7) "metrics": (map[string]interface {}) (len=1) {
        	            	-  (string) (len=10) "metric.foo": (map[string]interface {}) (len=2) {
        	            	-   (string) (len=6) "counts": ([]interface {}) (len=4) {
        	            	-    (float64) 4,
        	            	-    (float64) 5,
        	            	-    (float64) 6,
        	            	-    (float64) 7
        	            	-   },
        	            	-   (string) (len=6) "values": ([]interface {}) (len=4) {
        	            	-    (float64) 2,
        	            	-    (float64) 4.5,
        	            	-    (float64) 5.5,
        	            	-    (float64) 6
        	            	-   }
        	            	-  }
        	            	+  (string) (len=10) "metric.sum": (float64) 1.5
        	            	  },
        	            	@@ -28,3 +15,4 @@
        	            	   (string) (len=24) "dropped_attributes_count": (float64) 0
        	            	- }
        	            	+ },
        	            	+ (string) (len=15) "start_timestamp": (string) (len=30) "1970-01-01T02:00:00.000000000Z"
        	            	 }
        	Test:       	TestExporterMetrics/otel_mode
        	Messages:   	item 1 document

=== FAIL: . TestExporterMetrics (0.26s)
panic: runtime error: index out of range [2] with length 2 [recovered]
	panic: runtime error: index out of range [2] with length 2

goroutine 673 [running]:
testing.tRunner.func1.2({0x141797520, 0xc0000bb9b0})
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1631 +0x3f7
testing.tRunner.func1()
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1634 +0x6b6
panic({0x141797520?, 0xc0000bb9b0?})
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/panic.go:770 +0x132
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.assertItemsEqual(0xc0001e2340, {0xc00027dcd0, 0x4, 0xc0002a00c0?}, {0xc00007c6c0, 0x2, 0x14005b1e0?}, 0x0)
	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:55 +0x65e
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.TestExporterMetrics.func10(0xc0001e2340)
	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/exporter_test.go:1200 +0x3025
testing.tRunner(0xc0001e2340, 0x1418d20a0)
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1689 +0x21f
created by testing.(*T).Run in goroutine 540
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1742 +0x826

=== FAIL: . TestConfig (unknown)
@songy23 songy23 added needs triage New item requiring triage flaky test a test is flaky os:windows labels Oct 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@songy23 songy23 mentioned this issue Oct 22, 2024
Merged
@atoulme atoulme removed the needs triage New item requiring triage label Oct 25, 2024
@github-actions github-actions bot mentioned this issue Oct 29, 2024
Closed
@pjanotti
Copy link
Contributor

pjanotti commented Nov 1, 2024

Not exactly the same, but, likely related:

https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11632160146/job/32394694302#step:8:480

=== FAIL: . TestExporterMetrics/otel_mode_aggregate_metric_double_hint (0.15s)
    utils_test.go:53: 
ERROR rerun aborted because previous run had a suspected panic and some test may not have run
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:53
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/exporter_test.go:1325
        	Error:      	Not equal: 
        	            	expected: 2
        	            	actual  : 1
make[2]: *** [../../Makefile.Common:131: test] Error 3
        	Test:       	TestExporterMetrics/otel_mode_aggregate_metric_double_hint
        	Messages:   	want 2 items, got 1
    utils_test.go:56: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:56
make[1]: *** [Makefile:200: exporter/elasticsearchexporter] Error 2
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/exporter_test.go:1325
make[1]: *** Waiting for unfinished jobs....
        	Error:      	Not equal: 
        	            	expected: map[string]interface {}{"create":map[string]interface {}{"_index":"metrics-generic.otel-default", "dynamic_templates":map[string]interface {}{"metrics.exphistogram.summary":"summary"}}}
        	            	actual  : map[string]interface {}{"create":map[string]interface {}{"_index":"metrics-generic.otel-default", "dynamic_templates":map[string]interface {}{"metrics.histogram.summary":"summary"}}}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,3 @@
        	            	   (string) (len=17) "dynamic_templates": (map[string]interface {}) (len=1) {
        	            	-   (string) (len=28) "metrics.exphistogram.summary": (string) (len=7) "summary"
        	            	+   (string) (len=25) "metrics.histogram.summary": (string) (len=7) "summary"
        	            	   }
        	Test:       	TestExporterMetrics/otel_mode_aggregate_metric_double_hint
        	Messages:   	item 0 action
    utils_test.go:57: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:57
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/exporter_test.go:1325
        	Error:      	Not equal: 
        	            	expected: map[string]interface {}{"@timestamp":"1970-01-01T01:00:00.000000000Z", "_doc_count":10, "data_stream":map[string]interface {}{"dataset":"generic.otel", "namespace":"default", "type":"metrics"}, "metrics":map[string]interface {}{"exphistogram.summary":map[string]interface {}{"sum":1, "value_count":10}}, "resource":map[string]interface {}{"dropped_attributes_count":0}, "scope":map[string]interface {}{"dropped_attributes_count":0}}
        	            	actual  : map[string]interface {}{"@timestamp":"1970-01-01T00:00:00.000000000Z", "_doc_count":10, "data_stream":map[string]interface {}{"dataset":"generic.otel", "namespace":"default", "type":"metrics"}, "metrics":map[string]interface {}{"histogram.summary":map[string]interface {}{"sum":1, "value_count":10}}, "resource":map[string]interface {}{"dropped_attributes_count":0}, "scope":map[string]interface {}{"dropped_attributes_count":0}}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,3 +1,3 @@
        	            	 (map[string]interface {}) (len=6) {
        	            	- (string) (len=10) "@timestamp": (string) (len=30) "1970-01-01T01:00:00.000000000Z",
        	            	+ (string) (len=10) "@timestamp": (string) (len=30) "1970-01-01T00:00:00.000000000Z",
        	            	  (string) (len=10) "_doc_count": (float64) 10,
        	            	@@ -9,3 +9,3 @@
        	            	  (string) (len=7) "metrics": (map[string]interface {}) (len=1) {
        	            	-  (string) (len=20) "exphistogram.summary": (map[string]interface {}) (len=2) {
        	            	+  (string) (len=17) "histogram.summary": (map[string]interface {}) (len=2) {
        	            	    (string) (len=3) "sum": (float64) 1,
        	Test:       	TestExporterMetrics/otel_mode_aggregate_metric_double_hint
        	Messages:   	item 0 document

=== FAIL: . TestExporterMetrics (0.51s)
panic: runtime error: index out of range [1] with length 1 [recovered]
	panic: runtime error: index out of range [1] with length 1

goroutine 709 [running]:
testing.tRunner.func1.2({0x1417afd00, 0xc0000ba7b0})
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1631 +0x3f7
testing.tRunner.func1()
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1634 +0x6b6
panic({0x1417afd00?, 0xc0000ba7b0?})
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/panic.go:770 +0x132
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.assertItemsEqual(0xc0001a2680, {0xc000205be8, 0x2, 0xc0002247e0?}, {0xc0002248a0, 0x1, 0x1416392a0?}, 0x0)
	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/utils_test.go:55 +0x65e
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.TestExporterMetrics.func13(0xc0001a2680)
	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/elasticsearchexporter/exporter_test.go:1325 +0x1d8b
testing.tRunner(0xc0001a2680, 0x1418ed4a8)
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1689 +0x21f
created by testing.(*T).Run in goroutine [551](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11632160146/job/32394694302#step:8:552)
	C:/hostedtoolcache/windows/go/1.22.8/x64/src/testing/testing.go:1742 +0x826

=== FAIL: . TestConfig (unknown)

@carsonip
Copy link
Contributor

carsonip commented Nov 7, 2024

sorry, I'll take a look

carsonip added a commit to carsonip/opentelemetry-collector-contrib that referenced this issue Nov 7, 2024
@carsonip
[chore][exporter/elasticsearch] Fix flaky recorder assertions
a2f8d9d 
Fixes open-telemetry#35924
@carsonip carsonip mentioned this issue Nov 7, 2024
Merged
pull bot pushed a commit to abaguas/opentelemetry-collector-contrib that referenced this issue Nov 8, 2024
@carsonip @pull
[chore][exporter/elasticsearch] Fix flaky recorder assertions (open-t…
73bb335 
…elemetry#36255)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Incorrect rec.WaitItems call causes test to be flaky. Refactor to avoid
future occurrences.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#35924

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
sbylica-splunk pushed a commit to sbylica-splunk/opentelemetry-collector-contrib that referenced this issue Dec 17, 2024
@carsonip @sbylica-splunk
[chore][exporter/elasticsearch] Fix flaky recorder assertions (open-t…
29ea7ce 
…elemetry#36255)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Incorrect rec.WaitItems call causes test to be flaky. Refactor to avoid
future occurrences.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#35924

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@carsonip carsonip

Labels
exporter/elasticsearch flaky test a test is flaky os:windows
Projects
None yet
Milestone
No milestone
4 participants
@atoulme @pjanotti @carsonip @songy23