[exporter][queuebatching] Implemented "DisabledQueue" which is used when batch is enabled but queue is disabled #12118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sfc-gh-sili
force-pushed
the
sili-batching
branch
2 times, most recently
from
ff4a975 to
c52e885
Compare
Codecov ReportAttention: Patch coverage is
❌ Your patch check has failed because the patch coverage (91.22%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #12118 +/- ##
==========================================
- Coverage 91.24% 91.23% -0.01%
==========================================
Files 465 466 +1
Lines 25689 25738 +49
==========================================
+ Hits 23439 23483 +44
- Misses 1837 1841 +4
- Partials 413 414 +1 ☔ View full report in Codecov by Sentry. |
sfc-gh-sili
force-pushed
the
sili-batching
branch
2 times, most recently
from
20a4b36 to
1b1ad11
Compare
sfc-gh-sili
requested review from
bogdandrutu,
dmitryax and
a team
as code owners
sfc-gh-sili
changed the title
sfc-gh-sili
force-pushed
the
sili-batching
branch
2 times, most recently
from
1b1ad11 to
3b269bc
Compare
|
@sfc-gh-sili I read this PR trying to understand why when the queue sender is disabled, we have to create a "real" queue in some sense. For example, this PR you allocate a |
|
@jmacd Hi Joshua, thanks for the feedback! Taking a step back and thinking through the expected behavior, I think of queue as an abstraction that provides asynchronous-ity - when
You see, we now have two different components for batching - I agree it's time to have some component-level documentation that describes the expected behavior - will work on it. |
…12159) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description <!-- Issue number if applicable --> - Deprecates `component.TelemetrySettings.MetricsLevel` - Sets the value passed to components to always be `configtelemetry.LevelDetailed` #### Link to tracking issue Updates open-telemetry#11061
This PR does a couple of things that I couldn't quite split up so I put together a PR w/ individual commits to help reviewers get through it. This PR does the following: 1. update `go.opentelemetry.io/contrib/config` package to latest. this brings in breaking changes. in order to prevent those breaking changes from impacting end users, i've also added a layer of config unmarshaling 2. updates the collector to instantiate the meter provider (and exporters) via the config package. this allows us to remove all the code in `otelinit`. the reason for including this change was that unmarshaling the config was causing circular dependencies i didn't want to address by moving code that could be deleted around. Replacement for open-telemetry#11458. Fixes open-telemetry#12021 --------- Signed-off-by: Alex Boten <[email protected]>
…emetry#12181) <!--Describe the documentation added.--> #### Documentation This is a documentation-only change to fix some typos in the Pipeline Component Telemetry RFC doc.
…12179) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Completion of open-telemetry#12167 , cc @bogdandrutu Updated one test to verify the change, will update the rest ones after this PR merges. <!-- Issue number if applicable --> #### Link to tracking issue n/a <!--Describe what testing was performed and which tests were added.--> #### Testing Added <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
…open-telemetry#12184) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description A follow-up PR of open-telemetry#12179 , cc @bogdandrutu Note: This PR also removes those metric assertions which is actually testing metrics that doesn't belong to the package. They might be added due to the limitation of the AssertMetrics function. e.g. changes in `processor/memorylimitprocessor/memorylimiter_test.go` <!-- Issue number if applicable --> #### Link to tracking issue Relevant to open-telemetry#12179 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation n/a <!--Please delete paragraphs that you did not use before submitting.-->
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Remove deprecated CheckScraperMetrics functions <!-- Issue number if applicable --> #### Link to tracking issue Relevant to open-telemetry#12105 <!--Describe what testing was performed and which tests were added.--> #### Testing n/a <!--Describe the documentation added.--> #### Documentation n/a <!--Please delete paragraphs that you did not use before submitting.--> --------- Co-authored-by: Bogdan Drutu <[email protected]>
…lemetry#12111) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Pass the missing async error channel into telemetry.Settings <!-- Issue number if applicable --> #### Link to tracking issue Fixes open-telemetry#11417 <!--Describe what testing was performed and which tests were added.--> #### Testing With the same setup as in open-telemetry#11417, building new otelcol with the changes in this PR, and running 2 instances with the same config using the same metric port, we would see proper crash error messages: ``` # config used: receivers: nop: exporters: nop: service: pipelines: logs: receivers: - nop exporters: - nop telemetry: metrics: readers: - pull: exporter: prometheus: host: localhost port: 8889 ``` ``` # first instance log: ./otelcol-custom --config otel-config.yaml 2025-01-16T17:36:34.638-0800 info [email protected]/service.go:165 Setting up own telemetry... 2025-01-16T17:36:34.638-0800 info telemetry/metrics.go:70 Serving metrics {"address": "localhost:8889", "metrics level": "Normal"} 2025-01-16T17:36:34.639-0800 info [email protected]/service.go:231 Starting otelcol-custom... {"Version": "", "NumCPU": 16} 2025-01-16T17:36:34.639-0800 info extensions/extensions.go:39 Starting extensions... 2025-01-16T17:36:34.639-0800 info [email protected]/service.go:254 Everything is ready. Begin running and processing data. ``` ``` # second instance's log (using same config) ./otelcol-custom --config otel-config.yaml 2025-01-16T17:36:37.270-0800 info [email protected]/service.go:165 Setting up own telemetry... 2025-01-16T17:36:37.270-0800 info telemetry/metrics.go:70 Serving metrics {"address": "localhost:8889", "metrics level": "Normal"} 2025-01-16T17:36:37.271-0800 info [email protected]/service.go:231 Starting otelcol-custom... {"Version": "", "NumCPU": 16} 2025-01-16T17:36:37.271-0800 info extensions/extensions.go:39 Starting extensions... 2025-01-16T17:36:37.271-0800 info [email protected]/service.go:254 Everything is ready. Begin running and processing data. 2025-01-16T17:36:37.273-0800 error [email protected]/collector.go:325 Asynchronous error received, terminating process {"error": "listen tcp 127.0.0.1:8889: bind: address already in use"} go.opentelemetry.io/collector/otelcol.(*Collector).Run go.opentelemetry.io/collector/[email protected]/collector.go:325 go.opentelemetry.io/collector/otelcol.NewCommand.func1 go.opentelemetry.io/collector/[email protected]/command.go:36 github.com/spf13/cobra.(*Command).execute github.com/spf13/[email protected]/command.go:985 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/[email protected]/command.go:1117 github.com/spf13/cobra.(*Command).Execute github.com/spf13/[email protected]/command.go:1041 main.runInteractive go.opentelemetry.io/collector/cmd/builder/main.go:49 main.run go.opentelemetry.io/collector/cmd/builder/main_others.go:10 main.main go.opentelemetry.io/collector/cmd/builder/main.go:42 runtime.main runtime/proc.go:272 2025-01-16T17:36:37.273-0800 info [email protected]/service.go:296 Starting shutdown... 2025-01-16T17:36:37.274-0800 info extensions/extensions.go:66 Stopping extensions... 2025-01-16T17:36:37.274-0800 info [email protected]/service.go:310 Shutdown complete. ``` <!--Please delete paragraphs that you did not use before submitting.--> --------- Co-authored-by: Antoine Toulme <[email protected]>
Co-authored-by: Pablo Baeyens <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Fix several issues on optional metrics: 1. Add missing stability 2. Fix description and function in generated file <!-- Issue number if applicable --> #### Link to tracking issue n/a <!--Describe what testing was performed and which tests were added.--> #### Testing n/a <!--Describe the documentation added.--> #### Documentation n/a <!--Please delete paragraphs that you did not use before submitting.-->
…-telemetry#12186) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckExporterLogs` functions, also verified `CheckExporterLogs` function is not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
…-telemetry#12187) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckReceiverLogs` functions, also verified `CheckReceiverLogs` function is not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
…2193) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google.golang.org/protobuf](https://redirect.github.com/protocolbuffers/protobuf-go) | `v1.36.3` -> `v1.36.4` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>protocolbuffers/protobuf-go (google.golang.org/protobuf)</summary> ### [`v1.36.4`](https://redirect.github.com/protocolbuffers/protobuf-go/releases/tag/v1.36.4) [Compare Source](https://redirect.github.com/protocolbuffers/protobuf-go/compare/v1.36.3...v1.36.4) **Full Changelog**: protocolbuffers/protobuf-go@v1.36.3...v1.36.4 Bug fixes: [CL/642975](https://go-review.googlesource.com/c/protobuf/+/642975): reflect/protodesc: fix panic when working with dynamicpb Maintenance: [CL/643276](https://go-review.googlesource.com/c/protobuf/+/643276): internal_gengo: avoid allocations in rawDescGZIP() accessors [CL/642857](https://go-review.googlesource.com/c/protobuf/+/642857): internal_gengo: switch back from string literal to hex byte slice [CL/642055](https://go-review.googlesource.com/c/protobuf/+/642055): internal_gengo: use unsafe.StringData() to avoid a descriptor copy [CL/638135](https://go-review.googlesource.com/c/protobuf/+/638135): internal_gengo: store raw descriptor in .rodata section </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMjUuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]>
…telemetry#12194) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [go.opentelemetry.io/build-tools/checkfile](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools) | `v0.16.0` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [go.opentelemetry.io/build-tools/chloggen](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools) | `v0.16.0` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [go.opentelemetry.io/build-tools/crosslink](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools) | `v0.16.0` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [go.opentelemetry.io/build-tools/multimod](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools) | `v0.16.0` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [go.opentelemetry.io/build-tools/semconvgen](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools) | `v0.16.0` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-go-build-tools (go.opentelemetry.io/build-tools/checkfile)</summary> ### [`v0.17.0`](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools/blob/HEAD/CHANGELOG.md#v0170) [Compare Source](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools/compare/v0.16.0...v0.17.0) ##### 🚀 New components 🚀 - `githubgen`: Moved githubgen tool here from open-telemetry/opentelemetry-collector-contrib ([#&open-telemetry#8203;639](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools/issues/639)) ##### 💡 Enhancements 💡 - `crosslink`: Added `--skip` flag to `crosslink tidylist` subcommand ([#&open-telemetry#8203;662](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools/issues/662)) - `githubgen`: Enhanced githubgen tool with more options to better fit arbitrary repos, added unit tests ([#&open-telemetry#8203;655](https://redirect.github.com/open-telemetry/opentelemetry-go-build-tools/issues/655)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMjUuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]>
…open-telemetry#12198) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckExporterEnqueue*` functions, also verified `CheckExporterEnqueue*` functions are not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
Vulnerability #1: GO-2025-3373 Usage of IPv6 zone IDs can bypass URI name constraints in crypto/x50[9](https://github.com/open-telemetry/opentelemetry-collector/actions/runs/13003405341/job/36266082673?pr=12193#step:6:10) More info: https://pkg.go.dev/vuln/GO-2025-3373 Standard library Found in: crypto/[email protected].[10](https://github.com/open-telemetry/opentelemetry-collector/actions/runs/13003405341/job/36266082673?pr=12193#step:6:11) Fixed in: crypto/[email protected].[11](https://github.com/open-telemetry/opentelemetry-collector/actions/runs/13003405341/job/36266082673?pr=12193#step:6:12) Signed-off-by: Bogdan Drutu <[email protected]>
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [codecov/codecov-action](https://redirect.github.com/codecov/codecov-action) | action | minor | `5.1.2` -> `5.3.1` | | [github/codeql-action](https://redirect.github.com/github/codeql-action) | action | patch | `v3.28.2` -> `v3.28.6` | # Warnings (1) Please correct - or verify that you can safely ignore - these warnings before you merge this PR. - `codecov/codecov-action`: Could not determine new digest for update (github-tags package codecov/codecov-action) --- --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>codecov/codecov-action (codecov/codecov-action)</summary> ### [`v5.3.1`](https://redirect.github.com/codecov/codecov-action/blob/HEAD/CHANGELOG.md#v531) [Compare Source](https://redirect.github.com/codecov/codecov-action/compare/v5.3.0...v5.3.1) ##### What's Changed **Full Changelog**: https://github.com/codecov/codecov-action/compare/v5.3.0..v5.3.1 ### [`v5.3.0`](https://redirect.github.com/codecov/codecov-action/blob/HEAD/CHANGELOG.md#v530) [Compare Source](https://redirect.github.com/codecov/codecov-action/compare/v5.2.0...v5.3.0) ##### What's Changed **Full Changelog**: https://github.com/codecov/codecov-action/compare/v5.2.0..v5.3.0 ### [`v5.2.0`](https://redirect.github.com/codecov/codecov-action/blob/HEAD/CHANGELOG.md#v520) [Compare Source](https://redirect.github.com/codecov/codecov-action/compare/v5.1.2...v5.2.0) ##### What's Changed - Fix typo in README by [@&open-telemetry#8203;tserg](https://redirect.github.com/tserg) in [https://github.com/codecov/codecov-action/pull/1747](https://redirect.github.com/codecov/codecov-action/pull/1747) - Th/add commands by [@&open-telemetry#8203;thomasrockhu-codecov](https://redirect.github.com/thomasrockhu-codecov) in [https://github.com/codecov/codecov-action/pull/1745](https://redirect.github.com/codecov/codecov-action/pull/1745) - use correct audience when requesting oidc token by [@&open-telemetry#8203;juho9000](https://redirect.github.com/juho9000) in [https://github.com/codecov/codecov-action/pull/1744](https://redirect.github.com/codecov/codecov-action/pull/1744) - build(deps): bump github/codeql-action from 3.27.9 to 3.28.1 by [@&open-telemetry#8203;app/dependabot](https://redirect.github.com/app/dependabot) in [https://github.com/codecov/codecov-action/pull/1742](https://redirect.github.com/codecov/codecov-action/pull/1742) - build(deps): bump actions/upload-artifact from 4.4.3 to 4.6.0 by [@&open-telemetry#8203;app/dependabot](https://redirect.github.com/app/dependabot) in [https://github.com/codecov/codecov-action/pull/1743](https://redirect.github.com/codecov/codecov-action/pull/1743) - chore(deps): bump wrapper to 0.0.32 by [@&open-telemetry#8203;thomasrockhu-codecov](https://redirect.github.com/thomasrockhu-codecov) in [https://github.com/codecov/codecov-action/pull/1740](https://redirect.github.com/codecov/codecov-action/pull/1740) - feat: add disable-telem feature by [@&open-telemetry#8203;thomasrockhu-codecov](https://redirect.github.com/thomasrockhu-codecov) in [https://github.com/codecov/codecov-action/pull/1739](https://redirect.github.com/codecov/codecov-action/pull/1739) - fix: remove erroneous linebreak in readme by [@&open-telemetry#8203;Vampire](https://redirect.github.com/Vampire) in [https://github.com/codecov/codecov-action/pull/1734](https://redirect.github.com/codecov/codecov-action/pull/1734) **Full Changelog**: https://github.com/codecov/codecov-action/compare/v5.1.2..v5.2.0 </details> <details> <summary>github/codeql-action (github/codeql-action)</summary> ### [`v3.28.6`](https://redirect.github.com/github/codeql-action/compare/v3.28.5...v3.28.6) [Compare Source](https://redirect.github.com/github/codeql-action/compare/v3.28.5...v3.28.6) ### [`v3.28.5`](https://redirect.github.com/github/codeql-action/releases/tag/v3.28.5) [Compare Source](https://redirect.github.com/github/codeql-action/compare/v3.28.4...v3.28.5) ##### CodeQL Action Changelog See the [releases page](https://redirect.github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs. ##### 3.28.5 - 24 Jan 2025 - Update default CodeQL bundle version to 2.20.3. [#&open-telemetry#8203;2717](https://redirect.github.com/github/codeql-action/pull/2717) See the full [CHANGELOG.md](https://redirect.github.com/github/codeql-action/blob/v3.28.5/CHANGELOG.md) for more information. ### [`v3.28.4`](https://redirect.github.com/github/codeql-action/releases/tag/v3.28.4) [Compare Source](https://redirect.github.com/github/codeql-action/compare/v3.28.3...v3.28.4) ##### CodeQL Action Changelog See the [releases page](https://redirect.github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs. ##### 3.28.4 - 23 Jan 2025 No user facing changes. See the full [CHANGELOG.md](https://redirect.github.com/github/codeql-action/blob/v3.28.4/CHANGELOG.md) for more information. ### [`v3.28.3`](https://redirect.github.com/github/codeql-action/releases/tag/v3.28.3) [Compare Source](https://redirect.github.com/github/codeql-action/compare/v3.28.2...v3.28.3) ##### CodeQL Action Changelog See the [releases page](https://redirect.github.com/github/codeql-action/releases) for the relevant changes to the CodeQL CLI and language packs. ##### 3.28.3 - 22 Jan 2025 - Update default CodeQL bundle version to 2.20.2. [#&open-telemetry#8203;2707](https://redirect.github.com/github/codeql-action/pull/2707) - Fix an issue downloading the CodeQL Bundle from a GitHub Enterprise Server instance which occurred when the CodeQL Bundle had been synced to the instance using the [CodeQL Action sync tool](https://redirect.github.com/github/codeql-action-sync-tool) and the Actions runner did not have Zstandard installed. [#&open-telemetry#8203;2710](https://redirect.github.com/github/codeql-action/pull/2710) - Uploading debug artifacts for CodeQL analysis is temporarily disabled. [#&open-telemetry#8203;2712](https://redirect.github.com/github/codeql-action/pull/2712) See the full [CHANGELOG.md](https://redirect.github.com/github/codeql-action/blob/v3.28.3/CHANGELOG.md) for more information. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMjUuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Bogdan Drutu <[email protected]>
This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google.golang.org/grpc](https://redirect.github.com/grpc/grpc-go) | `v1.69.4` -> `v1.70.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>grpc/grpc-go (google.golang.org/grpc)</summary> ### [`v1.70.0`](https://redirect.github.com/grpc/grpc-go/releases/tag/v1.70.0): Release 1.70.0 [Compare Source](https://redirect.github.com/grpc/grpc-go/compare/v1.69.4...v1.70.0) ### Behavior Changes - client: reject service configs containing an invalid retryPolicy in accordance with gRFCs [A21](https://redirect.github.com/grpc/proposal/blob/master/A21-service-config-error-handling.md) and [A6](https://redirect.github.com/grpc/proposal/blob/master/A6-client-retries.md). ([#&open-telemetry#8203;7905](https://redirect.github.com/grpc/grpc-go/issues/7905)) - Note that this is a potential breaking change for some users using an invalid configuration, but continuing to allow this behavior would violate our cross-language compatibility requirements. ### New Features - xdsclient: fallback to a secondary management server (if specified in the bootstrap configuration) when the primary is down is enabled by default. Can be disabled by setting the environment variable `GRPC_EXPERIMENTAL_XDS_FALLBACK` to `false`. ([#&open-telemetry#8203;7949](https://redirect.github.com/grpc/grpc-go/issues/7949)) - experimental/credentials: experimental transport credentials are added which don't enforce ALPN. ([#&open-telemetry#8203;7980](https://redirect.github.com/grpc/grpc-go/issues/7980)) - These credentials will be removed in an upcoming grpc-go release. Users must not rely on these credentials directly. Instead, they should either vendor a specific version of gRPC or copy the relevant credentials into their own codebase if absolutely necessary. ### Bug Fixes - xds: fix a possible deadlock that happens when both the client application and the xDS management server (responsible for configuring the client) are using the xds:/// scheme in their target URIs. ([#&open-telemetry#8203;8011](https://redirect.github.com/grpc/grpc-go/issues/8011)) ### Performance - server: for unary requests, free raw request message data as soon as parsing is finished instead of waiting until the method handler returns. ([#&open-telemetry#8203;7998](https://redirect.github.com/grpc/grpc-go/issues/7998)) - Special Thanks: [@&open-telemetry#8203;lqs](https://redirect.github.com/lqs) ### Documentation - examples/features/gracefulstop: add example to demonstrate server graceful stop. ([#&open-telemetry#8203;7865](https://redirect.github.com/grpc/grpc-go/issues/7865)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMjUuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]>
…pen-telemetry#11956) ### Context The [Pipeline Component Telemetry RFC](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/rfcs/component-universal-telemetry.md) was recently accepted (open-telemetry#11406). The document states the following regarding error monitoring: > For both [consumed and produced] metrics, an `outcome` attribute with possible values `success` and `failure` should be automatically recorded, corresponding to whether or not the corresponding function call returned an error. Specifically, consumed measurements will be recorded with `outcome` as `failure` when a call from the previous component the `ConsumeX` function returns an error, and `success` otherwise. Likewise, produced measurements will be recorded with `outcome` as `failure` when a call to the next consumer's `ConsumeX` function returns an error, and `success` otherwise. [Observability requirements for stable pipeline components](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#observability-requirements) were also recently merged (open-telemetry#11772). The document states the following regarding error monitoring: > The goal is to be able to easily pinpoint the source of data loss in the Collector pipeline, so this should either: > - only include errors internal to the component, or; > - allow distinguishing said errors from ones originating in an external service, or propagated from downstream Collector components. Because errors are typically propagated across `ConsumeX` calls in a pipeline (except for components with an internal queue like `processor/batch`), the error observability mechanism proposed by the RFC implies that Pipeline Telemetry will record failures for every component interface upstream of the component that actually emitted the error, which does not match the goals set out in the observability requirements, and makes it much harder to tell which component errors are coming from from the emitted telemetry. ### Description This PR amends the Pipeline Component Telemetry RFC with the following: - restrict the `outcome=failure` value to cases where the error comes from the very next component (the component on which `ConsumeX` was called); - add a third possible value for the `outcome` attribute: `rejected`, for cases where an error observed at an interface comes from further downstream (the component did not "fail", but its output was "rejected"); - propose a mechanism to determine which of the two values should be used. The current proposal for the mechanism is for the pipeline instrumentation layer to wrap errors in an unexported `downstream` struct, which upstream layers could check for with `errors.As` to check whether the error has already been "attributed" to a component. This is the same mechanism currently used for tracking permanent vs. retryable errors. Please check the diff for details. ### Possible alternatives There are a few alternatives to this amendment, which were discussed as part of the observability requirements PR: - loosen the observability requirements for stable components to not require distinguishing internal errors from downstream ones → makes it harder to identify the source of an error; - modify the way we use the `Consumer` API to no longer propagate errors upstream → prevents proper propagation of backpressure through the pipeline (although this is likely already a problem with the `batch` prcessor); - let component authors make their own custom telemetry to solve the problem → higher barrier to entry, especially for people wanting to opensource existing components. --------- Co-authored-by: Pablo Baeyens <[email protected]>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description I tried to improve the docs on watchers based on the issue below <!-- Issue number if applicable --> #### Link to tracking issue Fixes open-telemetry#12115 <!--Describe the documentation added.--> #### Documentation Clarified some code comments that were incorrect (there's no Retrieved.Get method) and included a new code sample for a Provider that uses the watcher func <!--Please delete paragraphs that you did not use before submitting.-->
…open-telemetry#12190) #### Description Add wording that allows us to update struct field tags if changes to the tags doesn't change the values of fields in the struct during serialization/deserialization. This will allow us to add `omitempty` to fields on structs in stable packages. Adding this tag will cause fields which have zero values to not be written when marshaling into YAML, but there is no functional difference since missing fields during unmarshaling simply leaves a struct field with a zero value. Related to open-telemetry#12191.
…pen-telemetry#12200) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckExporterMetrics` functions, also verified `CheckExporterMetrics` function is not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
…etry#12202) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description adds a caveat to ocb cmd/builder documentation about `go.mod` version. <!-- Issue number if applicable --> #### Link to tracking issue Fixes open-telemetry#11844 Replaces open-telemetry#11845 <!--Describe what testing was performed and which tests were added.--> #### Testing none <!--Describe the documentation added.--> #### Documentation README.md <!--Please delete paragraphs that you did not use before submitting.-->
…pen-telemetry#12199) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckExporterTraces` functions, also verified `CheckExporterTraces` function is not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.-->
…metry#12208) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description <!-- Issue number if applicable --> Changes `extension.Extension` to a proper interface that embeds `component.Component` instead of being an alias. #### Link to tracking issue Fixes the second point mentioned in open-telemetry#11443 (comment)
…telemetry#12204) This PR also solves one more issues with the new interface, which is the ability to record multiple observations for a single async metric using one callback. Signed-off-by: Bogdan Drutu <[email protected]>
…emetry#12174) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Allow users to mention their preferred curve types for ECDHE handshake. Add a new config option for this. This will provide users with more control over specific settings during the handshake process. <!--Describe what testing was performed and which tests were added.--> #### Testing Added <!--Describe the documentation added.--> #### Documentation Updated readme<!--Please delete paragraphs that you did not use before submitting.--> Note: Please let me know if I need to open an issue for this.
…#12215) Signed-off-by: Bogdan Drutu <[email protected]>
…try (open-telemetry#12216) This is the last functionality needed to replace `metadatatest.Telemetry` with `componenttest.Telemetry`. The `metadatatest.Telemetry` will be deprecated next release to give time to change contrib since it is a large change. Signed-off-by: Bogdan Drutu <[email protected]>
…ons (open-telemetry#12210) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Deprecate `CheckReceiverMetricGauge` functions, also verified `CheckReceiverMetricGauge` function is not being used in the contrib repo as well. <!-- Issue number if applicable --> #### Link to tracking issue Part of open-telemetry#12185 <!--Describe what testing was performed and which tests were added.--> #### Testing Updated <!--Describe the documentation added.--> #### Documentation Added <!--Please delete paragraphs that you did not use before submitting.--> --------- Co-authored-by: Bogdan Drutu <[email protected]>
…#12100) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Currently these are just printed, which hides validation issues with `telemetry.Config`. If we don't want to return these errors, we should document that and print them at a warning log level.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Profiles attributes are stored in a single `AttributeTable` attribute, and then referenced in each substruct as `AttributeIndices`. This means to read them as a `pcommon.Map`, a bit of pre-processing is required. This adds an helper method so each component manipulating profiles doesn't have to reimplement it. Benchmark: ``` go test -bench=BenchmarkFromAttributeIndices ./... goos: darwin goarch: arm64 pkg: go.opentelemetry.io/collector/pdata/pprofile cpu: Apple M1 Max BenchmarkFromAttributeIndices-10 9789714 123.2 ns/op 140 B/op 4 allocs/op PASS ok go.opentelemetry.io/collector/pdata/pprofile 1.952s PASS ok go.opentelemetry.io/collector/pdata/pprofile/pprofileotlp 0.407s ```
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description <!-- Issue number if applicable --> Changes the underlying type of `component.Kind` to be closer to [`pipeline.Signal`](https://pkg.go.dev/go.opentelemetry.io/collector/pipeline#Signal). Right now the type is defined within `component`, but it could be moved to an internal module so that we could have a different module exposing other value on this enum. This is already doable today, the only thing this PR gives us is 1. slightly more flexibility on things like making the concept of kind more complex (e.g. adding an adjective to a kind). 2. restricting external consumers from implementing their own component kinds without our explicit approval (with some kind of API we design) I am not convinced this is _necessary_ to do, but we may as well do it. This is technically a breaking change since `component.Kind(42)` was a valid expression and it no longer is. I think this is extremely rare, so I suggest if we go ahead we do so in one go. #### Link to tracking issue Fixes open-telemetry#11443
…2108) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Follow up to open-telemetry#12102. This prints the full path to where the error occurred in a config object using available reflection metadata. Currently some paths will be duplicated until we shift otelcol to call `component.ValidateConfig` on `otelcol.Config` and remove manual calls to `Validate`, but no information will be missing. If this is a concern we can do both steps at once. Example error messages: ``` collector server run finished with error: invalid configuration: receivers::otlp: must specify at least one protocol when using the OTLP receiver service::telemetry: collector telemetry metrics reader should exist when metric level is not None, level is Normal ``` I'll clean this PR up once open-telemetry#12102 is merged, but you can see the latest commit for the changes relevant to only this PR. --------- Co-authored-by: Evan Bradley <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Follow-up to open-telemetry#12108. This calls `component.ValidateConfig` on `otelcol.Config`, which causes `Validate` to be recursively called on all eligible structs in the config. This has the following benefits: 1. `Validate` will now automatically be called instead of config structs needing to call it on member structs. 2. Paths to the exact key that caused a validation error in the config are now consistently printed. 3. If there are multiple errors in the config, they are now all returned to the user instead of just the first error encountered.
sfc-gh-sili
force-pushed
the
sili-batching
branch
from
987e7e8 to
437aee0
Compare
sfc-gh-sili
requested review from
mx-psi,
dmathieu,
evan-bradley and
andrzej-stencel
as code owners
|
Thank you @sfc-gh-sili. I see also #12245 addresses this topic, I'll continue paying attention. I've closed both of my RFCs #11947 #11948 (they're out of date) and will re-open after the dust settles. |
|
Closing this PR as #12242 resolves the issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR implements a component called "DisabledQueue".
DisabledQueue is used in place of PersistentStorageQueue or BoundedMemoryQueue when batching is enabled but queueing is disabled. Having this component enables us to have a unified implementation for the case when queueing is enabled or not.
Link to tracking issue
#8122
#10368
Testing
Documentation