Skip to content

Commit

Permalink
7406 trace context support (#7508) (#7520)
Browse files Browse the repository at this point in the history
* 7406 trace context support (#7508)

* Cherry-pick c511b2f6 onto master branch

Initial commit of tracing plugin changes

At this point, the plugin should in theory have all translation implemented. However, there is no error checking present in the plugin. A basic unit test exists for this work, but all it does it ensure that no errors occurred. The unit test also does not yet validate the content of the output object.

* Only set GrpcService if OcagentAddress is empty

This was creating an error in Envoy - if the GrpcService object being sent to Envoy had an empty string, then it would not accept the configuration.

* Final version of opencensus changes

This commit includes a translation fix and simplification of the API. Originally, we were outputting an object of type GrpcService_EnvoyGrpc_ for the grpcAddress option, but the Envoy plugin only supports the GrpcService_GoogleGrpc_ type. So we fix that here. We also simplify the protobuf API to use a oneof for specifying HTTP or GRPC.

I'm committing this now because we have decided to drop using OpenCensus due to the fact that it cannot be reconfigured. A lot of the work on this effort will be discarded, so I am saving it here for posterity.

* Rename all strings in opencensus.proto to opentelemetry

* Rename opencensus.proto to opentelemetry.proto

* Rewrite everything to use opentelemetry instead

So far, I have only tested manually, but it seems to work okay. One odd behaviour is that Envoy only seems to report information to the echo server on shutdown.

Unit tests aren't doing anything serious at the moment and need to be improved. It will also be necessary to write an e2e test for this. But otherwise, things seem to be working much more smoothly than they did with opencensus.

* Replace "Census" with "Telemetry" in function name

* Remove autogenerated opencensus documentation file

* Update unit test

* Add changelog

* Add e2e test for opentelemetry

* Update docs/content/reference/api/github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/opentelemetry.proto.sk.md

Co-authored-by: Art <[email protected]>

* Incorporate some changes from Sam

* Remove race conditions from e2e tests

The major issue here was a pair of variables with near-global scope: collectorApiHit and collectionURLPath. Both of these were triggering go's race condition flags, probably because they were being written to and read from ginkgo's setup and teardown callbacks.

There was also an issue in shutting down the collection server, which is addressed by a comment around where the change was made.

* Some comments and docstring fixes

* Add default statements in tracing plugin type translation

* Changelog improvements

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Art <[email protected]>

* Move changelog to the v1.12.x directory

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Art <[email protected]>
  • Loading branch information
3 people authored Dec 7, 2022
1 parent 7a0a76e commit 8533ccc
Show file tree
Hide file tree
Showing 22 changed files with 1,053 additions and 176 deletions.
6 changes: 6 additions & 0 deletions changelog/v1.12.36/opentelemetry-tracing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
changelog:
- type: NEW_FEATURE
issueLink: https://github.com/solo-io/gloo/issues/7406
resolvesIssue: false
description: >
Update the Tracer plugin to support Envoy's OpenTelemetry filter: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/trace/v3/opentelemetry.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions docs/data/ProtoMap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1547,6 +1547,9 @@ apis:
solo.io.envoy.config.trace.v3.DatadogConfig:
relativepath: reference/api/github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/datadog.proto.sk/#DatadogConfig
package: solo.io.envoy.config.trace.v3
solo.io.envoy.config.trace.v3.OpenTelemetryConfig:
relativepath: reference/api/github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/opentelemetry.proto.sk/#OpenTelemetryConfig
package: solo.io.envoy.config.trace.v3
solo.io.envoy.config.trace.v3.ZipkinConfig:
relativepath: reference/api/github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/zipkin.proto.sk/#ZipkinConfig
package: solo.io.envoy.config.trace.v3
Expand Down
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ require (
github.com/Netflix/go-expect v0.0.0-20180928190340-9d1f4485533b
github.com/avast/retry-go v2.4.3+incompatible
github.com/aws/aws-sdk-go v1.34.9
github.com/bshuster-repo/logrus-logstash-hook v1.0.0 // indirect
github.com/census-instrumentation/opencensus-proto v0.3.0 // indirect
github.com/cratonica/2goarray v0.0.0-20190331194516-514510793eaa
github.com/envoyproxy/go-control-plane v0.10.3
github.com/envoyproxy/protoc-gen-validate v0.6.7
Expand Down Expand Up @@ -113,7 +115,6 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/bugsnag/bugsnag-go v1.5.0 // indirect
github.com/census-instrumentation/opencensus-proto v0.3.0 // indirect
github.com/cespare/xxhash/v2 v2.1.1 // indirect
github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 // indirect
github.com/cncf/xds/go v0.0.0-20220314180256-7f1daf1720fc // indirect
Expand Down
3 changes: 2 additions & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,9 @@ github.com/bluekeyes/hatpear v0.1.1/go.mod h1:2bh+rl4wLhqzzL0hT7Q4SVGXIivrE8oKgH
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
github.com/bmizerany/perks v0.0.0-20141205001514-d9a9656a3a4b/go.mod h1:ac9efd0D1fsDb3EJvhqgXRbFx7bs2wqZ10HQPeU8U/Q=
github.com/bradleyfalzon/ghinstallation v1.1.1/go.mod h1:vyCmHTciHx/uuyN82Zc3rXN3X2KTK8nUTCrTMwAhcug=
github.com/bshuster-repo/logrus-logstash-hook v0.4.1 h1:pgAtgj+A31JBVtEHu2uHuEx0n+2ukqUJnS2vVe5pQNA=
github.com/bshuster-repo/logrus-logstash-hook v0.4.1/go.mod h1:zsTqEiSzDgAa/8GZR7E1qaXrhYNDKBYy5/dWPTIflbk=
github.com/bshuster-repo/logrus-logstash-hook v1.0.0 h1:e+C0SB5R1pu//O4MQ3f9cFuPGoOVeF2fE4Og9otCc70=
github.com/bshuster-repo/logrus-logstash-hook v1.0.0/go.mod h1:zsTqEiSzDgAa/8GZR7E1qaXrhYNDKBYy5/dWPTIflbk=
github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8=
github.com/bugsnag/bugsnag-go v1.4.0/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8=
github.com/bugsnag/bugsnag-go v1.5.0 h1:tP8hiPv1pGGW3LA6LKy5lW6WG+y9J2xWUdPd3WC452k=
Expand Down
36 changes: 36 additions & 0 deletions install/helm/gloo/crds/gateway.solo.io_v1_Gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,18 @@ spec:
type: string
type: object
type: array
openTelemetryConfig:
properties:
clusterName:
type: string
collectorUpstreamRef:
properties:
name:
type: string
namespace:
type: string
type: object
type: object
requestHeadersForTags:
items:
nullable: true
Expand Down Expand Up @@ -1111,6 +1123,18 @@ spec:
type: string
type: object
type: array
openTelemetryConfig:
properties:
clusterName:
type: string
collectorUpstreamRef:
properties:
name:
type: string
namespace:
type: string
type: object
type: object
requestHeadersForTags:
items:
nullable: true
Expand Down Expand Up @@ -1964,6 +1988,18 @@ spec:
type: string
type: object
type: array
openTelemetryConfig:
properties:
clusterName:
type: string
collectorUpstreamRef:
properties:
name:
type: string
namespace:
type: string
type: object
type: object
requestHeadersForTags:
items:
nullable: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -685,6 +685,18 @@ spec:
type: string
type: object
type: array
openTelemetryConfig:
properties:
clusterName:
type: string
collectorUpstreamRef:
properties:
name:
type: string
namespace:
type: string
type: object
type: object
requestHeadersForTags:
items:
nullable: true
Expand Down
27 changes: 21 additions & 6 deletions pkg/utils/api_conversion/trace.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package api_conversion

import (
envoy_config_core_v3 "github.com/envoyproxy/go-control-plane/envoy/config/core/v3"
envoytrace "github.com/envoyproxy/go-control-plane/envoy/config/trace/v3"
envoytrace_gloo "github.com/solo-io/gloo/projects/gloo/pkg/api/external/envoy/config/trace/v3"
envoytracegloo "github.com/solo-io/gloo/projects/gloo/pkg/api/external/envoy/config/trace/v3"
)

// Converts between Envoy and Gloo/solokit versions of envoy protos
Expand All @@ -12,15 +13,15 @@ import (
// we should work to remove that assumption from solokit and delete this code:
// https://github.com/solo-io/gloo/issues/1793

func ToEnvoyDatadogConfiguration(glooDatadogConfig *envoytrace_gloo.DatadogConfig, clusterName string) (*envoytrace.DatadogConfig, error) {
func ToEnvoyDatadogConfiguration(glooDatadogConfig *envoytracegloo.DatadogConfig, clusterName string) (*envoytrace.DatadogConfig, error) {
envoyDatadogConfig := &envoytrace.DatadogConfig{
CollectorCluster: clusterName,
ServiceName: glooDatadogConfig.GetServiceName().GetValue(),
}
return envoyDatadogConfig, nil
}

func ToEnvoyZipkinConfiguration(glooZipkinConfig *envoytrace_gloo.ZipkinConfig, clusterName string) (*envoytrace.ZipkinConfig, error) {
func ToEnvoyZipkinConfiguration(glooZipkinConfig *envoytracegloo.ZipkinConfig, clusterName string) (*envoytrace.ZipkinConfig, error) {
envoyZipkinConfig := &envoytrace.ZipkinConfig{
CollectorCluster: clusterName,
CollectorEndpoint: glooZipkinConfig.GetCollectorEndpoint(),
Expand All @@ -31,11 +32,25 @@ func ToEnvoyZipkinConfiguration(glooZipkinConfig *envoytrace_gloo.ZipkinConfig,
return envoyZipkinConfig, nil
}

func ToEnvoyZipkinCollectorEndpointVersion(version envoytrace_gloo.ZipkinConfig_CollectorEndpointVersion) envoytrace.ZipkinConfig_CollectorEndpointVersion {
func ToEnvoyOpenTelemetryonfiguration(glooOpenTelemetryConfig *envoytracegloo.OpenTelemetryConfig, clusterName string) (*envoytrace.OpenTelemetryConfig, error) {
envoyOpenTelemetryConfig := &envoytrace.OpenTelemetryConfig{
GrpcService: &envoy_config_core_v3.GrpcService{
TargetSpecifier: &envoy_config_core_v3.GrpcService_EnvoyGrpc_{
EnvoyGrpc: &envoy_config_core_v3.GrpcService_EnvoyGrpc{
ClusterName: clusterName,
},
},
},
}

return envoyOpenTelemetryConfig, nil

}
func ToEnvoyZipkinCollectorEndpointVersion(version envoytracegloo.ZipkinConfig_CollectorEndpointVersion) envoytrace.ZipkinConfig_CollectorEndpointVersion {
switch str := version.String(); str {
case envoytrace_gloo.ZipkinConfig_CollectorEndpointVersion_name[int32(envoytrace_gloo.ZipkinConfig_HTTP_JSON)]:
case envoytracegloo.ZipkinConfig_CollectorEndpointVersion_name[int32(envoytracegloo.ZipkinConfig_HTTP_JSON)]:
return envoytrace.ZipkinConfig_HTTP_JSON
case envoytrace_gloo.ZipkinConfig_CollectorEndpointVersion_name[int32(envoytrace_gloo.ZipkinConfig_HTTP_PROTO)]:
case envoytracegloo.ZipkinConfig_CollectorEndpointVersion_name[int32(envoytracegloo.ZipkinConfig_HTTP_PROTO)]:
return envoytrace.ZipkinConfig_HTTP_PROTO
}
return envoytrace.ZipkinConfig_HTTP_JSON
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
syntax = "proto3";

package solo.io.envoy.config.trace.v3;

import "udpa/annotations/migrate.proto";
import "udpa/annotations/status.proto";
import "udpa/annotations/versioning.proto";
import "validate/validate.proto";

import "github.com/solo-io/solo-kit/api/v1/ref.proto";

option java_package = "io.envoyproxy.solo.io.envoy.config.trace.v3";
option java_outer_classname = "OpentelemetryProto";
option java_multiple_files = true;
option (solo.io.udpa.annotations.file_migrate).move_to_package = "envoy.extensions.tracers.opentelemetry.v4alpha";
option (solo.io.udpa.annotations.file_status).package_version_status = ACTIVE;

// [#protodoc-title: OpenTelemetry tracer]

// Configuration for the OpenTelemetry tracer.
// [#extension: envoy.tracers.opentelemetry]
message OpenTelemetryConfig {
option (solo.io.udpa.annotations.versioning).previous_message_type =
"envoy.config.trace.v2.OpenTelemetryConfig";

// The cluster to use for submitting traces to the OpenTelemetry agent.
oneof collector_cluster {
// The upstream to use for submitting traces to the OpenTelemetry agent.
core.solo.io.ResourceRef collector_upstream_ref = 1;
// The name of the Envoy cluster to use for submitting traces to the
// OpenTelemetry agent
string cluster_name = 2;
}
}

option go_package = "github.com/solo-io/gloo/projects/gloo/pkg/api/external/envoy/config/trace/v3";
import "extproto/ext.proto";
option (extproto.hash_all) = true;
option (extproto.clone_all) = true;
option (extproto.equal_all) = true;

4 changes: 3 additions & 1 deletion projects/gloo/api/v1/options/tracing/tracing.proto
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ option go_package = "github.com/solo-io/gloo/projects/gloo/pkg/api/v1/options/tr

import "github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/zipkin.proto";
import "github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/datadog.proto";
import "github.com/solo-io/gloo/projects/gloo/api/external/envoy/config/trace/v3/opentelemetry.proto";


import "google/protobuf/wrappers.proto";
Expand Down Expand Up @@ -34,6 +35,7 @@ message ListenerTracingSettings {
oneof provider_config {
.solo.io.envoy.config.trace.v3.ZipkinConfig zipkin_config = 4;
.solo.io.envoy.config.trace.v3.DatadogConfig datadog_config = 5;
.solo.io.envoy.config.trace.v3.OpenTelemetryConfig open_telemetry_config = 8;
}
// Optional. If specified, Envoy will include the environment variables with the given tag as tracing tags.
repeated TracingTagEnvironmentVariable environment_variables_for_tags = 6;
Expand Down Expand Up @@ -92,4 +94,4 @@ message TracingTagLiteral {
google.protobuf.StringValue tag = 1;
// Static literal value to populate the tag value.
google.protobuf.StringValue value = 2;
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 8533ccc

Please sign in to comment.