Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Middleware to containerinsightrecieve/resourcedetection #265

Merged
merged 7 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/otelcontribcol/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,7 @@ require (
github.com/alecthomas/participle/v2 v2.1.1 // indirect
github.com/alecthomas/units v0.0.0-20231202071711-9a357b53e9c9 // indirect
github.com/aliyun/aliyun-log-go-sdk v0.1.76 // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20240419190856-2f880467f335 // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20241204155332-be45e31638de // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/override/aws v0.0.0-20240415183253-230331014d2c // indirect
github.com/andybalholm/brotli v1.1.0 // indirect
github.com/apache/arrow/go/v15 v15.0.0 // indirect
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ require (
github.com/alecthomas/participle/v2 v2.1.1 // indirect
github.com/alecthomas/units v0.0.0-20231202071711-9a357b53e9c9 // indirect
github.com/aliyun/aliyun-log-go-sdk v0.1.76 // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20240419190856-2f880467f335 // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20241204155332-be45e31638de // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/override/aws v0.0.0-20240415183253-230331014d2c // indirect
github.com/andybalholm/brotli v1.1.0 // indirect
github.com/apache/arrow/go/v15 v15.0.0 // indirect
Expand Down
7 changes: 7 additions & 0 deletions internal/metadataproviders/aws/ec2/metadata.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ package ec2 // import "github.com/open-telemetry/opentelemetry-collector-contrib
import (
"context"

"github.com/aws/aws-sdk-go/aws/request"
Paramadon marked this conversation as resolved.
Show resolved Hide resolved

override "github.com/amazon-contributing/opentelemetry-collector-contrib/override/aws"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
Expand All @@ -14,6 +16,7 @@ import (

type Provider interface {
Get(ctx context.Context) (ec2metadata.EC2InstanceIdentityDocument, error)
GetHandlers(ctx context.Context) *request.Handlers
Paramadon marked this conversation as resolved.
Show resolved Hide resolved
Hostname(ctx context.Context) (string, error)
InstanceID(ctx context.Context) (string, error)
}
Expand Down Expand Up @@ -58,3 +61,7 @@ func (c *metadataClient) Get(_ context.Context) (ec2metadata.EC2InstanceIdentity
}
return c.metadataFallbackEnable.GetInstanceIdentityDocument()
}

func (c *metadataClient) GetHandlers(_ context.Context) *request.Handlers {
return &c.metadata.Handlers
}
5 changes: 5 additions & 0 deletions processor/resourcedetectionprocessor/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
package resourcedetectionprocessor // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor"

import (
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"

"github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor/internal"
Expand Down Expand Up @@ -41,6 +42,10 @@ type Config struct {
// If a supplied attribute is not a valid attribute of a supplied detector it will be ignored.
// Deprecated: Please use detector's resource_attributes config instead
Attributes []string `mapstructure:"attributes"`

// MiddlewareID is an ID for an extension that can be used to configure the
// AWS client.
MiddlewareID *component.ID `mapstructure:"middleware,omitempty"`
Paramadon marked this conversation as resolved.
Show resolved Hide resolved
}

// DetectorConfig contains user-specified configurations unique to all individual detectors
Expand Down
3 changes: 3 additions & 0 deletions processor/resourcedetectionprocessor/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.22.5
require (
cloud.google.com/go/compute/metadata v0.3.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.23.0
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20241204155332-be45e31638de
Paramadon marked this conversation as resolved.
Show resolved Hide resolved
github.com/aws/aws-sdk-go v1.53.11
github.com/google/go-cmp v0.6.0
github.com/hashicorp/consul/api v1.29.1
Expand Down Expand Up @@ -38,6 +39,8 @@ require (
github.com/Showmax/go-fqdn v1.0.0 // indirect
github.com/amazon-contributing/opentelemetry-collector-contrib/override/aws v0.0.0-20240415183253-230331014d2c // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/aws/aws-sdk-go-v2 v1.22.2 // indirect
github.com/aws/smithy-go v1.16.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
Expand Down
24 changes: 24 additions & 0 deletions processor/resourcedetectionprocessor/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions processor/resourcedetectionprocessor/internal/aws/ec2/ec2.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import (
"net/http"
"regexp"

"github.com/aws/aws-sdk-go/aws/request"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
Expand Down Expand Up @@ -122,6 +124,10 @@ func (d *Detector) Detect(ctx context.Context) (resource pcommon.Resource, schem
return res, conventions.SchemaURL, nil
}

func (d *Detector) ExposeHandlers(ctx context.Context) (handlers *request.Handlers) {
return d.metadataProvider.GetHandlers(ctx)
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: For consistency and to make it easier to follow, keep the names of the functions used to access the request handlers the same. Would suggest keeping it the same as the internal/aws/cwlogs client.

func (client *Client) Handlers() *request.Handlers {
return &client.svc.(*cloudwatchlogs.CloudWatchLogs).Handlers
}


func getClientConfig(ctx context.Context, logger *zap.Logger) *http.Client {
client, err := internal.ClientFromContext(ctx)
if err != nil {
Expand Down
17 changes: 17 additions & 0 deletions processor/resourcedetectionprocessor/internal/resourcedetection.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ import (
"sync"
"time"

"github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware"
"github.com/aws/aws-sdk-go/aws/request"
"go.opentelemetry.io/collector/component"

"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/processor"
"go.uber.org/zap"
Expand All @@ -23,6 +27,10 @@ type Detector interface {
Detect(ctx context.Context) (resource pcommon.Resource, schemaURL string, err error)
}

type ExposeHandlerDetector interface {
Detector // Embed the existing Detector interface
ExposeHandlers(ctx context.Context) *request.Handlers
}
Paramadon marked this conversation as resolved.
Show resolved Hide resolved
type DetectorConfig any

type ResourceDetectorConfig interface {
Expand Down Expand Up @@ -116,6 +124,14 @@ func (p *ResourceProvider) Get(ctx context.Context, client *http.Client) (resour
return p.detectedResource.resource, p.detectedResource.schemaURL, p.detectedResource.err
}

func (p *ResourceProvider) ConfigureHandlers(ctx context.Context, host component.Host, middlewareId component.ID) {
for _, detector := range p.detectors {
if handlerDetector, ok := detector.(ExposeHandlerDetector); ok {
awsmiddleware.TryConfigure(p.logger, host, middlewareId, awsmiddleware.SDKv1(handlerDetector.ExposeHandlers(ctx)))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could grab the Configurer once and pass that into this function instead of the host and middlewareID so it isn't trying to get it each time.

}
}
}

func (p *ResourceProvider) detectResource(ctx context.Context) {
p.detectedResource = &resourceResult{}

Expand Down Expand Up @@ -196,3 +212,4 @@ func MergeResource(to, from pcommon.Resource, overrideTo bool) {
func IsEmptyResource(res pcommon.Resource) bool {
return res.Attributes().Len() == 0
}

Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import (
"context"
"errors"
"fmt"
"github.com/aws/aws-sdk-go/aws/request"
"go.opentelemetry.io/collector/component"
"net/http"
"sync"
"testing"
Expand Down Expand Up @@ -317,3 +319,100 @@ func TestFilterAttributes_NoAttributes(t *testing.T) {

assert.Equal(t, len(droppedAttributes), 0)
}

// mockDetectorWithHandler is a mock detector that implements ExposeHandlerDetector
type mockDetectorWithHandler struct {
handlersCalled bool
}

func (m *mockDetectorWithHandler) Detect(ctx context.Context) (resource pcommon.Resource, schemaURL string, err error) {
return pcommon.NewResource(), "", nil
}

func (m *mockDetectorWithHandler) ExposeHandlers(ctx context.Context) *request.Handlers {
m.handlersCalled = true
return &request.Handlers{}
}

// mockExtension implements component.Component
type mockExtension struct {
configured bool
}

func (m *mockExtension) Start(context.Context, component.Host) error {
return nil
}

func (m *mockExtension) Shutdown(context.Context) error {
return nil
}

// mockHost implements component.Host
type mockHost struct {
extensions map[component.ID]component.Component
}

// mockDetector is a basic detector that doesn't implement ExposeHandlerDetector
type mockDetector struct{}

func (m *mockDetector) Detect(ctx context.Context) (resource pcommon.Resource, schemaURL string, err error) {
return pcommon.NewResource(), "", nil
}

func newMockHost() component.Host {
return &mockHost{
extensions: make(map[component.ID]component.Component),
}
}

func (m *mockHost) GetExtension(id component.ID) (component.Component, error) {
if ext, ok := m.extensions[id]; ok {
return ext, nil
}
return nil, nil
}

func (m *mockHost) ReportFatalError(err error) {}

func (m *mockHost) GetFactory(kind component.Kind, componentType component.Type) component.Factory {
return nil
}

func (m *mockHost) GetExtensions() map[component.ID]component.Component {
return m.extensions
}

func (m *mockHost) GetExporters() map[component.DataType]map[component.ID]component.Component {
return nil
}

func TestConfigureHandlers(t *testing.T) {
testType, _ := component.NewType("awsmiddleware")
mockDetector := &mockDetectorWithHandler{}
logger := zap.NewNop()
provider := NewResourceProvider(logger, 0, nil, mockDetector)
mockExt := &mockExtension{}
host := newMockHost()
host.(*mockHost).extensions[component.NewID(testType)] = mockExt

middlewareID := component.NewID(testType)
ctx := context.Background()
provider.ConfigureHandlers(ctx, host, middlewareID)

assert.True(t, mockDetector.handlersCalled, "ExposeHandlers should have been called")
}

func TestConfigureHandlersWithNonHandlerDetector(t *testing.T) {
testType, _ := component.NewType("awsmiddleware")
basicDetector := &mockDetector{}
logger := zap.NewNop()
provider := NewResourceProvider(logger, 0, nil, basicDetector)

mockExt := &mockExtension{}
host := newMockHost()
host.(*mockHost).extensions[component.NewID(testType)] = mockExt
middlewareID := component.NewID(testType)

ctx := context.Background()
provider.ConfigureHandlers(ctx, host, middlewareID)
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
)

type resourceDetectionProcessor struct {
middlewareID *component.ID
provider *internal.ResourceProvider
resource pcommon.Resource
schemaURL string
Expand All @@ -30,6 +31,9 @@ func (rdp *resourceDetectionProcessor) Start(ctx context.Context, host component
client, _ := rdp.httpClientSettings.ToClient(ctx, host, rdp.telemetrySettings)
ctx = internal.ContextWithClient(ctx, client)
var err error
if host != nil && rdp.middlewareID != nil{
rdp.provider.ConfigureHandlers(ctx, host, *rdp.middlewareID) //configuring middleware in all clients of detectors
}
rdp.resource, rdp.schemaURL, err = rdp.provider.Get(ctx, client)
return err
}
Expand Down
6 changes: 6 additions & 0 deletions receiver/awscontainerinsightreceiver/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ package awscontainerinsightreceiver // import "github.com/open-telemetry/opentel
import (
"time"

"go.opentelemetry.io/collector/component"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil"
)

Expand Down Expand Up @@ -72,4 +74,8 @@ type Config struct {

// RunOnSystemd is an optional attribute to run the receiver in an EC2 environment
RunOnSystemd bool `mapstructure:"run_on_systemd,omitempty"`

// MiddlewareID is an ID for an extension that can be used to configure the
// AWS client.
MiddlewareID *component.ID `mapstructure:"middleware,omitempty"`
Paramadon marked this conversation as resolved.
Show resolved Hide resolved
}
3 changes: 3 additions & 0 deletions receiver/awscontainerinsightreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ go 1.22.5

require (
github.com/Microsoft/hcsshim v0.12.0-rc.3
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.0.0-20241204155332-be45e31638de
github.com/amazon-contributing/opentelemetry-collector-contrib/override/aws v0.0.0-00010101000000-000000000000
github.com/aws/aws-sdk-go v1.53.11
github.com/go-kit/log v0.2.1
Expand Down Expand Up @@ -53,6 +54,8 @@ require (
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/alecthomas/units v0.0.0-20231202071711-9a357b53e9c9 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/aws/aws-sdk-go-v2 v1.22.2 // indirect
github.com/aws/smithy-go v1.16.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/semver/v4 v4.0.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
Expand Down
24 changes: 24 additions & 0 deletions receiver/awscontainerinsightreceiver/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading