Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial import of lightstepreceiver #16

Merged
merged 11 commits into from
Jun 10, 2024
30 changes: 30 additions & 0 deletions collector/components/lightstepreceiver/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Design

## Summary

This receiver mimics what is provided by the Lightstep Microsatellites:

* Clock correction: The legacy tracers and this receiver send each other
timestamp values in order to sync up any missmatched clocks.
- `ReportRequest`: `TimestampOffsetMicros` includes an offset that MUST
be applied to all timestamps being reported. This value is zero if
no clock correction is required.
- `ReportResponse`: This receiver sends two timestamps, `Receive` and
`Transmit`, with the times at which the latest request was received
and later answered with a response, in order to help the tracers
adjust their offsets.

## TODO

* Implement gRPC support.
* Implement OBSReport.
* Consider mapping semantic conventions:
- Values that can be consumed within the processor, e.g. detect event names from Logs, detect `StatusKind` from error tags.
- Values that affect the entire OT ecosystem. Probably can be offered as a separate processor instead.
- Lightstep-specific tags (attributes) that _may_ need to be mapped to become useful for OTel processors.
* `Baggage` is being sent as part of Lightstep's `SpanContext`, but it is not exported in any way at this point.
* Find all special Tags (e.g. "lightstep.component_name") and think which ones we should map
(there is already a document about this somewhere).
* Legacy tracers send payloads using the `application/octet-stream` content type and using the
`/api/v2/reports` path. We don't check for it but worth verifying this.
* Top level `ReporterId` is not being used at this moment.
27 changes: 27 additions & 0 deletions collector/components/lightstepreceiver/big_endian_converter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

// Partially copied from github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal/idutils

package lightstepreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/lightstepreceiver"

import (
"encoding/binary"

"go.opentelemetry.io/collector/pdata/pcommon"
)

// UInt64ToTraceID converts the pair of uint64 representation of a TraceID to pcommon.TraceID.
func UInt64ToTraceID(high, low uint64) pcommon.TraceID {
traceID := [16]byte{}
binary.BigEndian.PutUint64(traceID[:8], high)
binary.BigEndian.PutUint64(traceID[8:], low)
return traceID
}

// UInt64ToSpanID converts the uint64 representation of a SpanID to pcommon.SpanID.
func UInt64ToSpanID(id uint64) pcommon.SpanID {
spanID := [8]byte{}
binary.BigEndian.PutUint64(spanID[:], id)
return pcommon.SpanID(spanID)
}
64 changes: 64 additions & 0 deletions collector/components/lightstepreceiver/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package lightstepreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/lightstepreceiver"

import (
"errors"
"fmt"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"
"go.opentelemetry.io/collector/confmap"
)

const (
protocolsFieldName = "protocols"
protoHTTP = "http"
)

type HTTPConfig struct {
*confighttp.ServerConfig `mapstructure:",squash"`
}

// Protocols is the configuration for the supported protocols.
type Protocols struct {
HTTP *HTTPConfig `mapstructure:"http"`
}

// Config defines configuration for the Lightstep receiver.
type Config struct {
// Protocols is the configuration for the supported protocols, currently HTTP.
Protocols `mapstructure:"protocols"`
}

var _ component.Config = (*Config)(nil)

// Validate checks the receiver configuration is valid
func (cfg *Config) Validate() error {
if cfg.HTTP == nil {
return errors.New("must specify at least one protocol when using the Lightstep receiver")
}
return nil
}

// Unmarshal a confmap.Conf into the config struct.
func (cfg *Config) Unmarshal(componentParser *confmap.Conf) error {
if componentParser == nil {
return fmt.Errorf("nil config for lightstepreceiver")
}

err := componentParser.Unmarshal(cfg)
if err != nil {
return err
}
protocols, err := componentParser.Sub(protocolsFieldName)
if err != nil {
return err
}

if !protocols.IsSet(protoHTTP) {
cfg.HTTP = nil
}
return nil
}
121 changes: 121 additions & 0 deletions collector/components/lightstepreceiver/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package lightstepreceiver

import (
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"
"go.opentelemetry.io/collector/config/configtls"
"go.opentelemetry.io/collector/confmap"
"go.opentelemetry.io/collector/confmap/confmaptest"
)

func TestUnmarshalDefaultConfig(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "default.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))
assert.Equal(t, factory.CreateDefaultConfig(), cfg)
}

func TestUnmarshalConfigOnlyHTTP(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "only_http.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))

defaultOnlyHTTP := factory.CreateDefaultConfig().(*Config)
assert.Equal(t, defaultOnlyHTTP, cfg)
}

func TestUnmarshalConfigOnlyHTTPNull(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "only_http_null.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))

defaultOnlyHTTP := factory.CreateDefaultConfig().(*Config)
assert.Equal(t, defaultOnlyHTTP, cfg)
}

func TestUnmarshalConfigOnlyHTTPEmptyMap(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "only_http_empty_map.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))

defaultOnlyHTTP := factory.CreateDefaultConfig().(*Config)
assert.Equal(t, defaultOnlyHTTP, cfg)
}

func TestUnmarshalConfig(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "config.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))
assert.Equal(t,
&Config{
Protocols: Protocols{
HTTP: &HTTPConfig{
ServerConfig: &confighttp.ServerConfig{
Endpoint: "0.0.0.0:443",
TLSSetting: &configtls.ServerConfig{
Config: configtls.Config{
CertFile: "test.crt",
KeyFile: "test.key",
},
},
CORS: &confighttp.CORSConfig{
AllowedOrigins: []string{"https://*.test.com", "https://test.com"},
MaxAge: 7200,
},
},
},
},
}, cfg)

}

func TestUnmarshalConfigTypoDefaultProtocol(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "typo_default_proto_config.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.EqualError(t, component.UnmarshalConfig(cm, cfg), "1 error(s) decoding:\n\n* 'protocols' has invalid keys: htttp")
}

func TestUnmarshalConfigInvalidProtocol(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "bad_proto_config.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.EqualError(t, component.UnmarshalConfig(cm, cfg), "1 error(s) decoding:\n\n* 'protocols' has invalid keys: thrift")
}

func TestUnmarshalConfigEmptyProtocols(t *testing.T) {
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "bad_no_proto_config.yaml"))
require.NoError(t, err)
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(cm, cfg))
assert.EqualError(t, component.ValidateConfig(cfg), "must specify at least one protocol when using the Lightstep receiver")
}

func TestUnmarshalConfigEmpty(t *testing.T) {
factory := NewFactory()
cfg := factory.CreateDefaultConfig()
assert.NoError(t, component.UnmarshalConfig(confmap.New(), cfg))
assert.EqualError(t, component.ValidateConfig(cfg), "must specify at least one protocol when using the Lightstep receiver")
}
55 changes: 55 additions & 0 deletions collector/components/lightstepreceiver/factory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package lightstepreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/lightstepreceiver"

import (
"context"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"
"go.opentelemetry.io/collector/consumer"
"go.opentelemetry.io/collector/receiver"

"github.com/lightstep/sn-collector/collector/lightstepreceiver/internal/metadata"
)

// This file implements factory bits for the Lightstep receiver.

const (
// TODO: Define a new port for us to use.
defaultBindEndpoint = "0.0.0.0:443"
)

// NewFactory creates a new Lightstep receiver factory
func NewFactory() receiver.Factory {
return receiver.NewFactory(
component.MustNewType(metadata.Type),
createDefaultConfig,
receiver.WithTraces(createTracesReceiver, metadata.TracesStability),
)
}

// createDefaultConfig creates the default configuration for Lightstep receiver.
func createDefaultConfig() component.Config {
return &Config{
Protocols: Protocols{
HTTP: &HTTPConfig{
ServerConfig: &confighttp.ServerConfig{
Endpoint: defaultBindEndpoint,
},
},
},
}
}

// createTracesReceiver creates a trace receiver based on provided config.
func createTracesReceiver(
_ context.Context,
set receiver.CreateSettings,
cfg component.Config,
consumer consumer.Traces,
) (receiver.Traces, error) {
rCfg := cfg.(*Config)
return newReceiver(rCfg, consumer, set)
}
41 changes: 41 additions & 0 deletions collector/components/lightstepreceiver/factory_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package lightstepreceiver

import (
"context"
"testing"

"github.com/stretchr/testify/assert"

"go.opentelemetry.io/collector/component/componenttest"
"go.opentelemetry.io/collector/consumer/consumertest"
"go.opentelemetry.io/collector/receiver/receivertest"
)

func TestCreateDefaultConfig(t *testing.T) {
cfg := createDefaultConfig()
assert.NotNil(t, cfg, "failed to create default config")
assert.NoError(t, componenttest.CheckConfigStruct(cfg))
}

func TestCreateReceiver(t *testing.T) {
cfg := createDefaultConfig()

tReceiver, err := createTracesReceiver(
context.Background(),
receivertest.NewNopCreateSettings(),
cfg,
consumertest.NewNop())
assert.NoError(t, err, "receiver creation failed")
assert.NotNil(t, tReceiver, "receiver creation failed")

tReceiver, err = createTracesReceiver(
context.Background(),
receivertest.NewNopCreateSettings(),
cfg,
consumertest.NewNop())
assert.NoError(t, err, "receiver creation failed")
assert.NotNil(t, tReceiver, "receiver creation failed")
}
Loading
Loading