Skip to content

Commit

Permalink
dockerfile works
Browse files Browse the repository at this point in the history
  • Loading branch information
jmacd committed Nov 9, 2023
1 parent fdc85f5 commit b148a83
Show file tree
Hide file tree
Showing 9 changed files with 173 additions and 2,008 deletions.
17 changes: 7 additions & 10 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,16 @@ ENV CGO_ENABLED=0
# to the last-successful version of the OpenTelemetry collector.
RUN go install go.opentelemetry.io/collector/cmd/builder@latest

# This command generates main.go, go.mod, and then builds using the
# container's go toolchain. Note the 'exit 0' at the end of this
# command ignores the result of the builder. See commands in
# Makefile above the `genotelarrow` rule for an explanation.
RUN builder --skip-compilation --config=collector/otelarrowcol-build.yaml; exit 0
# This command generates main.go, go.mod but does not update deps.
RUN builder --skip-compilation --skip-get-modules --config=collector/otelarrowcol-build.yaml

# This two-stage build will succeed because there is a `go.work`
# checked-in to the repository.
RUN go install ./collector/cmd/otelarrowcol
# This build will update the go.mod, using the checked-in go.work file
# in the repository.
RUN go build -o otelarrowcol ./collector/cmd/otelarrowcol

# This build uses an Alpine Linux container.
FROM alpine AS release
COPY --from=sandbox /otel-arrow/collector/cmd/otelarrowcol/otelarrowcol /
COPY --from=sandbox /otel-arrow/otelarrowcol /

# Network ports
# 4317 - OpenTelemetry gRPC services:
Expand All @@ -41,4 +38,4 @@ COPY --from=sandbox /otel-arrow/collector/cmd/otelarrowcol/otelarrowcol /
# 1777 - Profiling support
EXPOSE 4317/tcp 1777/tcp

CMD ["/otelarrowcol"]
ENTRYPOINT ["/otelarrowcol"]
10 changes: 4 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -105,17 +105,15 @@ endif
git commit -m "add multimod changes $(RELEASE_CANDIDATE)" || (echo "no multimod changes to commit")

# Install OTC's builder at the latest version
BUILDER := builder
BUILDER = builder
.PHONY: $(BUILDER)
builder:
$(GOCMD) install go.opentelemetry.io/collector/cmd/builder@latest

# Note the /bin/true at the end of the builder command is because with
# current practice (for some reason) the initial `go mod tidy` will
# fail and not recognize the go.work. Immediately after, the `make
# otelarrowcol` step is expected to succeed.
.PHONY: genotelarrowcol
genotelarrowcol: builder
$(BUILDER) --skip-compilation --config collector/otelarrowcol-build.yaml || (echo "build did not succeed, but this may be a known module problem; please run make otelarrowcol next"; true)
rm -f collector/cmd/otelarrowcol/*
$(BUILDER) --skip-compilation --skip-get-modules --config collector/otelarrowcol-build.yaml

.PHONY: otelarrowcol
otelarrowcol:
Expand Down
65 changes: 42 additions & 23 deletions collector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ We are aware that building and distributing OpenTelemetry collectors
is not a simple task and have prepared dedicated instructions for
building and testing the components in this repository.

[Instructions for building a OpenTelemetry Collector with support for
OpenTelemetry Protocol with Apache Arrow](./BUILDING.md).
[Instructions for building an OpenTelemetry Collector with support for
OpenTelemetry Protocol with Apache Arrow.][BUILDING]

[We would prefer to include these components in the OpenTelemetry
Contrib Collector, because it is an officially maintained artifact.
Expand All @@ -32,47 +32,66 @@ maintainers. These components are:

### For production use

- `processor/concurrentbatchprocessor`: Derived from the upstream
[batchprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/batchprocessor),
- [`processor/concurrentbatchprocessor`][CONCURRENTBATCHPROCESSOR]:
Derived from the upstream [batchprocessor][UPSTREAMBATCHPROCESSOR],
this component is enhanced with the ability to send batches
concurrently, with an overall in-flight-bytes limit.

### For research and validation

- `exporter/fileexporter`: Derived from the upstream
[fileexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/fileexporter),
this component supports writing files that can be read by the
corresponding `filereceiver` in this package (unlike the upstream).
- `receiver/filereceiver`: Derived from the upstream
[filereceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filereceiver),
this component supports reading files written by the corresponding
`fileexporter` in this package (unlike the upstream).
- `processor/obfuscationprocessor`: Supports obfuscation of
OpenTelemetry data using a [Feistel
- [`exporter/fileexporter`][ARROWFILEEXPORTER]: Derived from the
upstream [fileexporter][UPSTREAMFILEEXPORTER], this component
supports writing files that can be read by the corresponding
`filereceiver` in this package (which the upstream cannot do).
- [`receiver/filereceiver`][ARROWFILERECEIVER]: Derived from the
upstream [filereceiver][UPSTREAMFILERECEIVER], this component
supports reading files written by the corresponding `fileexporter`
in this package (unlike the upstream).
- [`processor/obfuscationprocessor`][OBFUSCATIONPROCESSOR]: Supports
obfuscation of OpenTelemetry data using a [Feistel
cipher](https://en.wikipedia.org/wiki/Feistel_cipher).
- `processor/experimentprocessor`: A probabilistic routing component
for conducting experiments between exporters.
- `connector/validationconnector`: A component for on-the-fly
validation of a local pipeline.
- [`processor/experimentprocessor`][EXPERIMENTPROCESSOR]: A
probabilistic routing component for conducting experiments between
exporters.
- [`connector/validationconnector`][VALIDATIONCONNECTOR]: A component
for on-the-fly validation of a local pipeline.

## Other extensions built into `otelarrowcol`

Several Collector-Contrib extensions are included in the build:

- [basicauth](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/basicauthextension/README.md): Allows use of username and password
- [headersetter](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/headerssetterextension/README.md): Allows propagating headers through a pipeline
- [pprof](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/pprofextension/README.md): Allows use of Golang profiling tools.
- [basicauth][BASICAUTHEXT]: Allows use of username and password for
authorization.
- [headersetter][HEADERSETTEREXT]: Allows propagating headers through
a pipeline
- [pprof][PPROFEXT]: Allows use of Golang profiling tools.

From the core collector repository:

- [otelhttpexporter](): Useful for debugging, sends standard OTLP over HTTP
- [loggingexporter](): Useful for debugging, prints OTLP data to the console
- [otelhttpexporter][UPSTREAMHTTPOTLP]: Useful for debugging, sends standard OTLP over HTTP
- [debugexporter][UPSTREAMDEBUG]: Useful for debugging, prints OTLP data to the console

Also, a synthetic telemetry data generator:

- [generator](https://github.com/lightstep/telemetry-generator/blob/main/README.md): Produces synthetic telemetry data.


[BUILDING]: ./BUILDING.md
[EXPORTER]: ./exporter/otelarrowexporter/README.md
[RECEIVER]: ./receiver/otelarrowreceiver/README.md
[CONTRIBUTION]: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/26491
[UPSTREAMBATCHPROCESSOR]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md
[CONCURRENTBATCHPROCESSOR]: ./processor/concurrentbatchprocessor/README.md
[ARROWFILEEXPORTER]: ./exporter/fileexporter/README.md
[ARROWFILERECEIVER]: ./receiver/filereceiver/README.md
[UPSTREAMFILEEXPORTER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/fileexporter/README.md
[UPSTREAMFILERECEIVER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filereceiver/README.md
[OBFUSCATIONPROCESSOR]: ./processor/obfuscationprocessor/README.md
[EXPERIMENTPROCESSOR]: ./processor/experimentprocessor/README.md
[VALIDATIONCONNECTOR]: ./connector/validationconnector/README.md
[BASICAUTHEXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/basicauthextension/README.md
[HEADERSETTEREXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/headerssetterextension/README.md
[PPROFEXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/pprofextension/README.md
https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md
[UPSTREAMHTTPOTLP]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/README.md
[UPSTREAMDEBUG]:
12 changes: 6 additions & 6 deletions collector/cmd/otelarrowcol/components.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit b148a83

Please sign in to comment.