Skip to content

Commit

Permalink
Replace Dockerfile with Scratch as Base Image (#214)
Browse files Browse the repository at this point in the history
*Issue #, if available:*
Currently amazon linux is used as the base image for OTEL python
distribution. This image contains a lot of unnecessary packages that
inflate the image and also cause issues during high sev scans. Ideally
we would like to use an image with minimal content.

`Scratch` was the base image that was proposed to be used with
`cpUtility` installed to enable the `cp` command to copy the python
image to the sample app pod. The
[cpUtility](https://github.com/aws-observability/aws-otel-java-instrumentation/tree/main/tools/cp-utility)
is copied from aws-otel-java-instrumentation repo, but there is a small
issue where if copied from a folder, the destination directory is not
correct.

This piece of
[code](https://github.com/aws-observability/aws-otel-java-instrumentation/blob/main/tools/cp-utility/src/main.rs#L79-L87)
in cp-utility causes the directory to change if the destination folder
already exists. It was
[modified](https://github.com/aws-observability/aws-otel-python-instrumentation/pull/214/files#diff-255e1973e4371a91e310ea9c1b94c1507bab752b6aa20b39322542e6d205616aR77-R80)
so that that if the destination folder exists, just reuse it and don't
create a new folder. Other than also modifying the [unit
test](https://github.com/aws-observability/aws-otel-python-instrumentation/pull/214/files#diff-255e1973e4371a91e310ea9c1b94c1507bab752b6aa20b39322542e6d205616aR269-R308),
the code is identical to the otel java

Test run:
https://github.com/aws-observability/aws-otel-python-instrumentation/actions/runs/9586337462

Also manually built the image locally and deployed to EKS cluster with
sample app. Verified that correct telemetry was generated and
`opentelemetry` folder is located in correct path

Additionally, ran testing in ARM64 EC2 instance

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
  • Loading branch information
harrryr authored Jul 5, 2024
2 parents 9ff3b90 + 4488008 commit 6177b7e
Show file tree
Hide file tree
Showing 7 changed files with 685 additions and 4 deletions.
2 changes: 1 addition & 1 deletion .codespellrc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[codespell]
# skipping auto generated folders
skip = ./.tox,./.mypy_cache,./target,*/LICENSE,./venv,*/sql_dialect_keywords.json
ignore-words-list = afterall,assertIn
ignore-words-list = afterall,assertIn, crate
39 changes: 36 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
# The packages are installed in the `/autoinstrumentation` directory. This is required as when instrumenting the pod by CWOperator,
# one init container will be created to copy all the content in `/autoinstrumentation` directory to app's container. Then
# update the `PYTHONPATH` environment variable accordingly. Then in the second stage, copy the directory to `/autoinstrumentation`.

# Stage 1: Install ADOT Python in the /operator-build folder
FROM python:3.11 AS build

WORKDIR /operator-build
Expand All @@ -18,11 +20,42 @@ RUN sed -i "/opentelemetry-exporter-otlp-proto-grpc/d" ./aws-opentelemetry-distr

RUN mkdir workspace && pip install --target workspace ./aws-opentelemetry-distro

FROM public.ecr.aws/amazonlinux/amazonlinux:minimal
# Stage 2: Build the cp-utility binary
FROM rust:1.75 as builder

WORKDIR /usr/src/cp-utility
COPY ./tools/cp-utility .

## TARGETARCH is defined by buildx
# https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
ARG TARGETARCH

# Run validations and audit only on amd64 because it is faster and those two steps
# are only used to validate the source code and don't require anything that is
# architecture specific.

# Validations
# Validate formatting
RUN if [ $TARGETARCH = "amd64" ]; then rustup component add rustfmt && cargo fmt --check ; fi

# Audit dependencies
RUN if [ $TARGETARCH = "amd64" ]; then cargo install cargo-audit && cargo audit ; fi


# Cross-compile based on the target platform.
RUN if [ $TARGETARCH = "amd64" ]; then export ARCH="x86_64" ; \
elif [ $TARGETARCH = "arm64" ]; then export ARCH="aarch64" ; \
else false; \
fi \
&& rustup target add ${ARCH}-unknown-linux-musl \
&& cargo test --target ${ARCH}-unknown-linux-musl \
&& cargo install --target ${ARCH}-unknown-linux-musl --path . --root .

# Stage 3: Build the distribution image by copying the THIRD-PARTY-LICENSES, the custom built cp command from stage 2, and the installed ADOT Python from stage 1 to their respective destinations
FROM scratch

# Required to copy attribute files to distributed docker images
ADD THIRD-PARTY-LICENSES ./THIRD-PARTY-LICENSES

COPY --from=builder /usr/src/cp-utility/bin/cp-utility /bin/cp
COPY --from=build /operator-build/workspace /autoinstrumentation

RUN chmod -R go+r /autoinstrumentation
3 changes: 3 additions & 0 deletions tools/cp-utility/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Use git CLI to fetch the source code instead of relying on the rust git implementation
[net]
git-fetch-with-cli = true
221 changes: 221 additions & 0 deletions tools/cp-utility/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions tools/cp-utility/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
[package]
name = "cp-utility"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
# No dependencies here

[dev-dependencies]
# dependencies only used during tests
tempfile = "3.9.0"
uuid = { version = "1.5.0", features = ["v4", "fast-rng"] }

[profile.release]
# Levers to optimize the binary for size
strip = true # Strip symbols
opt-level = "z" # Size optimization
lto = true # linking time optimizations


57 changes: 57 additions & 0 deletions tools/cp-utility/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Introduction

This copy utility is intended to be used as a base image for OpenTelemetry Operator
autoinstrumentation images. The copy utility will allow the ADOT Java agent jar to be
copied from the init container to the final destination volume.

## Development

### Pre-requirements
* Install rust

```
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

* Install rustfmt

```
rustup component add rustfmt
```

### Development

* Auto formatting the code

This step is important and it might fail the build if the files are not properly
formatted.

```
cargo fmt
```

* Testing the code
```
cargo test
```

* Building the code

```
cargo build
```

NOTE: this will build the code for tests locally. It will not statically link the libc used by it.


* Building the code statically linked

```
cargo build --target x86_64-unknown-linux-musl
```


### Docker image

In the root of this project, there is a Dockerfile that is supposed to be used during release.
This Dockerfile can be used with buildx to generate images for the arm64 and x86_64 platforms.
Loading

0 comments on commit 6177b7e

Please sign in to comment.