Skip to content

Commit

Permalink
Add command to generate schemas to uuid-annotator Docker image (#59)
Browse files Browse the repository at this point in the history
* Add command to generate uuid-annotator schemas
* Build generate-schemas command
* Add hint about generate-schemas to README.md
  • Loading branch information
stephen-soltesz authored Nov 6, 2023
1 parent 7509693 commit a9f9600
Show file tree
Hide file tree
Showing 5 changed files with 328 additions and 2 deletions.
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@ RUN go get -v . && \
CGO_ENABLED=0 go install -v \
-ldflags "-X github.com/m-lab/go/prometheusx.GitShortCommit=$(git log -1 --format=%h)" \
.
RUN cd ./cmd/generate-schemas && CGO_ENABLED=0 go install -v .

# Put it in its own image.
FROM alpine
FROM alpine:3.18
COPY --from=build /go/bin/uuid-annotator /uuid-annotator
COPY --from=build /go/bin/generate-schemas /generate-schemas
COPY ./data/asnames.ipinfo.csv /data/asnames.ipinfo.csv
# In the fullness of time, we would like to replace this local file with a
# download from a GCS url that we control and is passed in as a command-line
Expand All @@ -18,6 +20,7 @@ COPY ./data/asnames.ipinfo.csv /data/asnames.ipinfo.csv
# worth it, we ship the 3.7MB data file with the binary.
ENV ASNAME_URL file:///data/asnames.ipinfo.csv
WORKDIR /
# Make sure /uuid-annotator can run (has no missing external dependencies).
# Make sure binaries can run (has no missing external dependencies).
RUN /uuid-annotator -h 2> /dev/null
RUN /generate-schemas -h 2> /dev/null
ENTRYPOINT ["/uuid-annotator"]
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,13 @@ docker run -v $PWD/testdata:/testdata -it local-annotator \
-routeview-v4.url=file:///testdata/RouteViewIPv4.pfx2as.gz \
-routeview-v6.url=file:///testdata/RouteViewIPv6.pfx2as.gz
```

### Generate Schemas

If using uuid-annotator data as part of the autoloader pipeline, you may
generate the data type schemas using the `generate-schemas` command:

```sh
docker run -v $PWD:/schemas --entrypoint /generate-schemas -it local-annotator \
-ann2 /schemas/ann2.json -hop2 /schemas/hop2.json
```
55 changes: 55 additions & 0 deletions cmd/generate-schemas/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
module github.com/m-lab/uuid-annotator/cmd/generate-schemas

go 1.20

require (
cloud.google.com/go/bigquery v1.57.1
github.com/m-lab/go v0.1.66
github.com/m-lab/uuid-annotator v0.5.3
)

require (
cloud.google.com/go v0.110.8 // indirect
cloud.google.com/go/compute v1.23.1 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.3 // indirect
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/apache/arrow/go/v12 v12.0.0 // indirect
github.com/apache/thrift v0.16.0 // indirect
github.com/goccy/go-json v0.9.11 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/flatbuffers v2.0.8+incompatible // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/uuid v1.4.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/m-lab/tcp-info v1.5.3 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1 // indirect
golang.org/x/mod v0.10.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.13.0 // indirect
golang.org/x/sync v0.4.0 // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/tools v0.9.1 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/api v0.149.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
)
Loading

0 comments on commit a9f9600

Please sign in to comment.