Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vc/incoming device api #38

Merged
merged 21 commits into from
Apr 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
16960e5
update golang and golangci-lint
DoctorVin Mar 21, 2024
eb1da37
add a skeleton for incoming CIS data
DoctorVin Mar 21, 2024
b8dc43d
implement attribute upsert for incoming inventory
DoctorVin Mar 22, 2024
638183e
WIP: inventory updates
DoctorVin Mar 26, 2024
aaaf521
move component type initialization to FleetDB
DoctorVin Apr 3, 2024
cb2a829
refactor updateAnyAttribute so we can do component attributes too
DoctorVin Apr 4, 2024
70b5431
add real component initialization to setup and fix test fallout
DoctorVin Apr 5, 2024
bff98be
flesh out the incoming inventory API a bit
DoctorVin Apr 5, 2024
fd4e080
parallel tests are problematic with a big global database
DoctorVin Apr 5, 2024
c6935f7
generalize the routines for upserting attributes and versioned attrib…
DoctorVin Apr 5, 2024
9b2c286
implement bios and dimms
DoctorVin Apr 8, 2024
7ccad4f
implement bmc and mainboard
DoctorVin Apr 9, 2024
6cbaf7b
refactor the common parts out of the update functions
DoctorVin Apr 9, 2024
3128379
change the namespacing and fimware/status handling for vattr
DoctorVin Apr 12, 2024
f85fd81
implement generic deserialization routines
DoctorVin Apr 16, 2024
7a51fc3
update deps
DoctorVin Apr 16, 2024
7eeae19
convert to rivets.Server
DoctorVin Apr 18, 2024
48652ca
enforce a test timeout to catch hangs
DoctorVin Apr 18, 2024
6f7d7b0
update linter config and fix linting issues in code
DoctorVin Apr 18, 2024
1432f25
finish API and tests for inventory
DoctorVin Apr 19, 2024
60c4214
fix up some overly brittle tests
DoctorVin Apr 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/lint-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: '1.21.0'
go-version: '1.22.1'

DoctorVin marked this conversation as resolved.
Show resolved Hide resolved
- name: Install cockroach binary
run: curl https://binaries.cockroachdb.com/cockroach-v23.1.11.linux-amd64.tgz | tar -xz && sudo cp -i cockroach-v23.1.11.linux-amd64/cockroach /usr/local/bin/
Expand All @@ -32,12 +32,12 @@ jobs:
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: v1.52.2
version: v1.57.1
args: --timeout=5m

- name: Run go tests and generate coverage report
run: FLEETDB_CRDB_URI="host=localhost port=26257 user=root sslmode=disable dbname=fleetdb_test" go test -race -coverprofile=coverage.txt -covermode=atomic -tags testtools -p 1 ./...

- name: Stop test database
run: cockroach node drain --insecure --host=localhost:26257

Expand Down
4 changes: 3 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ linters-settings:
local-prefixes: github.com/metal-toolbox/fleetdb
gofumpt:
extra-rules: true
stylecheck:
checks: ["all", "-ST1000"]

linters:
enable:
Expand All @@ -31,7 +33,7 @@ linters:
- govet
- misspell
- noctx
- revive
# - revive XXX: fix up old code caught here
- stylecheck
- whitespace

Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ test: | unit-test integration-test
## run integration tests
integration-test: test-database
@echo Running integration tests...
@FLEETDB_CRDB_URI="${TEST_DB}" go test -cover -tags testtools,integration -p 1 ./...
@FLEETDB_CRDB_URI="${TEST_DB}" go test -cover -tags testtools,integration -p 1 -timeout 1m ./...

## run lint and unit tests
unit-test: | lint
Expand All @@ -24,7 +24,7 @@ unit-test: | lint
## check test coverage
coverage: | test-database
@echo Generating coverage report...
@FLEETDB_CRDB_URI="${TEST_DB}" go test ./... -race -coverprofile=coverage.out -covermode=atomic -tags testtools -p 1
@FLEETDB_CRDB_URI="${TEST_DB}" go test ./... -race -coverprofile=coverage.out -covermode=atomic -tags testtools,integration -p 1
@go tool cover -func=coverage.out
@go tool cover -html=coverage.out

Expand Down
7 changes: 7 additions & 0 deletions cmd/serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"go.infratographer.com/x/crdbx"
"go.infratographer.com/x/otelx"
"go.infratographer.com/x/viperx"
"go.uber.org/zap"
"gocloud.dev/secrets"

// import gocdk secret drivers
Expand Down Expand Up @@ -92,6 +93,12 @@ func serve(ctx context.Context) {

dbtools.RegisterHooks()

if err := dbtools.SetupComponentTypes(ctx, db); err != nil {
logger.With(
zap.Error(err),
).Fatal("set up component types")
}

keeper, err := secrets.OpenKeeper(ctx, viper.GetString("db.encryption_driver"))
if err != nil {
logger.Fatalw("failed to open secrets keeper", "error", err)
Expand Down
109 changes: 55 additions & 54 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,130 +1,131 @@
module github.com/metal-toolbox/fleetdb

go 1.19
go 1.22

require (
github.com/XSAM/otelsql v0.23.0 // indirect
github.com/cockroachdb/cockroach-go/v2 v2.3.5
github.com/cockroachdb/cockroach-go/v2 v2.3.6
github.com/friendsofgo/errors v0.9.2
github.com/gin-contrib/cors v1.4.0
github.com/gin-contrib/cors v1.5.0
github.com/gin-contrib/zap v1.1.0
github.com/gin-gonic/gin v1.9.1
github.com/google/uuid v1.3.1
github.com/gosimple/slug v1.13.1
github.com/google/uuid v1.6.0
github.com/gosimple/slug v1.14.0
github.com/jmoiron/sqlx v1.3.5
github.com/lib/pq v1.10.9
github.com/pkg/errors v0.9.1
github.com/pressly/goose/v3 v3.15.0 // indirect
github.com/prometheus/client_golang v1.16.0 // indirect
github.com/spf13/cobra v1.7.0
github.com/prometheus/client_golang v1.19.0 // indirect
github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.5 // indirect
github.com/spf13/viper v1.16.0
github.com/spf13/viper v1.18.2
github.com/stretchr/testify v1.9.0
github.com/volatiletech/null/v8 v8.1.2
github.com/volatiletech/randomize v0.0.1
github.com/volatiletech/sqlboiler/v4 v4.15.0
github.com/volatiletech/strmangle v0.0.5
github.com/volatiletech/sqlboiler/v4 v4.16.2
github.com/volatiletech/strmangle v0.0.6
github.com/zsais/go-gin-prometheus v0.1.0
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.42.0
go.opentelemetry.io/otel v1.16.0
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.49.0
go.opentelemetry.io/otel v1.25.0
go.opentelemetry.io/otel/exporters/jaeger v1.16.0 // indirect
go.opentelemetry.io/otel/sdk v1.16.0 // indirect
go.opentelemetry.io/otel/sdk v1.25.0 // indirect
go.uber.org/zap v1.27.0
gopkg.in/square/go-jose.v2 v2.6.0
)

require (
github.com/bmc-toolbox/common v0.0.0-20231204194243-7bcbccab7116
github.com/metal-toolbox/rivets v1.0.3
github.com/volatiletech/sqlboiler v3.7.1+incompatible
go.hollow.sh/toolbox v0.6.3
go.infratographer.com/x v0.3.7
gocloud.dev v0.33.0
golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63
go.infratographer.com/x v0.3.9
gocloud.dev v0.36.0
golang.org/x/exp v0.0.0-20240222234643-814bf88cf225
)

require (
cloud.google.com/go/kms v1.15.7 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bytedance/sonic v1.11.3 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d // indirect
github.com/chenzhuoyu/iasm v0.9.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/ericlagergren/decimal v0.0.0-20221120152707-495c53812d05 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/ericlagergren/decimal v0.0.0-20240305081647-93d586550569 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.19.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gofrs/uuid v4.4.0+incompatible // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/googleapis/gax-go/v2 v2.12.2 // indirect
github.com/gosimple/unidecode v1.0.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgconn v1.14.1 // indirect
github.com/jackc/pgconn v1.14.3 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.3.2 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/pgtype v1.14.0 // indirect
github.com/jackc/pgx/v4 v4.18.1 // indirect
github.com/jackc/pgproto3/v2 v2.3.3 // indirect
github.com/jackc/pgservicefile v0.0.0-20231201235250-de7065d80cb9 // indirect
github.com/jackc/pgtype v1.14.2 // indirect
github.com/jackc/pgx/v4 v4.18.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.16.7 // indirect
github.com/klauspost/compress v1.17.7 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/nats-io/nats.go v1.28.0 // indirect
github.com/nats-io/nkeys v0.4.4 // indirect
github.com/nats-io/nats.go v1.33.1 // indirect
github.com/nats-io/nkeys v0.4.7 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/pelletier/go-toml/v2 v2.2.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_model v0.4.0 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.11.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.49.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/sagikazarmark/locafero v0.4.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/spf13/afero v1.9.5 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.6.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/volatiletech/inflect v0.0.1 // indirect
github.com/volatiletech/null v8.0.0+incompatible // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.16.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.25.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.25.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.25.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
go.opentelemetry.io/otel/metric v1.25.0 // indirect
go.opentelemetry.io/otel/trace v1.25.0 // indirect
go.opentelemetry.io/proto/otlp v1.2.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/arch v0.7.0 // indirect
golang.org/x/crypto v0.21.0 // indirect
golang.org/x/net v0.22.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/api v0.137.0 // indirect
google.golang.org/genproto v0.0.0-20230815205213-6bfd019c3878 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230815205213-6bfd019c3878 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230815205213-6bfd019c3878 // indirect
google.golang.org/grpc v1.57.0 // indirect
google.golang.org/api v0.168.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240415180920-8c6c420018be // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be // indirect
google.golang.org/grpc v1.63.2 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading
Loading