Skip to content

Commit

Permalink
Merge pull request #12 from statsbomb/region_from_ec2imds
Browse files Browse the repository at this point in the history
Allow for explicitly setting the region and add EC2 IMDS as a fallback
  • Loading branch information
joshm91 authored Dec 2, 2021
2 parents 026f8f0 + d544de0 commit d429063
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 9 deletions.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ $ ./prometheus-msk-discovery -scrape-interval 10s -filter 'primary'

An example output file can be found [here](examples/msk_file_sd.yml)

## Region Precedence
When no region is specified with the `-region` flag the process first attempts to load the default SDK configuration checking for an `AWS_REGION` environment variable or reading any region specified in the standard [configuration file](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html). If no region is found it will attempt to retrieve it from the EC2 Instance Metadata Service.

**Explicitly setting a region with the** `-region` **flag takes precedence over everything else.**

## Integration with kube-prometheus-stack

The [Docker image](https://hub.docker.com/r/statsbomb/prometheus-msk-discovery) for this project can be used to inject a container into the Prometheus Spec of a [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) installation by using the following snippet in `values.yaml`:
Expand Down
21 changes: 12 additions & 9 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,18 @@ import (
"gopkg.in/yaml.v2"
)

const jmxExporterPort = 11001
const nodeExporterPort = 11002
const (
jmxExporterPort = 11001
nodeExporterPort = 11002
)

var outFile = flag.String("output", "msk_file_sd.yml", "path of the file to write MSK discovery information to")
var interval = flag.Duration("scrape-interval", 5*time.Minute, "interval at which to scrape the AWS API for MSK cluster information")
var jobPrefix = flag.String("job-prefix", "msk", "string with which to prefix each job label")
var clusterFilter = flag.String("filter", "", "a regex pattern to filter cluster names from the results")
var (
outFile = flag.String("output", "msk_file_sd.yml", "path of the file to write MSK discovery information to")
interval = flag.Duration("scrape-interval", 5*time.Minute, "interval at which to scrape the AWS API for MSK cluster information")
jobPrefix = flag.String("job-prefix", "msk", "string with which to prefix each job label")
clusterFilter = flag.String("filter", "", "a regex pattern to filter cluster names from the results")
awsRegion = flag.String("region", "", "the aws region in which to scan for MSK clusters")
)

type kafkaClient interface {
ListClusters(ctx context.Context, params *kafka.ListClustersInput, optFns ...func(*kafka.Options)) (*kafka.ListClustersOutput, error)
Expand Down Expand Up @@ -171,7 +176,7 @@ func GetStaticConfigs(svc kafkaClient, opt_filter ...regexp.Regexp) ([]Prometheu
func main() {
flag.Parse()

cfg, err := config.LoadDefaultConfig(context.TODO())
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion(*awsRegion), config.WithEC2IMDSRegion())
if err != nil {
fmt.Println(err)
return
Expand All @@ -180,7 +185,6 @@ func main() {
client := kafka.NewFromConfig(cfg)

work := func() {

regexpFilter, err := regexp.Compile(*clusterFilter)
if err != nil {
fmt.Println(err)
Expand Down Expand Up @@ -216,5 +220,4 @@ func main() {
}
work()
}

}

0 comments on commit d429063

Please sign in to comment.