Merge pull request #32 from SumoLogic/upstream-changes

Upstream changes

CHANGELOG.md

@@ -1,18 +1,31 @@
 ## v1.16.0 [unreleased]
+## v1.16.1 [2020-10-28]
+
+#### Release Notes
+
+  - [#8318](https://github.com/influxdata/telegraf/pull/8318) `common.kafka` kafka sasl-mechanism auth support for SCRAM-SHA-256, SCRAM-SHA-512, GSSAPI
+
+#### Bugfixes
+
+  - [#8331](https://github.com/influxdata/telegraf/pull/8331) `inputs.sqlserver` SQL Server Azure PerfCounters Fix
+  - [#8325](https://github.com/influxdata/telegraf/pull/8325) `inputs.sqlserver` SQL Server - PerformanceCounters - removed synthetic counters
+  - [#8324](https://github.com/influxdata/telegraf/pull/8324) `inputs.sqlserver` SQL Server - server_properties added sql_version_desc
+  - [#8317](https://github.com/influxdata/telegraf/pull/8317) `inputs.ras` Disable RAS input plugin on specific Linux architectures: mips64, mips64le, ppc64le, riscv64
+  - [#8309](https://github.com/influxdata/telegraf/pull/8309) `inputs.processes` processes: fix issue with stat no such file/dir
+  - [#8308](https://github.com/influxdata/telegraf/pull/8308) `inputs.win_perf_counters` fix issue with PDH_CALC_NEGATIVE_DENOMINATOR error
+  - [#8306](https://github.com/influxdata/telegraf/pull/8306) `inputs.ras` RAS plugin - fix for too many open files handlers
+
+
+## v1.16.0 [2020-10-21]
 
 #### Release Notes
 
-  - Many documentation updates
   - New [code examples](/plugins/processors/starlark/testdata) for the [Starlark processor](/plugins/processors/starlark/README.md)
-  - [#8220](https://github.com/influxdata/telegraf/pull/8220) `build` update to go 1.15
-  - [#7864](https://github.com/influxdata/telegraf/pull/7864) `processors.starlark` Add logic starlark example
-  - [#7865](https://github.com/influxdata/telegraf/pull/7865) `common.shim` shim logger improvements
   - [#7920](https://github.com/influxdata/telegraf/pull/7920) `inputs.rabbitmq` remove deprecated healthcheck
-  - [#7932](https://github.com/influxdata/telegraf/pull/7932) Support for AWS Cloudwatch Alarms #7931
   - [#7953](https://github.com/influxdata/telegraf/pull/7953) Add details to connect to InfluxDB OSS 2 and Cloud 2
-  - [#7980](https://github.com/influxdata/telegraf/pull/7980) `processors.starlark` add example input/outputs to starlark examples
   - [#8054](https://github.com/influxdata/telegraf/pull/8054) add guidelines run to external plugins with execd
   - [#8198](https://github.com/influxdata/telegraf/pull/8198) `inputs.influxdb_v2_listener` change default influxdb port from 9999 to 8086 to match OSS 2.0 release
+  - [starlark](https://github.com/influxdata/telegraf/tree/release-1.16/plugins/processors/starlark/testdata) `processors.starlark` add various code exampels for the Starlark processor
 
 #### Features
 
@@ -37,13 +50,17 @@
   - [#8189](https://github.com/influxdata/telegraf/pull/8189) `inputs.snmp_trap` If the community string is available, add it as a tag
   - [#8190](https://github.com/influxdata/telegraf/pull/8190) `inputs.tail` Semigroupoid multiline (#8167)
   - [#8196](https://github.com/influxdata/telegraf/pull/8196) `inputs.redis` add functionality to get values from redis commands
+  - [#8220](https://github.com/influxdata/telegraf/pull/8220) `build` update to Go 1.15
+  - [#8032](https://github.com/influxdata/telegraf/pull/8032) `inputs.http_response` http_response: match on status code
+  - [#8172](https://github.com/influxdata/telegraf/pull/8172) `inputs.sqlserver` New sql server queries (on-prem) - refactoring and formatting
 
 #### Bugfixes
 
   - [#7816](https://github.com/influxdata/telegraf/pull/7816) `shim` fix bug with loading plugins in shim with no config
   - [#7818](https://github.com/influxdata/telegraf/pull/7818) `build` Fix darwin package build flags
   - [#7819](https://github.com/influxdata/telegraf/pull/7819) `inputs.tail` Close file to ensure it has been flushed
   - [#7853](https://github.com/influxdata/telegraf/pull/7853) Initialize aggregation processors
+  - [#7865](https://github.com/influxdata/telegraf/pull/7865) `common.shim` shim logger improvements
   - [#7867](https://github.com/influxdata/telegraf/pull/7867) `inputs.execd` fix issue with execd restart_delay being ignored
   - [#7872](https://github.com/influxdata/telegraf/pull/7872) `inputs.gnmi` Recv next message after send returns EOF
   - [#7877](https://github.com/influxdata/telegraf/pull/7877) Fix arch name in deb/rpm builds
@@ -64,6 +81,8 @@
   - [#8210](https://github.com/influxdata/telegraf/pull/8210) update gopsutil: fix procstat performance regression
   - [#8162](https://github.com/influxdata/telegraf/pull/8162) Fix bool serialization when using carbon2
   - [#8240](https://github.com/influxdata/telegraf/pull/8240) Fix bugs found by LGTM analysis platform
+  - [#8251](https://github.com/influxdata/telegraf/pull/8251) `outputs.dynatrace` Dynatrace Output Plugin: Fixed behaviour when state map is cleared
+  - [#8274](https://github.com/influxdata/telegraf/pull/8274) `common.shim` fix issue with loading processor config from execd
 
 #### New Input Plugins
 
@@ -79,6 +98,7 @@
 
   - [dynatrace](/plugins/outputs/dynatrace/README.md) Dynatrace output plugin - Contributed by @thschue
   - [sumologic](/plugins/outputs/sumologic/README.md) Sumo Logic output plugin - Contributed by @pmalek-sumo
+  - [timestream](/plugins/outputs/timestream) Timestream Output Plugin - Contributed by @piotrwest
 
 #### New External Plugins
 
@@ -87,6 +107,14 @@
   - [awsalarms](https://github.com/vipinvkmenon/awsalarms) - Simple plugin to gather/monitor alarms generated  in AWS.
   - [youtube-telegraf-plugin](https://github.com/inabagumi/youtube-telegraf-plugin) - Gather view and subscriber stats from your youtube videos
   - [octoprint](https://github.com/BattleBas/octoprint-telegraf-plugin) - Gather 3d print information from the octoprint API.
+  - [systemd-timings](https://github.com/pdmorrow/telegraf-execd-systemd-timings) - Gather systemd boot and unit timestamp metrics.
+
+## v1.15.4 [2020-10-20]
+
+#### Bugfixes
+
+  - [#8274](https://github.com/influxdata/telegraf/pull/8274) `common.shim` fix issue with loading processor config from execd
+  - [#8176](https://github.com/influxdata/telegraf/pull/8176) `agent` fix panic on streaming processers using logging
 
 ## v1.15.3 [2020-09-11]
 

EXTERNAL_PLUGINS.md

@@ -12,3 +12,4 @@ Pull requests welcome.
 - [youtube](https://github.com/inabagumi/youtube-telegraf-plugin) - Gather account information from YouTube channels
 - [awsalarms](https://github.com/vipinvkmenon/awsalarms) - Simple plugin to gather/monitor alarms generated  in AWS.
 - [octoprint](https://github.com/BattleBas/octoprint-telegraf-plugin) - Gather 3d print information from the octoprint API.
+- [systemd-timings](https://github.com/pdmorrow/telegraf-execd-systemd-timings) - Gather systemd boot and unit timestamp metrics.

Makefile

@@ -1,7 +1,8 @@
-next_version := 1.16.0
+next_version :=  $(shell cat build_version.txt)
 tag := $(shell git describe --exact-match --tags 2>git_describe_error.tmp; rm -f git_describe_error.tmp)
 branch := $(shell git rev-parse --abbrev-ref HEAD)
 commit := $(shell git rev-parse --short=8 HEAD)
+glibc_version := 2.17
 
 ifdef NIGHTLY
 	version := $(next_version)
@@ -40,7 +41,7 @@ GOOS ?= $(shell go env GOOS)
 GOARCH ?= $(shell go env GOARCH)
 HOSTGO := env -u GOOS -u GOARCH -u GOARM -- go
 
-LDFLAGS := $(LDFLAGS) -X main.commit=$(commit) -X main.branch=$(branch)
+LDFLAGS := $(LDFLAGS) -X main.commit=$(commit) -X main.branch=$(branch) -X main.goos=$(GOOS) -X main.goarch=$(GOARCH)
 ifneq ($(tag),)
 	LDFLAGS += -X main.version=$(version)
 endif
@@ -228,6 +229,7 @@ install: $(buildbin)
 	@if [ $(GOOS) != "windows" ]; then cp -fv etc/telegraf.conf $(DESTDIR)$(sysconfdir)/telegraf/telegraf.conf$(conf_suffix); fi
 	@if [ $(GOOS) != "windows" ]; then cp -fv etc/logrotate.d/telegraf $(DESTDIR)$(sysconfdir)/logrotate.d; fi
 	@if [ $(GOOS) = "windows" ]; then cp -fv etc/telegraf_windows.conf $(DESTDIR)/telegraf.conf; fi
+	@if [ $(GOOS) = "linux" ]; then scripts/check-dynamic-glibc-versions.sh $(buildbin) $(glibc_version); fi
 	@if [ $(GOOS) = "linux" ]; then mkdir -pv $(DESTDIR)$(prefix)/lib/telegraf/scripts; fi
 	@if [ $(GOOS) = "linux" ]; then cp -fv scripts/telegraf.service $(DESTDIR)$(prefix)/lib/telegraf/scripts; fi
 	@if [ $(GOOS) = "linux" ]; then cp -fv scripts/init.sh $(DESTDIR)$(prefix)/lib/telegraf/scripts; fi

README.md

@@ -444,6 +444,7 @@ For documentation on the latest development code see the [documentation index][d
 * [instrumental](./plugins/outputs/instrumental)
 * [kafka](./plugins/outputs/kafka)
 * [librato](./plugins/outputs/librato)
+* [logz.io](./plugins/outputs/logzio)
 * [mqtt](./plugins/outputs/mqtt)
 * [nats](./plugins/outputs/nats)
 * [newrelic](./plugins/outputs/newrelic)

build_version.txt

@@ -0,0 +1 @@
+1.16.1

config/aws/credentials.go

@@ -29,8 +29,10 @@ func (c *CredentialConfig) Credentials() client.ConfigProvider {
 
 func (c *CredentialConfig) rootCredentials() client.ConfigProvider {
 	config := &aws.Config{
-		Region:   aws.String(c.Region),
-		Endpoint: &c.EndpointURL,
+		Region: aws.String(c.Region),
+	}
+	if c.EndpointURL != "" {
+		config.Endpoint = &c.EndpointURL
 	}
 	if c.AccessKey != "" || c.SecretKey != "" {
 		config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, c.Token)

config/config_test.go

@@ -12,6 +12,7 @@ import (
 	"github.com/influxdata/telegraf/plugins/inputs/http_listener_v2"
 	"github.com/influxdata/telegraf/plugins/inputs/memcached"
 	"github.com/influxdata/telegraf/plugins/inputs/procstat"
+	"github.com/influxdata/telegraf/plugins/outputs/azure_monitor"
 	httpOut "github.com/influxdata/telegraf/plugins/outputs/http"
 	"github.com/influxdata/telegraf/plugins/parsers"
 	"github.com/stretchr/testify/assert"
@@ -257,3 +258,23 @@ func TestConfig_BadOrdering(t *testing.T) {
 	require.Error(t, err, "bad ordering")
 	assert.Equal(t, "Error loading config file ./testdata/non_slice_slice.toml: Error parsing http array, line 4: cannot unmarshal TOML array into string (need slice)", err.Error())
 }
+
+func TestConfig_AzureMonitorNamespacePrefix(t *testing.T) {
+	// #8256 Cannot use empty string as the namespace prefix
+	c := NewConfig()
+	defaultPrefixConfig := `[[outputs.azure_monitor]]`
+	err := c.LoadConfigData([]byte(defaultPrefixConfig))
+	assert.NoError(t, err)
+	azureMonitor, ok := c.Outputs[0].Output.(*azure_monitor.AzureMonitor)
+	assert.Equal(t, "Telegraf/", azureMonitor.NamespacePrefix)
+	assert.Equal(t, true, ok)
+
+	c = NewConfig()
+	customPrefixConfig := `[[outputs.azure_monitor]]
+	namespace_prefix = ""`
+	err = c.LoadConfigData([]byte(customPrefixConfig))
+	assert.NoError(t, err)
+	azureMonitor, ok = c.Outputs[0].Output.(*azure_monitor.AzureMonitor)
+	assert.Equal(t, "", azureMonitor.NamespacePrefix)
+	assert.Equal(t, true, ok)
+}

docs/INPUTS.md

@@ -38,7 +38,8 @@ import (
 )
 
 type Simple struct {
-    Ok bool `toml:"ok"`
+    Ok  bool            `toml:"ok"`
+    Log telegraf.Logger `toml:"-"`
 }
 
 func (s *Simple) Description() string {
@@ -52,6 +53,7 @@ func (s *Simple) SampleConfig() string {
 `
 }
 
+// Init is for setup, and validating config.
 func (s *Simple) Init() error {
 	return nil
 }

docs/LICENSE_OF_DEPENDENCIES.md

@@ -94,7 +94,7 @@ following works:
 - github.com/kubernetes/apimachinery [Apache License 2.0](https://github.com/kubernetes/apimachinery/blob/master/LICENSE)
 - github.com/leodido/ragel-machinery [MIT License](https://github.com/leodido/ragel-machinery/blob/develop/LICENSE)
 - github.com/mailru/easyjson [MIT License](https://github.com/mailru/easyjson/blob/master/LICENSE)
-- github.com/mattn/go-sqlite3 [MIT License](https://github.com/mattn/go-sqlite3/blob/master/LICENSE)
+- github.com/mattn/go-isatty [MIT License](https://github.com/mattn/go-isatty/blob/master/LICENSE)
 - github.com/matttproud/golang_protobuf_extensions [Apache License 2.0](https://github.com/matttproud/golang_protobuf_extensions/blob/master/LICENSE)
 - github.com/mdlayher/apcupsd [MIT License](https://github.com/mdlayher/apcupsd/blob/master/LICENSE.md)
 - github.com/mdlayher/genetlink [MIT License](https://github.com/mdlayher/genetlink/blob/master/LICENSE.md)
@@ -141,6 +141,8 @@ following works:
 - github.com/wavefronthq/wavefront-sdk-go [Apache License 2.0](https://github.com/wavefrontHQ/wavefront-sdk-go/blob/master/LICENSE)
 - github.com/wvanbergen/kafka [MIT License](https://github.com/wvanbergen/kafka/blob/master/LICENSE)
 - github.com/wvanbergen/kazoo-go [MIT License](https://github.com/wvanbergen/kazoo-go/blob/master/MIT-LICENSE)
+- github.com/xdg/scram [Apache License 2.0](https://github.com/xdg-go/scram/blob/master/LICENSE)
+- github.com/xdg/stringprep [Apache License 2.0](https://github.com/xdg-go/stringprep/blob/master/LICENSE)
 - github.com/yuin/gopher-lua [MIT License](https://github.com/yuin/gopher-lua/blob/master/LICENSE)
 - go.opencensus.io [Apache License 2.0](https://github.com/census-instrumentation/opencensus-go/blob/master/LICENSE)
 - go.starlark.net [BSD 3-Clause "New" or "Revised" License](https://github.com/google/starlark-go/blob/master/LICENSE)
@@ -171,6 +173,10 @@ following works:
 - gopkg.in/olivere/elastic.v5 [MIT License](https://github.com/olivere/elastic/blob/v5.0.76/LICENSE)
 - gopkg.in/tomb.v1 [BSD 3-Clause Clear License](https://github.com/go-tomb/tomb/blob/v1/LICENSE)
 - gopkg.in/yaml.v2 [Apache License 2.0](https://github.com/go-yaml/yaml/blob/v2.2.2/LICENSE)
+- gopkg.in/yaml.v3 [Apache License 2.0](https://github.com/go-yaml/yaml/blob/v3/LICENSE)
+- modernc.org/libc [BSD 3-Clause "New" or "Revised" License](https://gitlab.com/cznic/libc/-/blob/master/LICENSE)
+- modernc.org/memory [BSD 3-Clause "New" or "Revised" License](https://gitlab.com/cznic/memory/-/blob/master/LICENSE)
+- modernc.org/sqlite [BSD 3-Clause "New" or "Revised" License](https://gitlab.com/cznic/sqlite/-/blob/master/LICENSE)
 
 ## telegraf used and modified code from these projects
 - github.com/DataDog/datadog-agent [Apache License 2.0](https://github.com/DataDog/datadog-agent/LICENSE)

docs/OUTPUTS.md

@@ -30,7 +30,8 @@ import (
 )
 
 type Simple struct {
-    Ok bool `toml:"ok"`
+    Ok  bool            `toml:"ok"`
+    Log telegraf.Logger `toml:"-"`
 }
 
 func (s *Simple) Description() string {
@@ -43,20 +44,25 @@ func (s *Simple) SampleConfig() string {
 `
 }
 
+// Init is for setup, and validating config.
 func (s *Simple) Init() error {
 	return nil
 }
 
 func (s *Simple) Connect() error {
-    // Make a connection to the URL here
+    // Make any connection required here
     return nil
 }
 
 func (s *Simple) Close() error {
-    // Close connection to the URL here
+    // Close any connections here.
+    // Write will not be called once Close is called, so there is no need to synchronize.
     return nil
 }
 
+// Write should write immediately to the output, and not buffer writes
+// (Telegraf manages the buffer for you). Returning an error will fail this
+// batch of writes and the entire batch will be retried automatically.
 func (s *Simple) Write(metrics []telegraf.Metric) error {
     for _, metric := range metrics {
         // write `metric` to the output sink here
@@ -102,9 +108,9 @@ Metrics are flushed to outputs when any of the following events happen:
 - The telegraf process has received a SIGUSR1 signal
 
 Note that if the flush takes longer than the `agent.interval` to write the metrics
-to the output, you'll see a message saying the output `did not complete within its 
-flush interval`. This may mean your output is not keeping up with the flow of metrics, 
-and you may want to look into enabling compression, reducing the size of your metrics, 
+to the output, you'll see a message saying the output `did not complete within its
+flush interval`. This may mean your output is not keeping up with the flow of metrics,
+and you may want to look into enabling compression, reducing the size of your metrics,
 or investigate other reasons why the writes might be taking longer than expected.
 
 [file]: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/file

docs/PROCESSORS.md

@@ -33,6 +33,7 @@ import (
 )
 
 type Printer struct {
+	Log telegraf.Logger `toml:"-"`
 }
 
 var sampleConfig = `
@@ -46,6 +47,7 @@ func (p *Printer) Description() string {
 	return "Print all metrics that pass through this filter."
 }
 
+// Init is for setup, and validating config.
 func (p *Printer) Init() error {
 	return nil
 }
@@ -97,6 +99,7 @@ import (
 )
 
 type Printer struct {
+	Log telegraf.Logger `toml:"-"`
 }
 
 var sampleConfig = `
@@ -110,13 +113,27 @@ func (p *Printer) Description() string {
 	return "Print all metrics that pass through this filter."
 }
 
+// Init is for setup, and validating config.
 func (p *Printer) Init() error {
 	return nil
 }
 
+// Start is called once when the plugin starts; it is only called once per
+// plugin instance, and never in parallel.
+// Start should return once it is ready to receive metrics.
+// The passed in accumulator is the same as the one passed to Add(), so you
+// can choose to save it in the plugin, or use the one received from Add().
 func (p *Printer) Start(acc telegraf.Accumulator) error {
 }
 
+// Add is called for each metric to be processed. The Add() function does not
+// need to wait for the metric to be processed before returning, and it may
+// be acceptable to let background goroutine(s) handle the processing if you
+// have slow processing you need to do in parallel.
+// Keep in mind Add() should not spawn unbounded goroutines, so you may need
+// to use a semaphore or pool of workers (eg: reverse_dns plugin does this).
+// Metrics you don't want to pass downstream should have metric.Drop() called,
+// rather than simply omitting the acc.AddMetric() call
 func (p *Printer) Add(metric telegraf.Metric, acc telegraf.Accumulator) error {
 	// print!
 	fmt.Println(metric.String())
@@ -127,6 +144,12 @@ func (p *Printer) Add(metric telegraf.Metric, acc telegraf.Accumulator) error {
 	return nil
 }
 
+// Stop gives you an opportunity to gracefully shut down the processor.
+// Once Stop() is called, Add() will not be called any more. If you are using
+// goroutines, you should wait for any in-progress metrics to be processed
+// before returning from Stop().
+// When stop returns, you should no longer be writing metrics to the
+// accumulator.
 func (p *Printer) Stop() error {
 }
 

etc/telegraf.conf

@@ -1328,16 +1328,13 @@
 #   ## Timeout used for HTTP request
 #   # timeout = "5s"
 #
-#   ## HTTP method, one of: "POST" or "PUT". "POST" is used by default if unset.
-#   # method = "POST"
-#
 #   ## Max HTTP request body size in bytes before compression (if applied).
 #   ## By default 1MB is recommended.
 #   ## NOTE:
 #   ## Bear in mind that in some serializer a metric even though serialized to multiple
 #   ## lines cannot be split any further so setting this very low might not work
 #   ## as expected.
-#   # max_request_body_size = 1_000_000
+#   # max_request_body_size = 1000000
 #
 #   ## Additional, Sumo specific options.
 #   ## Full list can be found here: