docs: Prepare Changelog for 8.16.0 (#41617) (#41621)

* docs: Close changelog for 8.16.0 * Update CHANGELOG.asciidoc --------- Co-authored-by: elasticmachine <[email protected]> Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit 96968c9) Co-authored-by: elastic-vault-github-plugin-prod[bot] <150874479+elastic-vault-github-plugin-prod[bot]@users.noreply.github.com> Co-authored-by: Pierre HILBERT <[email protected]>
elastic · Nov 18, 2024 · 0be3f94 · 0be3f94
1 parent d1e9e8e
commit 0be3f94
Show file tree

Hide file tree

Showing 3 changed files with 96 additions and 38 deletions.
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
@@ -3,6 +3,98 @@
 :issue: https://github.com/elastic/beats/issues/
 :pull: https://github.com/elastic/beats/pull/
 
+[[release-notes-8.16.0]]
+=== Beats version 8.16.0
+https://github.com/elastic/beats/compare/v8.15.4\...v8.16.0[View commits]
+
+==== Breaking changes
+
+*Affecting all Beats*
+
+- Fix FQDN being lowercased when used as `host.hostname`. {issue}39993[39993]
+- Beats won't log start up information when running under the Elastic Agent. {pull}40390[40390]
+- Filebeat now needs `dup3`, `faccessat2`, `prctl` and `setrlimit` syscalls to run the journald input. If this input is not being used, the syscalls are not needed. All Beats have those syscalls allowed now because the default seccomp policy is global to all Beats. {pull}40061[40061]
+- Beats will rate limit the logs about errors when indexing events on Elasticsearch, logging a summary every 10s. The logs sent to the event log is unchanged. {issue}40157[40157]
+
+*Filebeat*
+
+- Filebeat, when running with Elastic-Agent, reports status for Filestream input. {pull}40121[40121]
+- Added support for hyphens in extension keys in `decode_cef` Filebeat processor. {pull}40427[40427]
+- Journald: removed configuration options `include_matches.or`, `include_matches.and`, `backoff`, `max_backoff`, `cursor_seek_fallback`. {pull}40061[40061]
+- Journald: `include_matches.match` now behaves in the same way as matchers in `journalctl`. Users should carefully update their input configuration. {pull}40061[40061]
+- Journald: `seek` and `since` behaviour have been simplified, if there is a cursor (state) `seek` and `since` are ignored and the cursor is used. {pull}40061[40061]
+- Redis: Added replication role as a field to submitted slowlogs.
+- Added `container.image.name` to `journald` Filebeat input's Docker-specific translated fields. {pull}40450[40450]
+- Remove deprecated awscloudwatch field from Filebeat. {pull}41089[41089]
+- The performance of ingesting SQS data with the S3 input has improved by up to 60x for queues with many small events. `max_number_of_messages` config for SQS mode is now ignored, as the new design no longer needs a manual cap on messages. Instead, use `number_of_workers` to scale ingestion rate in both S3 and SQS modes. The increased efficiency may increase network bandwidth consumption, which can be throttled by lowering `number_of_workers`. It may also increase number of events stored in memory, which can be throttled by lowering the configured size of the internal queue. {pull}40699[40699]
+
+*Metricbeat*
+
+- Add support for specifying a custom endpoint for GCP service clients. {issue}40848[40848] {pull}40918[40918]
+
+==== Bugfixes
+
+*Auditbeat*
+
+- Request status from a separate socket to avoid data congestion. {pull}41207[41207]
+
+*Filebeat*
+
+- Fix crashes in the journald input. {pull}40061[40061]
+- Fix long filepaths in diagnostics exceeding max path limits on Windows. {pull}40909[40909]
+- Fix a bug in Salesforce input to only handle responses with 200 status code. {pull}41015[41015]
+- Fixed failed job handling and removed false-positive error logs in the GCS input. {pull}41142[41142]
+- Bump github.com/elastic/go-sfdc dependency used by x-pack/filebeat/input/salesforce. {pull}41192[41192]
+- Journald input now can read events from all boots {issue}41083[41083] {pull}41244[41244]
+- Fix errors in SQS host resolution in the `aws-s3` input when using custom (non-AWS) endpoints. {pull}41504[41504]
+
+*Metricbeat*
+
+- Add GCP 'instance_id' resource label in ECS cloud fields. {issue}40033[40033] {pull}40062[40062]
+- Remove excessive info-level logs in cgroups setup. {pull}40491[40491]
+- Fix http server helper SSL config. {pull}39405[39405]
+
+==== Added
+
+*Filebeat*
+
+- Implement Elastic Agent status and health reporting for Netflow Filebeat input. {pull}40080[40080]
+- Add SSL and username support for Redis input, now the input includes support for Redis 6.0+. {pull}40111[40111]
+- Add scaling up support for Netflow input. {issue}37761[37761] {pull}40122[40122]
+- Update CEL mito extensions to v1.15.0. {pull}40294[40294]
+- Improve logging in Okta Entity Analytics provider. {issue}40106[40106] {pull}40347[40347]
+- Document `winlog` input. {issue}40074[40074] {pull}40462[40462]
+- Added retry logic to websocket connections in the streaming input. {issue}40271[40271] {pull}40601[40601]
+- Disable event normalization for netflow input. {pull}40635[40635]
+- Allow attribute selection in the Active Directory entity analytics provider. {issue}40482[40482] {pull}40662[40662]
+- Improve error quality when CEL program does not correctly return an events array. {pull}40580[40580]
+- Added support for Microsoft Entra ID RBAC authentication. {issue}40434[40434] {pull}40879[40879]
+- Add `use_kubeadm` config option for filebeat (both filbeat.input and autodiscovery) in order to toggle kubeadm-config api requests. {pull}40301[40301]
+- Make HTTP library function inclusion non-conditional in CEL input. {pull}40912[40912]
+- Add support for Crowdstrike streaming API to the streaming input. {issue}40264[40264] {pull}40838[40838]
+- Add support to CEL for reading host environment variables. {issue}40762[40762] {pull}40779[40779]
+- Add CSV decoder to awss3 input. {pull}40896[40896]
+- Change request trace logging to include headers instead of complete request. {pull}41072[41072]
+- Improved GCS input documentation. {pull}41143[41143]
+- Add CSV decoding capacity to azureblobstorage input. {pull}40978[40978]
+- Add CSV decoding capacity to gcs input. {pull}40979[40979]
+- Add support to source AWS cloudwatch logs from linked accounts. {pull}41188[41188]
+- Jounrald input now supports filtering by facilities. {pull}41061[41061]
+- Add support to include AWS cloudwatch linked accounts when using log_group_name_prefix to define log group names. {pull}41206[41206]
+
+*Heartbeat*
+
+- Add journey duration to synthetics browser events. {pull}40230[40230]
+
+*Metricbeat*
+
+- Add new metrics fot datastore and minor changes to overall vSphere metrics. {pull}40766[40766]
+- Add new metricset datastorecluster for vSphere module. {pull}40634[40634] {pull}40694[40694]
+- Add AWS Cloudwatch capability to retrieve tags from AWS/ApiGateway resources. {pull}40755[40755]
+- Add new metrics for the vSphere Virtualmachine metricset. {pull}40485[40485]
+- Add `metrics_count` to Prometheus module if `metrics_count: true` is set. {pull}40411[40411]
+
+
 [[release-notes-8.15.4]]
 === Beats version 8.15.4
 https://github.com/elastic/beats/compare/v8.15.3\...v8.15.4[View commits]

diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc
@@ -58,23 +58,12 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 *Metricbeat*
 
 - Setting period for counter cache for Prometheus remote_write at least to 60sec {pull}38553[38553]
-- Add support of Graphite series 1.1.0+ tagging extension for statsd module. {pull}39619[39619]
-- Allow metricsets to report their status via control v2 protocol. {pull}40025[40025]
 - Remove fallback to the node limit for the `kubernetes.pod.cpu.usage.limit.pct` and `kubernetes.pod.memory.usage.limit.pct` metrics calculation
-- Add support for Kibana status metricset in v8 format {pull}40275[40275]
-- Add new metrics for the vSphere Datastore metricset. {pull}40441[40441]
-- Update metrics for the vSphere Host metricset. {pull}40429[40429]
-- Mark system process metricsets as running if metrics are partially available {pull}40565[40565]
-- Added back `elasticsearch.node.stats.jvm.mem.pools.*` to the `node_stats` metricset {pull}40571[40571]
-- Add support for snapshot in vSphere virtualmachine metricset {pull}40683[40683]
-- Add support for specifying a custom endpoint for GCP service clients. {issue}40848[40848] {pull}40918[40918]
 
 
 *Osquerybeat*
 
 - Add action responses data stream, allowing osquerybeat to post action results directly to elasticsearch. {pull}39143[39143]
-- Disable allow_unsafe osquery configuration. {pull}40130[40130]
-- Upgrade to osquery 5.12.1. {pull}40368[40368]
 
 *Osquerybeat*
 
@@ -118,7 +107,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 
 *Auditbeat*
 
-- Request status from a separate socket to avoid data congestion {pull}41207[41207]
 
 *Filebeat*
 
@@ -146,10 +134,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Updated Websocket input title to align with existing inputs {pull}39006[39006]
 - Restore netflow input on Windows {pull}39024[39024]
 - Upgrade azure-event-hubs-go and azure-storage-blob-go dependencies. {pull}38861[38861]
-- Fix concurrency/error handling bugs in the AWS S3 input that could drop data and prevent ingestion of large buckets. {pull}39131[39131]
-- Fix EntraID query handling. {issue}39419[39419] {pull}39420[39420]
 - Fix request trace filename handling in http_endpoint input. {pull}39410[39410]
-- Fix filestream not correctly tracking the offset of a file when using the `include_message` parser. {pull}39873[39873] {issue}39653[39653]
 - Upgrade github.com/hashicorp/go-retryablehttp to mitigate CVE-2024-6104 {pull}40036[40036]
 - Fix for Google Workspace duplicate events issue by adding canonical sorting over fingerprint keys array to maintain key order. {pull}40055[40055] {issue}39859[39859]
 - Fix handling of deeply nested numeric values in HTTP Endpoint CEL programs. {pull}40115[40115]
@@ -177,8 +162,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Journald input now can read events from all boots {issue}41083[41083] {pull}41244[41244]
 - Fix double encoding of client_secret in the Entity Analytics input's Azure Active Directory provider {pull}41393[41393]
 - Fix aws region in aws-s3 input s3 polling mode.  {pull}41572[41572]
-- Fix errors in SQS host resolution in the `aws-s3` input when using custom (non-AWS) endpoints. {pull}41504[41504]
-- The azure-eventhub input now correctly reports its status to the Elastic Agent on fatal errors {pull}41469[41469]
 
 *Heartbeat*
 
@@ -194,8 +177,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Fix handling of access errors when reading process metrics {pull}39627[39627]
 - Fix behavior of cgroups path discovery when monitoring the host system from within a container {pull}39627[39627]
 - Fix issue where beats may report incorrect metrics for its own process when running inside a container {pull}39627[39627]
-- Fix for MySQL/Performance - Query failure for MySQL versions below v8.0.1, for performance metric `quantile_95`. {pull}38710[38710]
-- Fix Prometheus helper text parser to store each metric family type. {pull}39743[39743]
 - Normalize AWS RDS CPU Utilization values before making the metadata API call. {pull}39664[39664]
 - Fix behavior of pagetypeinfo metrics {pull}39985[39985]
 - Fix query logic for temp and non-temp tablespaces in Oracle module. {issue}38051[38051] {pull}39787[39787]
@@ -213,7 +194,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Fix Kubernetes metadata sometimes not being present after startup {pull}41216[41216]
 - Log Cisco Meraki `getDevicePerformanceScores` errors without stopping metrics collection. {pull}41622[41622]
 
-
 *Osquerybeat*
 
 
@@ -287,24 +267,8 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Parse more fields from Elasticsearch slowlogs {pull}38295[38295]
 - added benchmark input {pull}37437[37437]
 - added benchmark input and discard output {pull}37437[37437]
-- Ensure all responses sent by HTTP Endpoint are HTML-escaped. {pull}39329[39329]
 - Update CEL mito extensions to v1.11.0 to improve type checking. {pull}39460[39460]
-- Improve logging of request and response with request trace logging in error conditions. {pull}39455[39455]
-- Implement Elastic Agent status and health reporting for CEL Filebeat input. {pull}39209[39209]
-- Add HTTP metrics to CEL input. {issue}39501[39501] {pull}39503[39503]
-- Add default user-agent to CEL HTTP requests. {issue}39502[39502] {pull}39587[39587]
-- Improve reindexing support in security module pipelines. {issue}38224[38224] {pull}39588[39588]
-- Make HTTP Endpoint input GA. {issue}38979[38979] {pull}39410[39410]
 - Update CEL mito extensions to v1.12.2. {pull}39755[39755]
-- Add support for base64-encoded HMAC headers to HTTP Endpoint. {pull}39655[39655]
-- Add user group membership support to Okta entity analytics provider. {issue}39814[39814] {pull}39815[39815]
-- Add request trace support for Okta and EntraID entity analytics providers. {pull}39821[39821]
-- Fix handling of infinite rate values in CEL rate limit handling logic. {pull}39940[39940]
-- Allow elision of set and append failure logging. {issue}34544[34544] {pull}39929[39929]
-- Add ability to remove request trace logs from CEL input. {pull}39969[39969]
-- Add ability to remove request trace logs from HTTPJSON input. {pull}40003[40003]
-- Update CEL mito extensions to v1.13.0. {pull}40035[40035]
-- Add Jamf entity analytics provider. {pull}39996[39996]
 - Add ability to remove request trace logs from http_endpoint input. {pull}40005[40005]
 - Add ability to remove request trace logs from entityanalytics input. {pull}40004[40004]
 - Relax constraint on Base DN in entity analytics Active Directory provider. {pull}40054[40054]
@@ -351,8 +315,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 
 - Added status to monitor run log report.
 - Upgrade node to latest LTS v18.20.3. {pull}40038[40038]
-- Add journey duration to synthetics browser events. {pull}40230[40230]
-- Add monitor status reporter under managed mode. {pull}41077[41077]
 
 *Metricbeat*
 
@@ -459,3 +421,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 
 
 
+
+
+
diff --git a/libbeat/docs/release.asciidoc b/libbeat/docs/release.asciidoc
@@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read
 <<breaking-changes>> for more detail about changes that affect
 upgrade.
 
+* <<release-notes-8.16.0>>
 * <<release-notes-8.15.4>>
 * <<release-notes-8.15.3>>
 * <<release-notes-8.15.2>>