Merge remote-tracking branch 'upstream/main' into race-cond-check-pro…

…gress

changelog/8.10.4.asciidoc

@@ -0,0 +1,55 @@
+// begin 8.10.4 relnotes
+
+[[release-notes-8.10.4]]
+==  8.10.4
+
+Review important information about the  8.10.4 release.
+
+
+
+[discrete]
+[[breaking-changes-8.10.4]]
+=== Breaking changes
+
+Breaking changes can prevent your application from optimal operation and
+performance. Before you upgrade, review the breaking changes, then mitigate the
+impact to your application.
+
+elastic-agent::
+
+[discrete]
+[[breaking-3591]]
+.`elastic-agent-autodiscover` library has been updated to version 0.6.4, disabling metadata For `kubernetes.deployment` and `kubernetes.cronjob` fields.
+[%collapsible]
+====
+*Details* +
+The `elastic-agent-autodiscover` Kubernetes library by default comes with `add_resource_metadata.deployment=false` and `add_resource_metadata.cronjob=false`.
+*Impact* +
+Pods that will be created from deployments or cronjobs will not have the extra metadata field for `kubernetes.deployment` or `kubernetes.cronjob`, respectively. This change was made to avoid the memory impact of keeping the feature enabled in big Kubernetes clusters.
+For more information, refer to {agent-pull}3591[#3591].
+====
+
+
+
+
+
+[discrete]
+[[new-features-8.10.4]]
+=== New features
+
+The 8.10.4 release adds the following new and notable features.
+
+
+elastic-agent::
+
+* Secondary Fallback For Package Signature Verification. {elastic-agent-pull}https://github.com/elastic/elastic-agent/pull/3453[#https://github.com/elastic/elastic-agent/pull/3453] {elastic-agent-issue}https://github.com/elastic/elastic-agent/issues/3264[#https://github.com/elastic/elastic-agent/issues/3264]
++
+Ability to upgrade securely in air-gapped environment where fleet server is the only reachable URI.
+
+
+
+
+
+
+
+// end 8.10.4 relnotes

changelog/8.10.4.yaml

@@ -0,0 +1,25 @@
+version: 8.10.4
+entries:
+    - kind: feature
+      summary: Secondary fallback for package signature verification
+      description: Ability to upgrade securely in air-gapped environment where fleet server is the only reachable URI.
+      component: elastic-agent
+      pr:
+        - https://github.com/elastic/elastic-agent/pull/3453
+      issue:
+        - https://github.com/elastic/elastic-agent/issues/3264
+      timestamp: 1695289867
+      file:
+        name: 1695289867-Secondary-fallback-for-package-signature-verification.yaml
+        checksum: 8f8c39d9eef2f5b6922353bcab9c4ee1b74b1378
+    - kind: breaking-change
+      summary: Elastic-agent-autodiscover to v0.6.4. Disables metadata for deployment and cronjob
+      description: Elastic-agent-autodiscover library by default comes with add_resource_metadata.deployment=false and add_resource_metadata.cronjob=false. Pods that will be created from deployments or cronjobs will not have the extra metadata field for kubernetes.deployment or kubernetes.cronjob respectively.
+      component: elastic-agent
+      pr:
+        - https://github.com/elastic/elastic-agent/pull/3591
+      issue: []
+      timestamp: 1697102363
+      file:
+        name: 1697102363-updating_agentautodiscovery_810.yaml
+        checksum: fe9015185dc4d3fe85f9c2ebf9f47e64e26fc67d

changelog/fragments/1695289867-Secondary-fallback-for-package-signature-verification.yaml → changelog/fragments/1689328899-Elastic-Agent-container-runs-on-Azure-Container-Instances-.yaml

@@ -8,14 +8,14 @@
 # - security: impacts on the security of a product or a user’s deployment.
 # - upgrade: important information for someone upgrading from a prior version
 # - other: does not fit into any of the other categories
-kind: feature
+kind: bug
 
 # Change summary; a 80ish characters long description of the change.
-summary: Secondary fallback for package signature verification
+summary: Elastic-Agent container runs on Azure Container Instances 
 
 # Long description; in case the summary is not enough to describe the change
 # this field accommodate a description without length limits.
-description: Ability to upgrade securely in Air gapped environment where fleet server is the only reachable URI.
+#description: 
 
 # Affected component; a word indicating the component this changeset affects.
 component: elastic-agent
@@ -24,8 +24,8 @@ component: elastic-agent
 # If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
 # NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
 # Please provide it if you are adding a fragment for a different PR.
-pr: https://github.com/elastic/elastic-agent/pull/3453
+pr: 3576
 
 # Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
 # If not present is automatically filled by the tooling with the issue linked to the PR number.
-issue: https://github.com/elastic/elastic-agent/issues/3264
+issue: 82

changelog/fragments/1695920792-Prevent-multiple-stops-of-services.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: bug-fix
+
+# Change summary; a 80ish characters long description of the change.
+summary: Prevent multiple attempts to stop an already stopped service
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component: runtime
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+#pr: https://github.com/owner/repo/1234
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue: https://github.com/owner/repo/1234

changelog/fragments/1696319263-add-assetbeat-dependency.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Add assetbeat among the external dependencies needed to package Elastic Agent
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
+component: elastic-agent
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+#pr: https://github.com/owner/repo/1234
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: https://github.com/elastic/obs-infraobs-team/issues/1114

changelog/fragments/1696935756-add-default-host.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Hints Autodiscovery for Elastic Agent - Add default host for each container in a pod
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
+component: elastic-agent
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: https://github.com/elastic/elastic-agent/pull/3575
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: https://github.com/elastic/elastic-agent/issues/1453

changelog/fragments/1697554456-Improve-upgrade-watcher-to-no-longer-need-root-access.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Improve upgrade watcher to no longer need root access
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
+component:
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: https://github.com/elastic/elastic-agent/pull/3622
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue: https://github.com/owner/repo/1234

changelog/fragments/1697662109-upgrade-elastic-agent-libs-to-v0.6.0.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: bug-fix
+
+# Change summary; a 80ish characters long description of the change.
+summary: upgrade elastic-agent-libs to v0.6.0
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+description: 'allows elastic-agent running as a windows service to receive more than one change request.'
+
+# Affected component; a word indicating the component this changeset affects.
+component: agent
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+#pr: https://github.com/owner/repo/1234
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue: https://github.com/owner/repo/1234

control_v2.proto

@@ -166,6 +166,8 @@ message StateAgentInfo {
   string buildTime = 4;
   // Current running version is a snapshot.
   bool snapshot = 5;
+  // Current running PID.
+  int32 pid = 6;
 }
 
 // StateResponse is the current state of Elastic Agent.
@@ -214,7 +216,7 @@ enum AdditionalDiagnosticRequest {
   CPU = 0;
 }
 
-// DiagnosticComponentsRequest is the message to request diagnostics from individual components. 
+// DiagnosticComponentsRequest is the message to request diagnostics from individual components.
 message DiagnosticComponentsRequest {
   repeated DiagnosticComponentRequest components  = 1;
   repeated AdditionalDiagnosticRequest additional_metrics = 2;

deploy/kubernetes/Makefile

@@ -1,7 +1,9 @@
-ALL=elastic-agent-standalone elastic-agent-managed
+SHELL := /bin/bash
 
-BEAT_VERSION=$(shell grep '^:stack-version:' ../../version/docs/version.asciidoc | cut -c 17- )
-BRANCH_VERSION=$(shell grep '^:doc-branch:' ../../version/docs/version.asciidoc | cut -c 14- )
+ALL=elastic-agent-standalone elastic-agent-managed
+WORKSPACE=$(shell git rev-parse --show-toplevel)
+BEAT_VERSION=$(shell grep -oE '[0-9]+\.[0-9]+\.[0-9]+(\-[a-zA-Z]+[0-9]+)?' "${WORKSPACE}/version/version.go")
+BRANCH_VERSION=$(shell cut -d. -f1-2 <<< '${BEAT_VERSION}')
 
 #variables needed for ci-create-kubernetes-templates-pull-request
 ELASTIC_AGENT_REPO=kibana

deploy/kubernetes/elastic-agent-managed-kubernetes.yaml

@@ -30,7 +30,7 @@ spec:
       dnsPolicy: ClusterFirstWithHostNet
       containers:
         - name: elastic-agent
-          image: docker.elastic.co/beats/elastic-agent:8.8.1
+          image: docker.elastic.co/beats/elastic-agent:8.12.0
           env:
             # Set to 1 for enrollment into Fleet server. If not set, Elastic Agent is run in standalone mode
             - name: FLEET_ENROLL

deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml

@@ -683,13 +683,13 @@ spec:
       #      - -c
       #      - >-
       #        mkdir -p /etc/elastic-agent/inputs.d &&
-      #        wget -O - https://github.com/elastic/elastic-agent/archive/main.tar.gz | tar xz -C /etc/elastic-agent/inputs.d --strip=5 "elastic-agent-main/deploy/kubernetes/elastic-agent-standalone/templates.d"
+      #        wget -O - https://github.com/elastic/elastic-agent/archive/8.12.tar.gz | tar xz -C /etc/elastic-agent/inputs.d --strip=5 "elastic-agent-8.12/deploy/kubernetes/elastic-agent-standalone/templates.d"
       #    volumeMounts:
       #      - name: external-inputs
       #        mountPath: /etc/elastic-agent/inputs.d
       containers:
         - name: elastic-agent-standalone
-          image: docker.elastic.co/beats/elastic-agent:8.8.1
+          image: docker.elastic.co/beats/elastic-agent:8.12.0
           args: ["-c", "/etc/elastic-agent/agent.yml", "-e"]
           env:
             # The basic authentication username used to connect to Elasticsearch

dev-tools/mage/manifest/manifest.go

@@ -87,6 +87,7 @@ func DownloadComponentsFromManifest(manifest string, platforms []string, platfor
 		"beats":                 {"auditbeat", "filebeat", "heartbeat", "metricbeat", "osquerybeat", "packetbeat"},
 		"cloud-defend":          {"cloud-defend"},
 		"cloudbeat":             {"cloudbeat"},
+		"assetbeat":             {"assetbeat"},
 		"elastic-agent-shipper": {"elastic-agent-shipper"},
 		"endpoint-dev":          {"endpoint-security"},
 		"fleet-server":          {"fleet-server"},