Merge remote-tracking branch 'origin/main' into nfi-cache-unit-test-r…

…esults

.changelog/17107.txt

@@ -0,0 +1,3 @@
+```release-note:breaking-change
+api: RaftLeaderTransfer now requires an id string. An empty string can be specified to keep the old behavior.
+```

.changelog/17936.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+acl: Add new `acl.tokens.dns` config field which specifies the token used implicitly during dns checks.
+```

.changelog/18322.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+catalog api: fixes a bug with catalog api where filter query parameter was not working correctly for the `/v1/catalog/services` endpoint
+```

.changelog/18324.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+api-gateway: add retry and timeout filters
+```

.changelog/18336.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+xds: Add a built-in Envoy extension that appends OpenTelemetry Access Logging (otel-access-logging) to the HTTP Connection Manager filter.
+```
+
+```release-note:feature
+xds: Add support for patching outbound listeners to the built-in Envoy External Authorization extension.
+```

.changelog/18381.txt

@@ -0,0 +1,6 @@
+```release-note:improvement
+checks: It is now possible to configure agent TCP checks to use TLS with
+optional server SNI and mutual authentication. To use TLS with a TCP check, the
+check must enable the `tcp_use_tls` boolean. By default the agent will use the
+TLS configuration in the `tls.default` stanza. 
+```

.changelog/18437.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+Inherit locality from services when registering sidecar proxies.
+```

.changelog/18439.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+Support custom watches on the Consul Controller framework.
+```

.changelog/18464.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+UI : Nodes list view was breaking for synthetic-nodes. Fix handles non existence of consul-version meta for node.
+```

.changelog/18504.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+dataplane: Allow getting bootstrap parameters when using V2 APIs
+```

.changelog/18558.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+check: prevent go routine leakage when existing Defercheck of same check id is not nil
+```

.changelog/18560.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+ui: Use Community verbiage
+```

.changelog/18573.txt

@@ -0,0 +1,3 @@
+```release-note:enhancement
+xds: Use downstream protocol when connecting to local app
+```

.changelog/18583.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+mesh: **(Enterprise only)** Adds rate limiting config to service-defaults
+```

.changelog/18584.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+Reduce the frequency of metric exports from Consul to HCP from every 10s to every 1m
+```

.changelog/18617.txt

@@ -0,0 +1,4 @@
+```release-note:improvement
+log: Currently consul logs files like this consul-{timestamp}.log. This change makes sure that there is always
+consul.log file with the latest logs in it.
+```

.changelog/18625.txt

@@ -0,0 +1,5 @@
+```release-note:improvement
+Adds flag -append-filename (which works on values version, dc, node and status) to consul snapshot save command.
+Adding the flag -append-filename version,dc,node,status will add consul version, consul datacenter, node name and leader/follower
+(status) in the file name given in the snapshot save command before the file extension.
+```

.changelog/18636.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: Fix issue where Envoy endpoints would not populate correctly after a snapshot restore.
+```

.changelog/18646.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+api-gateway: Add support for response header modifiers on http-route configuration entry
+```

.changelog/18667.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+api: Add support for listing ACL tokens by service name.
+```

.changelog/18668.txt

@@ -0,0 +1,3 @@
+```release-note:breaking-change
+audit-logging: **(Enterprise only)** allowing timestamp based filename only on rotation. initially the filename will be just file.json
+```

.changelog/18681.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+api: Fix `/v1/agent/self` not returning latest configuration 
+```

.changelog/18708.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+acl: Added ACL Templated policies to simplify getting the right ACL token.
+```
+
+```release-note:improvement
+cli: Added `-templated-policy`, `-templated-policy-file`, `-replace-templated-policy`, `-append-templated-policy`, `-replace-templated-policy-file`, `-append-templated-policy-file` and `-var` flags for creating or updating tokens/roles.
+```

.changelog/18719.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+acl: Add BindRule support for templated policies. Add new BindType: templated-policy and BindVar field for templated policy variables.
+```
+
+```release-note:feature
+cli: Add `bind-var` flag to `consul acl binding-rule` for templated policy variables.
+```

.changelog/18724.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+telemetry: emit consul version metric on a regular interval.
+```

.changelog/18742.txt

@@ -0,0 +1,8 @@
+```release-note:security
+Upgrade to use Go 1.20.8. This resolves CVEs
+[CVE-2023-39320](https://github.com/advisories/GHSA-rxv8-v965-v333) (`cmd/go`),
+[CVE-2023-39318](https://github.com/advisories/GHSA-vq7j-gx56-rxjh) (`html/template`),
+[CVE-2023-39319](https://github.com/advisories/GHSA-vv9m-32rr-3g55) (`html/template`),
+[CVE-2023-39321](https://github.com/advisories/GHSA-9v7r-x7cv-v437) (`crypto/tls`), and
+[CVE-2023-39322](https://github.com/advisories/GHSA-892h-r6cr-53g4) (`crypto/tls`)
+```

.changelog/18769.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+acl: Adds a new ACL rule for workload identities
+```

.changelog/18773.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+ca: Vault provider now cleans up the previous Vault issuer and key when generating a new leaf signing certificate [[GH-18779](https://github.com/hashicorp/consul/issues/18779)]
+```

.changelog/18797.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+command: Adds -since flag in consul debug command which internally calls hcdiag for debug information in the past.
+```

.changelog/18813.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+acl: Use templated policy to generate synthetic policies for tokens/roles with node and/or service identities
+```

.changelog/18816.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: Add `consul acl templated-policy` commands to read, list and preview templated policies. 
+```

.changelog/18831.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+gateways: Fix a bug where gateway to service mappings weren't being cleaned up properly when externally registered proxies were being deregistered.
+```

.changelog/18943.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+api: added `CheckRegisterOpts` to Agent API
+```

.changelog/18959.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+gateways: Fix a bug where a service in a peered datacenter could not access an external node service through a terminating gateway
+```

.changelog/18983.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+api: added `Token` field to `ServiceRegisterOpts` type in Agent API
+```

.changelog/18994.txt

@@ -0,0 +1,20 @@
+```release-note:feature
+# Catalog v2 feature preview
+This release provides the ability to preview Consul's v2 Catalog and Resource API if enabled. The new model supports
+multi-port application deployments with only a single Envoy proxy. Note that the v1 and v2 catalogs are not cross
+compatible, and not all Consul features are available within this v2 feature preview. See the [v2 Catalog and Resource
+API documentation](https://developer.hashicorp.com/consul/docs/architecture/v2) for more information. The v2 Catalog and
+Resources API should be considered a feature preview within this release and should not be used in production
+environments.
+
+### Limitations
+* The v2 catalog API feature preview does not support connections with client agents. As a result, it is only available for Kubernetes deployments, which use [Consul dataplanes](consul/docs/connect/dataplane) instead of client agents.
+* The v1 and v2 catalog APIs cannot run concurrently.
+* The Consul UI does not support multi-port services or the v2 catalog API in this release.
+* HCP Consul does not support multi-port services or the v2 catalog API in this release.
+
+[[Catalog resource controllers]](https://github.com/hashicorp/consul/tree/e6b724d06249d3e62cd75afe3ee6042ba1fd5415/internal/catalog/internal/controllers)
+[[Mesh resource controllers]](https://github.com/hashicorp/consul/tree/e6b724d06249d3e62cd75afe3ee6042ba1fd5415/internal/mesh/internal/controllers)
+[[Auth resource controllers]](https://github.com/hashicorp/consul/tree/e6b724d06249d3e62cd75afe3ee6042ba1fd5415/internal/auth/internal)
+[[V2 Protobufs]](https://github.com/hashicorp/consul/tree/e6b724d06249d3e62cd75afe3ee6042ba1fd5415/proto-public)
+```

.changelog/19031.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+api: add custom marshal/unmarshal for ServiceResolverConfigEntry.RequestTimeout so config entries that set this field can be read using the API.
+```

.changelog/19077.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+acl: Adds workload identity templated policy
+```

.changelog/19095.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+ca: ensure Vault CA provider respects Vault Enterprise namespace configuration.
+```

.changelog/19218.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+resource: lowercase names enforced for v2 resources only.
+```

.changelog/19225.txt

@@ -0,0 +1,9 @@
+```release-note:security
+Upgrade Go to 1.20.10.
+This resolves vulnerability [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
+/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487)(`net/http`).
+```
+```release-note:security
+Update `golang.org/x/net` to v0.17.0 to address [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
+/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487)(`x/net/http2`).
+```

.changelog/19268.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+Mesh Gateways: Fix a bug where replicated and peered mesh gateways with hostname-based WAN addresses fail to initialize.
+```

.changelog/19285.txt

@@ -0,0 +1,7 @@
+```release-note:bug
+ca: Fix bug with Vault CA provider where token renewal goroutines could leak if CA failed to initialize.
+```
+
+```release-note:bug
+ca: Fix bug with Vault CA provider where renewing a retracted token would cause retries in a tight loop, degrading performance.
+```

.changelog/19306.txt

@@ -0,0 +1,3 @@
+```release-note:security
+connect: update supported envoy versions to 1.24.12, 1.25.11, 1.26.6, 1.27.2 to address [CVE-2023-44487](https://github.com/envoyproxy/envoy/security/advisories/GHSA-jhv4-f7mr-xx76)
+```

.changelog/19311.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+raft:  Fix panic during downgrade from enterprise to oss.
+```

.changelog/19314.txt

@@ -0,0 +1,3 @@
+```release-note:enhancement
+raft: upgrade raft-wal library version to 0.4.1.
+```

.changelog/19339.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+connect: Fix bug where uncleanly closed xDS connections would influence connection balancing for too long and prevent envoy instances from starting. Two new configuration fields
+`performance.grpc_keepalive_timeout` and `performance.grpc_keepalive_interval` now exist to allow for configuration on how often these dead connections will be cleaned up.
+```

.changelog/19389.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+cli: stop simultaneous usage of -templated-policy and -templated-policy-file when creating a role or token.
+```

.changelog/19414.txt

@@ -0,0 +1,4 @@
+```release-note:security
+Upgrade `google.golang.org/grpc` to 1.56.3.
+This resolves vulnerability [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487).
+```

.changelog/_18366.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+config-entry(api-gateway): (Enterprise only) Add GatewayPolicy to APIGateway Config Entry listeners
+```

.changelog/_18422.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+config-entry(api-gateway): (Enterprise only) Add JWTFilter to HTTPRoute Filters
+```

.changelog/_6074.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: **(Enterprise only)** Fix bug where incorrect service-defaults entries were fetched to determine an upstream's protocol whenever the upstream did not explicitly define the namespace / partition. When this bug occurs, upstreams would use the protocol from a service-default entry in the default namespace / partition, rather than their own namespace / partition.
+```

.changelog/_6870.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+gateway: **(Enterprise only)** Add JWT authentication and authorization to APIGateway Listeners and HTTPRoutes.
+```

.changelog/_7406.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+server: **(Enterprise Only)** Fixed an issue where snake case keys were rejected when configuring the control-plane-request-limit config entry
+```

.copywrite.hcl

@@ -1,8 +1,8 @@
 schema_version = 1
 
 project {
-  license        = "MPL-2.0"
-  copyright_year = 2013
+  license        = "BUSL-1.1"
+  copyright_year = 2023
 
   # (OPTIONAL) A list of globs that should not have copyright/license headers.
   # Supports doublestar glob patterns for more flexibility in defining which
@@ -19,11 +19,21 @@ project {
 
     # ignore specific test data files
     "agent/uiserver/testdata/**",
+    "internal/resourcehcl/testdata/**",
 
     # generated files 
     "agent/structs/structs.deepcopy.go",
     "agent/proxycfg/proxycfg.deepcopy.go",
     "agent/grpc-middleware/rate_limit_mappings.gen.go",
     "agent/uiserver/dist/**",
+
+    # ignoring policy embedded files
+    "agent/structs/acltemplatedpolicy/policies/ce/**",
+
+    # licensed under MPL - ignoring for now until the copywrite tool can support
+    # multiple licenses per repo.
+    "sdk/**",
+    "api/**",
+    "proto-public/**",
   ]
 }

.github/ISSUE_TEMPLATE/config.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 blank_issues_enabled: false
 contact_links:

.github/dependabot.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 version: 2
 updates:

.github/pr-labeler.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 pr/dependencies:
     - vendor/**/*

.github/scripts/changelog_checker.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 
 set -euo pipefail

.github/scripts/filter_changed_files_go_test.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+# Copyright (c) HashiCorp, Inc.
+# SPDX-License-Identifier: BUSL-1.1
+
+set -euo pipefail
+
+# Get the list of changed files
+# Using `git merge-base` ensures that we're always comparing against the correct branch point. 
+#For example, given the commits:
+#
+# A---B---C---D---W---X---Y---Z # origin/main
+#             \---E---F         # feature/branch
+#
+# ... `git merge-base origin/$SKIP_CHECK_BRANCH HEAD` would return commit `D`
+# `...HEAD` specifies from the common ancestor to the latest commit on the current branch (HEAD)..
+files_to_check=$(git diff --name-only "$(git merge-base origin/$SKIP_CHECK_BRANCH HEAD~)"...HEAD)
+
+# Define the directories to check
+skipped_directories=("docs/" "ui/" "website/" "grafana/")
+
+# Loop through the changed files and find directories/files outside the skipped ones
+for file_to_check in "${files_to_check[@]}"; do
+	file_is_skipped=false
+	for dir in "${skipped_directories[@]}"; do
+		if [[ "$file_to_check" == "$dir"* ]] || [[ "$file_to_check" == *.md && "$dir" == *"/" ]]; then
+			file_is_skipped=true
+			break
+		fi
+	done
+	if [ "$file_is_skipped" != "true" ]; then
+		echo -e $file_to_check
+        SKIP_CI=false
+		echo "Changes detected in non-documentation files - skip-ci: $SKIP_CI"
+        echo "skip-ci=$SKIP_CI" >> "$GITHUB_OUTPUT"
+		exit 0 ## if file is outside of the skipped_directory exit script
+	fi
+done
+
+echo -e "$files_to_check"
+SKIP_CI=true
+echo "Changes detected in only documentation files - skip-ci: $SKIP_CI"
+echo "skip-ci=$SKIP_CI" >> "$GITHUB_OUTPUT"