Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Update secret values (API only) #156806

Merged
merged 8 commits into from
May 16, 2023

Conversation

hop-dev
Copy link
Contributor

@hop-dev hop-dev commented May 5, 2023

Summary

Part of #154731

Allow secrets to be updated via the API. When a secret value is updated, the secret reference is replaced with a "raw" value we detect this on the API and create a new secret document.

Once a secret reference is updated, we clean up the old secret document if it is not in use by another policy. This check is a simple lookup of the secret_references array on policies.

API integration tests updated.

@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@hop-dev hop-dev changed the title Update secret values [Fleet] Update secret values (API only) May 10, 2023
@hop-dev hop-dev force-pushed the update-secret-values branch from 697986d to c31067c Compare May 12, 2023 16:22
@hop-dev hop-dev force-pushed the update-secret-values branch from c31067c to c6ded32 Compare May 13, 2023 07:05
@hop-dev hop-dev self-assigned this May 13, 2023
@hop-dev hop-dev added release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team labels May 13, 2023
@@ -234,6 +234,23 @@ export const validatePackagePolicyConfig = (
}
}

if (varDef.secret === true && parsedValue && parsedValue.isSecretRef === true) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validation updated here to accept a secret reference in place of a value, e.g mongo password is a password field, but once a policy is created we only have a secret reference, so we cannot validate the field in the normal way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the updated unit test above for an example

secretReferences: PolicySecretReference[];
secretsToDelete: PolicySecretReference[];
}> {
const { oldPackagePolicy, packagePolicyUpdate, packageInfo, esClient } = opts;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is the core of the update flow. We extract the secrets from theo ld policy and the new policy, giving us two arrays of secret values and their paths.

We then get the diff of these arrays, we need to know if there are updated secrets which need to be created (toCreate), secrets which are no longer used (toDelete) and unchanged secrets (noChange)

For example, I have a policy with a mongo password, the password is a secret reference with ID 1. We update the password to newValue.

We need toCreate a secret for newValue, toDelete the old secret with ID 1 (if it isn't used by any other policies), and noChange would be empty.

@hop-dev hop-dev marked this pull request as ready for review May 13, 2023 07:45
@hop-dev hop-dev requested a review from a team as a code owner May 13, 2023 07:45
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

);
});

it('should have correctly deleted the secrets', async () => {
Copy link
Contributor

@juliaElastic juliaElastic May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this test depend on the result of the previous? It would be better to put the secrets verification to the same testcase.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does depend on the result of the previous, I think I chose to do this because the previous test already has loads of assertions, and this test requires an ES call and further assertions.

@juliaElastic
Copy link
Contributor

Looks good on a high level, added a few comments.

@hop-dev hop-dev requested a review from juliaElastic May 15, 2023 16:10
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, haven't tested locally yet, can do that tomorrow.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 128.4KB 128.7KB +260.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 47 49 +2
securitySolution 400 404 +4
total +8

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 57 59 +2
securitySolution 480 484 +4
total +8

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @hop-dev @jillguyonnet

@hop-dev hop-dev merged commit 9d5c1cb into elastic:main May 16, 2023
@hop-dev hop-dev deleted the update-secret-values branch May 16, 2023 08:56
@kibanamachine kibanamachine added v8.9.0 backport:skip This commit does not require backporting labels May 16, 2023
jasonrhodes pushed a commit that referenced this pull request May 17, 2023
## Summary

Part of #154731

Allow secrets to be updated via the API. When a secret value is updated,
the secret reference is replaced with a "raw" value we detect this on
the API and create a new secret document.

Once a secret reference is updated, we clean up the old secret document
if it is not in use by another policy. This check is a simple lookup of
the secret_references array on policies.

API integration tests updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v8.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants