This is a Golang program to assist with the reporting of Service Account key ages, and rotating said keys once they pass a specific age threshold.
The tool can update keys held in the following locations:
- CircleCI
- GCS
- Git
- GoCd
- K8S (GKE only)
- SSM (AWS Parameter Store)
The tool is packaged as an executable file for native invocation, and as a zip file for deployment as an AWS Lambda.
Darwin, Linux and Windows Binaries can be downloaded from the Releases page.
Try it out:
$ cloud-key-rotator -h
An Alpine-based Docker image is available here.
cloud-key-rotator picks up details about which key(s) to rotate, and locations
to update with new keys, from config.
Check out examples for example config files. Viper is used as the config framework, so config can be stored as JSON, TOML, YAML or HCL.
For native invocation, the file needs to be called "config" (before whatever
extension you're using), and be present either in /etc/cloud-key-rotator/ or
in the same directory the binary runs in. For AWS Lambda invocation, the config needs
to be set as a plaintext secret in the AWS Secrets Manager, using a default key name
of "ckr-config".
You'll need to provide cloud-key-rotator with the means of authenticating into
any key provider that it'll be updating.
Authorisation is handled by the Default Credential Provider Chains for both GCP and AWS.
cloud-key-rotator can operate in two different modes:
- Rotation mode - in which keys are rotated; and
- Non-rotation mode - which only posts the ages of keys to the Datadog Metric API.
The boolean field RotationMode config controls the mode of operation.
You can set the age threshold to whatever you want in the config, using the
DefaultRotationAgeThresholdMins field in config, or you can override on a
per-service-account-basis with the RotationAgeThresholdMins field. Key ages
are always measured in minutes.
cloud-key-rotator will not attempt to rotate a key until it's passed the age
threshold you've set (either default or the key-specific). This allows you to
run the tool as frequently as you want without worrying about keys being rotated
excessively.
"Key locations" is the term used for the places where keys are stored, which will ultimately be updated with the new keys that are generated.
Currently, the following locations are supported:
- EnvVars in CircleCI
- GoCd
- GCS
- Secrets in GKE
- Files (encrypted via mantle which integrates with KMS) in Git
- SSM (AWS Parameter Store)
The tool attempts to verify its actions as much as possible and aborts immediately if it encounters an error. By design, the tool does not attempt to handle errors gracefully and continue, since this can lead to a "split-brain effect", with keys out-of-sync in various locations.
It should be quick to re-run the tool (with new keys being created) once issues have been resolved. Note that cloud providers usually limit the number of keys you can have attached to a Service Account at any one time, so it is worth bearing this in mind when re-running manually after seeing errors.
Only the first key of a Service Account is handled by cloud-key-rotator. If
it handled more than one key, it could lead to complications when updating
single sources multiple times.
The AccountKeyLocations section of config holds details of the places where the keys
are stored, e.g.:
"AccountKeyLocations": [{
"ServiceAccountName": "cloud-key-client-test",
"RotationAgeThresholdMins": 60,
"Git": {
"FilePath": "service-account.txt",
"OrgRepo": "ovotech/cloud-key-rotator",
"VerifyCircleCISuccess": true,
"CircleCIDeployJobName": "dummy_deploy_with_wait"
},
"CircleCI": [{
"UsernameProject": "ovotech/cloud-key-rotator",
"KeyEnvVar": "ENV_VAR_NAME"
}],
"K8s": [{
"Project": "my_project",
"Location": "europe-west2-b",
"ClusterName": "cluster_name",
"Namespace": "uat",
"SecretName": "key-rotate-test-secret",
"DataName": "my-key.json"
}]
}]cloud-key-rotator has integrations into GitHub and CircleCI, which allows it
not only to update those sources with the new key, but also to verify that a
deployment has been successful after committing to a GitHub repository. If that
verification isn't required, you can disable it using the VerifyCircleCISuccess
boolean.
For any Git key location, the whole process will be aborted
if there is no KmsKey value set. Unencrypted keys should never be committed
to a Git repository.
Commits to Git repositories are required to be GPG signed. In order to achieve this, you need to provide 4 things:
Usernameof the Git user commits will be made on behalf of, set in configEmailaddress of Git user, set in configArmouredKeyRing, aka GPG private key, stored in/etc/cloud-key-rotator/akr.ascPassphraseto the ArmouredKeyRing
e.g. along with the akr.asc file, you should set the following:
"AkrPass": "change_me",
"GitName": "git-name",
"GitEmail": "[email protected]",You may want to only include or exclude specific Service Accounts. This is
possible using AccountFilter.
E.g.:
"AccountFilter": {
"Mode": "include",
"Accounts": [{
"Provider": {
"Name": "gcp",
"Project": "my-project"
},
"ProviderAccounts": [
"cloud-key-client-test"
]
}]
}Mode field is either include or exclude. If you omit the AccountFilter,
the rotation process will fail.
Notice how the name of the Service Account is used. In the case of GCP, this is
everything preceding the @[project].iam.gserviceaccount.com string in the
Service Account's email address.
- Reduce keys to those of service accounts deemed to be valid (e.g. strip out user accounts if in rotation-mode)
- Filter keys to those deemed to be eligible (e.g. according to filtering rules configured by the user)
- For each eligible key:
- Create new key
- Update key locations
- Verify update has worked (where possible)
- Delete old key
Contributions are more than welcome. It should be straightforward plugging in integrations of new key providers and/or locations, so for that, or anything else, please branch or fork and raise a PR.