Skip to content

Commit

Permalink
Merge pull request #11 from mgvalverde/develop
Browse files Browse the repository at this point in the history
feat: add azure key vault
  • Loading branch information
mgvalverde authored Sep 23, 2024
2 parents bf15a0a + 8f5ce85 commit 70c6df3
Show file tree
Hide file tree
Showing 32 changed files with 626 additions and 122 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ assignees: ''
---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.
A clear and concise description of what the problem is.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,9 @@ jobs:
timeout_minutes: 5
max_attempts: 3
command: |
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ omegaconf_cloud_resolvers[aws,gcp]=="${TARGET_VERSION}"
python -c "import omegaconf_cloud_resolvers; print('Base import: OK')"
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ omegaconf-cloud-resolvers[aws,gcp,az]=="${TARGET_VERSION}" python -c "import omegaconf_cloud_resolvers; print('Base import: OK')"
python -c "from omegaconf_cloud_resolvers.resolvers.aws import AWSParameterStoreResolver; print('AWS import: OK')"
python -c "from omegaconf_cloud_resolvers.resolvers.az import AzureKeyVaultResolver; print('AZ import: OK')"
python -c "from omegaconf_cloud_resolvers.resolvers.gcp import GCPSecretManagerResolver; print('GCP import: OK')"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:

- name: Archive code coverage html report #
if: ${{ matrix.python-version == '3.9'}}
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: code-coverage-report
path: htmlcov
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ repos:
rev: 1.7.9
hooks:
- id: bandit
args: ["-r", "-x", "./.venv/*,./tests/*", "targets", "."]
args: [-r, --skip, "B101", -x, "./.venv/*,./tests/*,./example/*", "."] # skipping B101 temporally due to bug with -x

- repo: local
hooks:
Expand Down
18 changes: 13 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,24 @@ Currently, there are resolvers for:
* Parameter Store
* Google Cloud Platform (GCP):
* Secret Manager
* Microsoft Azure:
* Key Vault

## Installation

* AWS:
* AWS:
```
pip install omegaconf-cloud-resolvers[aws]
```

* GCP:
```
pip install omegaconf-cloud-resolvers[gcp]
```

* Azure:
```
pip install omegaconf-cloud-resolvers[az]
```

## Quickstart
Expand Down Expand Up @@ -60,8 +67,8 @@ print("Your secret is:", conf["secret"]) # THAT IS AN ILLUSTRATIVE EXAMPLE, NEV

## Roadmap

- [ ] Resolver for Azure Key Vault
- [ ] Support for older version for the AWS Secrets Manager Resolver
- [X] Resolver for Azure Key Vault
- [ ] Support for older secret version - AWS Secrets Manager Resolver
- [ ] Examples using AWS services - Lambda
- [ ] Examples using Google Cloud Platform services - Functions

Expand All @@ -72,5 +79,6 @@ This package is in a very early and experimental stage, use it under your own re

# Troubleshooting

* NoCredentialsError raised while resolving interpolation: Unable to locate credentials
You might not have configured a default profile or provided with a session to a AWS Resolver.

* **[AWS]** NoCredentialsError raised while resolving interpolation: Unable to locate credentials
You might not have configured a default profile or provided with a session to a AWS Resolver.
2 changes: 1 addition & 1 deletion docs/api/register.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Register Custom Resolvers

::: omegaconf_cloud_resolvers
::: omegaconf_cloud_resolvers
2 changes: 1 addition & 1 deletion docs/api/resolvers/aws/parameterstore.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ tags:
- AWS
---

# AWS Parameter Store
# AWS Parameter Store

::: omegaconf_cloud_resolvers.resolvers.aws.AWSParameterStoreResolver
options:
Expand Down
4 changes: 2 additions & 2 deletions docs/api/resolvers/aws/secretsmanager.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ tags:
- AWS
---

# AWS Secrets Manager
# AWS Secrets Manager

::: omegaconf_cloud_resolvers.resolvers.aws.AWSSecretsManagerResolver
::: omegaconf_cloud_resolvers.resolvers.aws.AWSSecretsManagerResolver
8 changes: 8 additions & 0 deletions docs/api/resolvers/az/keyvault.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
tags:
- AZ
---

# Azure Key Vault

::: omegaconf_cloud_resolvers.resolvers.az.AzureKeyVaultResolver
4 changes: 2 additions & 2 deletions docs/api/resolvers/gcp/secretmanager.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ tags:
- GCP
---

# GCP Secret Manager
# GCP Secret Manager

::: omegaconf_cloud_resolvers.resolvers.gcp.GCPSecretManagerResolver
::: omegaconf_cloud_resolvers.resolvers.gcp.GCPSecretManagerResolver
10 changes: 9 additions & 1 deletion docs/first/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,15 @@ pip install omegaconf-cloud-resolvers[aws]
pip install omegaconf-cloud-resolvers[gcp]
```

If you make use of `hydra` to load your configuration as well, it needs to be installed separately:
* For Azure

```bash
pip install omegaconf-cloud-resolvers[az]
```

# Hydra

If you make use of `hydra` to load your configuration as well, it needs to be installed separately:

```bash
pip install hydra-core
Expand Down
19 changes: 11 additions & 8 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sensitive values** that should not be hard-coded in your configuration files.

Having code in production is always challenging, and reports like the one provided by
[Unit42 from Palo Alto Networks](https://unit42.paloaltonetworks.com/large-scale-cloud-extortion-operation/)
are proof of them.
are proof of them.

This plugin is an attempt to mitigate this problem, letting you move your secrets from any file (like an `.env`)
to a more secure place, so they are retrieved only when needed.
Expand All @@ -17,13 +17,16 @@ The following diagram depicts the main idea in an AWS environment:
<img src="./img/diagram_welcome.png" alt="Welcome Diagram" width="400" />
</p>

It is built on top of OmegaConf, hence it can be also used together with [Hydra](https://hydra.cc/).
It is built on top of OmegaConf, hence it can be also used together with [Hydra](https://hydra.cc/).
To check out the examples using Hydra, go to [Integration with Hydra](./users-guide/integration-hydra.md)

Currently, the following cloud services are supported:

* AWS Secrets Manager
* AWS Parameter Store
* GCP Secret Manager
Currently, the following cloud services are supported:

# Collaborate
* AWS Secrets Manager
* AWS Parameter Store
* GCP Secret Manager

# Collaborate

If you thing something is missing or not working fine, just let us know
in [GitHub](https://github.com/mgvalverde/omegaconf-cloud-resolvers/issues).
8 changes: 4 additions & 4 deletions docs/users-guide/integration-hydra.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ If you are running this example from your local machine, ensure that you have co
You might need to install the [aws-cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
if you don't have it yet.

To verify that you have the necessary permissions to access the AWS SSM Parameter Store
To verify that you have the necessary permissions to access the AWS SSM Parameter Store
and to create and retrieve parameters, review
the [policies](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-access.html)
attached to your user's role. Ensure that your IAM user or role has the appropriate permissions.
Expand Down Expand Up @@ -61,7 +61,7 @@ import boto3
from omegaconf_cloud_resolvers import register_custom_resolvers
from omegaconf_cloud_resolvers.resolvers.aws import AWSParameterStoreResolver

session = boto3.Session(...) # Replace ... with you actual auth info, e.g: profile_name
session = boto3.Session(...) # Replace ... with you actual auth info, e.g: profile_name

resolvers = {
"get_aws_secret": AWSParameterStoreResolver(session=session),
Expand All @@ -85,7 +85,7 @@ if __name__ == "__main__":
Let's understand each line step by step.

1. **Define the resolver to retrieve the secret.**
Each key in the `resolvers` dictionary will be the name of the function that you can use in your configuration files.
Each key in the `resolvers` dictionary will be the name of the function that you can use in your configuration files.
The dictionary's value must be a callable, which in this case, it is, since all the cloud resolvers are
provided with a `__call__` method.

Expand Down Expand Up @@ -155,4 +155,4 @@ python main.py
```

By following these steps, you can integrate Hydra into your Python projects and leverage AWS services effectively
to retrieve your secrets and parameters securely.
to retrieve your secrets and parameters securely.
14 changes: 7 additions & 7 deletions docs/users-guide/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

## Introduction to OmegaConf

For those unfamiliar with OmegaConf, it is highly recommended to first review its official
[documentation](https://omegaconf.readthedocs.io/).
For those unfamiliar with OmegaConf, it is highly recommended to first review its official
[documentation](https://omegaconf.readthedocs.io/).
In brief, OmegaConf is a YAML-based configuration system that supports merging configurations from multiple sources.


One of OmegaConf's most compelling features is its ability to interpolate values within configuration files.
This is achieved through the use of [Resolvers](https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html#).
One of OmegaConf's most compelling features is its ability to interpolate values within configuration files.
This is achieved through the use of [Resolvers](https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html#).
The syntax for interpolation is: `"${<resolver-name>:<args>}"`.

Resolvers can be any type of function. Below is an example demonstrating how to register and use a custom resolver:
Expand All @@ -25,12 +25,12 @@ conf = OmegaConf.create({
print(conf["who"]) # 'WORLD'
```

Alternatively use `register_custom_resolvers`, which allows you to register
Alternatively use `register_custom_resolvers`, which allows you to register
many resolvers at once.
They key of the dictionary is the name of the resolver which you need to use
They key of the dictionary is the name of the resolver which you need to use
to interpolate.

```python
```python
from omegaconf import OmegaConf
from omegaconf_cloud_resolvers import register_custom_resolvers

Expand Down
Loading

0 comments on commit 70c6df3

Please sign in to comment.