Skip to content

Commit

Permalink
feat: add README
Browse files Browse the repository at this point in the history
  • Loading branch information
ovsds committed Feb 20, 2024
1 parent 9a9dea7 commit b90ef49
Show file tree
Hide file tree
Showing 14 changed files with 363 additions and 19 deletions.
319 changes: 318 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,323 @@
# Secret Transfer

YaML-based secret manager for secret load/set/transfer.
YaML-based secret manager for secret load/transfer from different sources to destinations.

## Usage

### Commands

#### Run

Run all transfers in the given YaML file.

```bash
secret-transfer run -f <path_to_yaml_file>
```

Options:

- `-f, --file <path_to_yaml_file>` - Path to the YaML file with secrets. [required]
- `-n, --name <name>` - Name of the transfer to run. [optional]

#### Clean

Clean all secrets in all transfer destinations in the given YaML file.

```bash
secret-transfer clean -f <path_to_yaml_file>
```

Options:

- `-f, --file <path_to_yaml_file>` - Path to the YaML file with secrets. [required]
- `-n, --name <name>` - Name of the transfer to clean. [optional]

### YaML schema

The YaML file should contain the following optional sections:

Custom class definitions for sources, destinations, collections, and transfers.

```yaml
source_classes:
<name>: # Name to be registered
module: <module> # Module to import from
class_name: <class_name> # Class to be registered
destination_classes: ...
collection_classes: ...
transfer_classes: ...
```
Sources, destinations, collections, and transfers.
```yaml
sources:
<name>:
class_name: <source_class_name> # Class to use, either from source_classes or built-in
init_args: # Arguments to pass to the class constructor, can be omitted if no arguments are needed
<arg_name>: <arg_value>
...
destinations:
...
collections:
...
transfers:
...
```
### Lasy loading
All items are lazily loaded, so they are not created until they are used. This allows to define objects in no particular order and reference them from one another. The only exception is built-in classes, which are registered at the start.
Cross-referencing is allowed, so you can reference sources, destinations, collections, transfers and their values from init_args. Circular references are not allowed.
### Cross-referencing
#### Instance references
To reference instances of sources, destinations, collections, and transfers, use the following syntax:
```yaml
$<type>[<name>]
```

Where:

- `<type>` - Type of the instance (source, destination, collection, or transfer).
- `<name>` - Name of the instance.

#### Value references

To reference values from sources or collections, use the following syntax:

```yaml
$<type>[<name>][<key>]
```

Where:

- `<type>` - Type of the gettable instance (usually, but not necessary source or collection).
- `<name>` - Name of the instance.
- `<key>` - Key of the value.

Where:

### Built-in classes

#### Sources

##### DotEnvSource

Load secrets from a .env file using `dotenv` package.
Arguments:

- `file_path` - Path to the .env file. [required]

Example:

```yaml
sources:
dotenv:
class_name: DotEnvSource
init_args:
file_path: .env
```
##### EnvSource
Load secrets from environment variables.
Arguments: none.
Already registered with name `env`, no need to define in the YaML file.

Example:

```yaml
collections:
default:
init_args:
TEST_KEY:
source: $sources[env]
```

##### PresetSource

Load secrets from a preset dictionary.
Arguments: key-value pairs of variables.

Example:

```yaml
sources:
preset:
class_name: PresetSource
init_args:
TEST_KEY: test_value
```

##### UserInputSource

Asks the user for input.
Arguments: none.

Already registered with name `user_input`, no need to define in the YaML file.

Example:

```yaml
collections:
default:
init_args:
TEST_KEY:
source: $sources[user_input]
```

##### VaultCLIKVSource

Load secrets from HashiCorp Vault KV using `vault kv get` CLI command.
Arguments:

- `address` - Address of the Vault server. [required]
- `mount` - Mount point of the KV engine. [required]
- `secret_name` - Name(path) of the secret. [required]

Pre-requisites:

- `vault` CLI installed and authenticated.

Example:

```yaml
sources:
vault:
class_name: VaultCLIKVSource
init_args:
address: https://vault.example.com
mount: secrets
secret_name: TEST_SECRET
```

##### YCCLILockboxSource

Load secrets from Yandex.Cloud Lockbox using `yc lockbox payload get` CLI command.
Arguments:

- `profile` - Name of the Yandex.Cloud CLI profile. [required]
- `folder` - Folder name. [required]
- `lockbox` - Lockbox name. [required]

Pre-requisites:

- `yc` CLI installed and authenticated.

Example:

```yaml
sources:
yc_lockbox:
class_name: YCCLILockboxSource
init_args:
profile: my-profile
folder: my-folder
lockbox: my-lockbox
```

#### Destinations

##### BashExportDestination

Print secrets as `export` commands to the console. Useful for setting environment variables. Never let stdout to be captured by a process, as it will expose the secrets.
Arguments: none.

Already registered with name `bash_export`, no need to define in the YaML file.

Example:

```yaml
transfers:
default:
init_args:
source: ...
destination: $destinations[bash_export]
```

##### EnvDestination

Set secrets as environment variables.
Arguments: none.

Already registered with name `env`, no need to define in the YaML file.

Example:

```yaml
transfers:
default:
init_args:
source: ...
destination: $destinations[env]
```

##### GithubCliSecretsDestination

Set secrets as GitHub repository secrets using `gh secret set` CLI command.
Arguments:

- `repo_name` - Name of the repository. [required]
- `owner_name` - Name of the repository owner. [required]
- `base_url` - Base URL of the GitHub API. [optional] (default: https://github.com)

Pre-requisites:

- `gh` CLI installed and authenticated.

Example:

```yaml
destinations:
github:
class_name: GithubCliSecretsDestination
init_args:
repo_name: my-repo
owner_name: my-org
```

#### Collections

##### DefaultCollection

Default collection to combine secrets from sources. Default collection class, so class_name can be omitted.

Example:

```yaml
collections:
default:
init_args:
COLLECTION_KEY:
source: $sources[env]
key: SOURCE_KEY
```

#### Transfers

##### DefaultTransfer

Default transfer to transfer secrets from collection to destination. Default transfer class, so class_name can be omitted.

Example:

```yaml
transfers:
default:
init_args:
collection: $collections[default]
destination: $destinations[env]
```

### Usage Examples

Check [examples](examples/README.md) for usage examples.

## Development

Expand Down
11 changes: 11 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Examples

## Basic

- [Transfer secrets from Lockbox to GitHub](basic/lockbox_to_GitHub/README.md)
- [Use secrets from Lockbox inside Python app](basic/lockbox_to_python_app/README.md)
- [Load secrets from Vault into terminal](basic/vault_to_terminal/README.md)

## Advanced

- [Import local files to use custom defined classes](advanced/import_local_files/README.md)
3 changes: 3 additions & 0 deletions examples/advanced/import_local_files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Import local files to use custom defined classes

This example demonstrates how to import local files to use custom defined classes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,21 @@ destination_classes:
class_name: CustomDestination

sources:
default: # use the custom source class
custom: # use the custom source class
class_name: CustomSource

destinations:
default: # use the custom destination class
custom: # use the custom destination class
class_name: CustomDestination

collections:
default:
init_args:
test_key:
source: $sources[default]
source: $sources[custom]

transfers:
default:
init_args:
collection: $collections[default]
destination: $destinations[default]
collection: $collections[custom]
destination: $destinations[custom]
3 changes: 3 additions & 0 deletions examples/basic/lockbox_to_github/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Transfer secrets from lockbox to GitHub

This example demonstrates how to transfer secrets from Yandex Cloud Lockbox to GitHub.
6 changes: 3 additions & 3 deletions examples/basic/lockbox_to_github/secrets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ sources:
lockbox:
class_name: YCCLILockboxSource
init_args:
profile: secret-transfer # name of the profile in ~/.config/yandex-cloud/config.yaml
folder: default # name of the folder in Yandex.Cloud Lockbox
lockbox: TEST_SECRET # name of the Lockbox secret
profile: secret-transfer
folder: default
lockbox: TEST_SECRET

destinations:
github:
Expand Down
3 changes: 3 additions & 0 deletions examples/basic/lockbox_to_python_app/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Use secrets from Lockbox inside Python app

This example demonstrates how to use secrets from Lockbox inside a Python app.
Loading

0 comments on commit b90ef49

Please sign in to comment.