Skip to content

Commit

Permalink
Introduce s3 config
Browse files Browse the repository at this point in the history
  • Loading branch information
jimirocks committed Apr 20, 2020
1 parent 9e41434 commit 49e5da5
Show file tree
Hide file tree
Showing 4 changed files with 307 additions and 1 deletion.
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,15 @@ Depends on following modules:
### Beginning with rclone
```puppet
class { 'rclone':
version => 'desired rclone version'
ensure => 'desired rclone version'
}
# configure s3
rclone::config::s3 { 'my_S3':
access_key_id => 'AKI...',
secret_access_key => '...',
os_user => 'my_user',
region => 'eu-west-1',
}
```

Expand Down
122 changes: 122 additions & 0 deletions REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ _Private Classes_
* `rclone::install`: Ensures Rclone installed
* `rclone::uninstall`: Removes rclone installed by this module

**Defined types**

* [`rclone::config::s3`](#rcloneconfigs3): S3 confguration for Rclone.

**Functions**

_Public Functions_
Expand Down Expand Up @@ -49,5 +53,123 @@ installed version, can be 'latest', 'absent' or valid version string

Default value: 'latest'

## Defined types

### rclone::config::s3

Ensures S3 Rclone configuration of given name and params. Include of `rclone` is required.
Currently only AWS provider is supported.

#### Examples

#####

```puppet
rclone::config::s3 { 's3_remote':
os_user => 'my_user',
access_key_id => 'SOME_ACCESS_KEY',
secret_access_key => 'SECRET_ACCESS_KEY',
region => 'us-east-1',
}
```

#### Parameters

The following parameters are available in the `rclone::config::s3` defined type.

##### `ensure`

Data type: `Enum['present', 'absent']`

configuration ensure

Default value: 'present'

##### `os_user`

Data type: `String`

operating system user - used to execute rclone commands, effective configuration owner

##### `config_name`

Data type: `String`

configuration name - should be unique among Rclone configurations, defaults to title

Default value: $title

##### `access_key_id`

Data type: `String`

S3 provider's access_key_id, maps to Rclone `access_key_id` property

##### `secret_access_key`

Data type: `String`

S3 provider's secret_access_key, maps to Rclone `secret_access_key` property

##### `region`

Data type: `String`

S3 provider's region, maps to Rclone `region` property

##### `s3_provider`

Data type: `Enum['AWS']`

S3 provider, maps to Rclone `provider` property

Default value: 'AWS'

##### `canned_acl`

Data type: `Optional[String]`

S3 canned ACL, maps to Rclone `acl` property

Default value: `undef`

##### `endpoint`

Data type: `Optional[String]`

S3 provider's endpoint, maps to Rclone `endpoint` property

Default value: `undef`

##### `location_constraint`

Data type: `Optional[String]`

S3 location_constraint, maps to Rclone `location_constraint` property

Default value: `undef`

##### `location_constraint`

S3 location_constraint, maps to Rclone `location_constraint` property

Default value: `undef`

##### `server_side_encryption`

Data type: `Optional[String]`

S3 server_side_encryption, maps to Rclone `server_side_encryption` property

Default value: `undef`

##### `storage_class`

Data type: `Optional[String]`

S3 storage_class, maps to Rclone `storage_class` property

Default value: `undef`

## Functions

97 changes: 97 additions & 0 deletions manifests/config/s3.pp
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# @summary S3 confguration for Rclone.
#
# Ensures S3 Rclone configuration of given name and params. Include of `rclone` is required.
# Currently only AWS provider is supported.
#
# @example
# rclone::config::s3 { 's3_remote':
# os_user => 'my_user',
# access_key_id => 'SOME_ACCESS_KEY',
# secret_access_key => 'SECRET_ACCESS_KEY',
# region => 'us-east-1',
# }
#
# @param ensure
# configuration ensure
# @param os_user
# operating system user - used to execute rclone commands, effective configuration owner
# @param config_name
# configuration name - should be unique among Rclone configurations, defaults to title
# @param access_key_id
# S3 provider's access_key_id, maps to Rclone `access_key_id` property
# @param secret_access_key
# S3 provider's secret_access_key, maps to Rclone `secret_access_key` property
# @param region
# S3 provider's region, maps to Rclone `region` property
# @param s3_provider
# S3 provider, maps to Rclone `provider` property
# @param canned_acl
# S3 canned ACL, maps to Rclone `acl` property
# @param endpoint
# S3 provider's endpoint, maps to Rclone `endpoint` property
# @param location_constraint
# S3 location_constraint, maps to Rclone `location_constraint` property
# @param location_constraint
# S3 location_constraint, maps to Rclone `location_constraint` property
# @param server_side_encryption
# S3 server_side_encryption, maps to Rclone `server_side_encryption` property
# @param storage_class
# S3 storage_class, maps to Rclone `storage_class` property
define rclone::config::s3 (
String $os_user,
String $access_key_id,
String $secret_access_key,
String $region,
Enum['present', 'absent'] $ensure = 'present',
String $config_name = $title,
Enum['AWS'] $s3_provider = 'AWS',
Optional[String] $canned_acl = undef,
Optional[String] $endpoint = undef,
Optional[String] $location_constraint = undef,
Optional[String] $server_side_encryption = undef,
Optional[String] $storage_class = undef,
) {

if ! defined(Class[rclone]) {
fail('You must include the rclone base class before using any defined resources')
}

$_rclone_exec_defaults = {
user => $os_user,
path => '/usr/bin',
require => Class[rclone],
}

case $ensure {
'present': {

$_options = {
acl => $canned_acl,
endpoint => $endpoint,
location_constraint => $location_constraint,
server_side_encryption => $server_side_encryption,
storage_class => $storage_class,
}
.filter |$key, $val| { $val != undef }.map |$key, $val| { "${key} ${val}" }.join(' ')

exec { default: *=> $_rclone_exec_defaults; "rclone create remote ${config_name} for user ${os_user}":
command => @("CMD")
rclone config create ${config_name} s3 \
provider ${s3_provider} env_auth false access_key_id ${access_key_id} secret_access_key ${secret_access_key} region ${region} \
${_options}
| CMD
}
}

'absent': {
exec { default: *=> $_rclone_exec_defaults; "rclone delete remote ${config_name} for user ${os_user}":
command => "rclone config delete ${config_name}",
}
}

default: {
fail("Invalid ensure value '${ensure}'")
}
}

}
79 changes: 79 additions & 0 deletions spec/defines/config/s3_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# frozen_string_literal: true

require 'spec_helper'

describe 'rclone::config::s3' do
let(:title) { 'test_r' }
let(:params) do
{
os_user: 'user',
access_key_id: 'aws_id',
secret_access_key: 'aws_key',
region: 'eu-west-1',
}
end

context 'on missing rclone main class' do
it { is_expected.to compile.and_raise_error(%r{include the rclone base class}) }
end

on_supported_os.each do |os, os_facts|
context "on #{os}" do
let(:facts) { os_facts }
let(:pre_condition) { 'include rclone' }

it { is_expected.to compile }

it {
is_expected.to contain_exec('rclone create remote test_r for user user')
.with(
command: 'rclone config create test_r s3 \\' \
"\nprovider AWS env_auth false access_key_id aws_id secret_access_key aws_key region eu-west-1 \\\n\n",
user: 'user',
path: '/usr/bin',
)
.that_requires('Class[rclone]')
}

context 'with optional params' do
let(:params) do
super().merge(
canned_acl: 'private',
endpoint: 'some-s3.com',
location_constraint: 'eu-west-1',
server_side_encryption: 'AES256',
storage_class: 'STANDARD',
)
end

it {
is_expected.to contain_exec('rclone create remote test_r for user user')
.with(
command: 'rclone config create test_r s3 \\' \
"\nprovider AWS env_auth false access_key_id aws_id secret_access_key aws_key region eu-west-1 \\" \
"\nacl private endpoint some-s3.com location_constraint eu-west-1 server_side_encryption AES256 storage_class STANDARD\n",
user: 'user',
path: '/usr/bin',
)
.that_requires('Class[rclone]')
}
end

context 'with ensure => absent' do
let(:params) do
super().merge(ensure: 'absent')
end

it {
is_expected.to contain_exec('rclone delete remote test_r for user user')
.with(
command: 'rclone config delete test_r',
user: 'user',
path: '/usr/bin',
)
.that_requires('Class[rclone]')
}
end
end
end
end

0 comments on commit 49e5da5

Please sign in to comment.