diff --git a/.fixtures.yml b/.fixtures.yml index a05b1f1..ba511ad 100644 --- a/.fixtures.yml +++ b/.fixtures.yml @@ -4,3 +4,4 @@ fixtures: forge_modules: archive: "puppet/archive" + stdlib: "puppetlabs/stdlib" diff --git a/README.md b/README.md index 34ca748..a08870d 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,15 @@ Depends on following modules: ### Beginning with rclone ```puppet class { 'rclone': - version => 'desired rclone version' + ensure => 'desired rclone version' +} + +# configure s3 +rclone::config::s3 { 'my_S3': + access_key_id => 'AKI...', + secret_access_key => '...', + os_user => 'my_user', + region => 'eu-west-1', } ``` diff --git a/REFERENCE.md b/REFERENCE.md index cf2c252..744db14 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -14,6 +14,12 @@ _Private Classes_ * `rclone::install`: Ensures Rclone installed * `rclone::uninstall`: Removes rclone installed by this module +**Defined types** + +* [`rclone::config`](#rcloneconfig): General configuration for Rclone. +* [`rclone::config::gdrive`](#rcloneconfiggdrive): Google Drive configuration for Rclone. +* [`rclone::config::s3`](#rcloneconfigs3): S3 configuration for Rclone. + **Functions** _Public Functions_ @@ -49,5 +55,298 @@ installed version, can be 'latest', 'absent' or valid version string Default value: 'latest' +## Defined types + +### rclone::config + +Ensures Rclone configuration of given name, type and params. Include of `rclone` is required. + +#### Examples + +##### + +```puppet +rclone::config { 'my_remote': + os_user => 'my_user', + type => 'ftp', + options => {...} +} +``` + +#### Parameters + +The following parameters are available in the `rclone::config` defined type. + +##### `ensure` + +Data type: `Enum['present', 'absent']` + +configuration ensure + +Default value: 'present' + +##### `os_user` + +Data type: `String` + +operating system user - used to execute rclone commands, effective configuration owner + +##### `type` + +Data type: `Enum[ + 'amazon cloud drive', + 'azureblob', + 'b2', + 'box', + 'crypt', + 'cache', + 'chunker', + 'drive', + 'dropbox', + 'fichier', + 'ftp', + 'google cloud storage', + 'google photos', + 'http', + 'swift', + 'hubic', + 'jottacloud', + 'koofr', + 'local', + 'mailru', + 'mega', + 'memory', + 'onedrive', + 'opendrive', + 'pcloud', + 'premiumizeme', + 'putio', + 'qingstor', + 's3', + 'sftp', + 'sharefile', + 'sugarsync', + 'union', + 'webdav', + 'yandex']` + +configuration remote type + +##### `options` + +Data type: `Hash[String, Optional[String]]` + +configuration options - Hash of options for `rclone config` command + +##### `config_name` + +Data type: `String` + +configuration name - should be unique among Rclone configurations, defaults to title + +Default value: $title + +### rclone::config::gdrive + +Ensures Drive Rclone configuration of given name and params. Include of `rclone` is required. +Support only service account credentials (token authentication requires human interaction when setup). + +#### Examples + +##### + +```puppet +rclone::config::gdrive { 'drive_remote': + os_user => 'my_user', + client_id => 'SOME_CLIENT_ID', + client_secret => 'SOME_CLIENT_SECRET', + service_account_credentials => 'SERVICE_CREDENTIALS', +} +``` + +#### Parameters + +The following parameters are available in the `rclone::config::gdrive` defined type. + +##### `ensure` + +Data type: `Enum['present', 'absent']` + +configuration ensure + +Default value: 'present' + +##### `os_user` + +Data type: `String` + +operating system user - used to execute rclone commands, effective configuration owner + +##### `config_name` + +Data type: `String` + +configuration name - should be unique among Rclone configurations, defaults to title + +Default value: $title + +##### `client_id` + +Data type: `String` + +Google drive client_id, maps to Rclone `client_id` property + +##### `client_secret` + +Data type: `String` + +Google drive client_secret, maps to Rclone `client_secret` property + +##### `service_account_credentials` + +Data type: `String` + +Google drive service_account_credentials, maps to Rclone `service_account_credentials` property + +##### `scope` + +Data type: `String` + +Google drive access scope, maps to Rclone `scope` property + +Default value: 'drive' + +##### `root_folder_id` + +Data type: `Optional[String]` + +Id of the drive root folder, maps to Rclone `root_folder_id` property + +Default value: `undef` + +##### `team_drive` + +Data type: `Optional[String]` + +Id of the team drive, maps to Rclone `team_drive` property + +Default value: `undef` + +### rclone::config::s3 + +Ensures S3 Rclone configuration of given name and params. Include of `rclone` is required. +Currently only AWS provider is supported. + +#### Examples + +##### + +```puppet +rclone::config::s3 { 's3_remote': + os_user => 'my_user', + access_key_id => 'SOME_ACCESS_KEY', + secret_access_key => 'SECRET_ACCESS_KEY', + region => 'us-east-1', +} +``` + +#### Parameters + +The following parameters are available in the `rclone::config::s3` defined type. + +##### `ensure` + +Data type: `Enum['present', 'absent']` + +configuration ensure + +Default value: 'present' + +##### `os_user` + +Data type: `String` + +operating system user - used to execute rclone commands, effective configuration owner + +##### `config_name` + +Data type: `String` + +configuration name - should be unique among Rclone configurations, defaults to title + +Default value: $title + +##### `access_key_id` + +Data type: `String` + +S3 provider's access_key_id, maps to Rclone `access_key_id` property + +##### `secret_access_key` + +Data type: `String` + +S3 provider's secret_access_key, maps to Rclone `secret_access_key` property + +##### `region` + +Data type: `String` + +S3 provider's region, maps to Rclone `region` property + +##### `s3_provider` + +Data type: `Enum['AWS']` + +S3 provider, maps to Rclone `provider` property + +Default value: 'AWS' + +##### `canned_acl` + +Data type: `Optional[String]` + +S3 canned ACL, maps to Rclone `acl` property + +Default value: `undef` + +##### `endpoint` + +Data type: `Optional[String]` + +S3 provider's endpoint, maps to Rclone `endpoint` property + +Default value: `undef` + +##### `location_constraint` + +Data type: `Optional[String]` + +S3 location_constraint, maps to Rclone `location_constraint` property + +Default value: `undef` + +##### `location_constraint` + +S3 location_constraint, maps to Rclone `location_constraint` property + +Default value: `undef` + +##### `server_side_encryption` + +Data type: `Optional[String]` + +S3 server_side_encryption, maps to Rclone `server_side_encryption` property + +Default value: `undef` + +##### `storage_class` + +Data type: `Optional[String]` + +S3 storage_class, maps to Rclone `storage_class` property + +Default value: `undef` + ## Functions diff --git a/manifests/config.pp b/manifests/config.pp new file mode 100644 index 0000000..533f238 --- /dev/null +++ b/manifests/config.pp @@ -0,0 +1,84 @@ +# @summary General configuration for Rclone. +# +# Ensures Rclone configuration of given name, type and params. Include of `rclone` is required. +# +# @example +# rclone::config { 'my_remote': +# os_user => 'my_user', +# type => 'ftp', +# options => {...} +# } +# +# @param ensure +# configuration ensure +# @param os_user +# operating system user - used to execute rclone commands, effective configuration owner +# @param type +# configuration remote type +# @param options +# configuration options - Hash of options for `rclone config` command +# @param config_name +# configuration name - should be unique among Rclone configurations, defaults to title +define rclone::config ( + String $os_user, + Enum[ + 'amazon cloud drive', + 'azureblob', + 'b2', + 'box', + 'crypt', + 'cache', + 'chunker', + 'drive', + 'dropbox', + 'fichier', + 'ftp', + 'google cloud storage', + 'google photos', + 'http', + 'swift', + 'hubic', + 'jottacloud', + 'koofr', + 'local', + 'mailru', + 'mega', + 'memory', + 'onedrive', + 'opendrive', + 'pcloud', + 'premiumizeme', + 'putio', + 'qingstor', + 's3', + 'sftp', + 'sharefile', + 'sugarsync', + 'union', + 'webdav', + 'yandex'] $type, + Hash[String, Optional[String]] $options, + Enum['present', 'absent'] $ensure = 'present', + String $config_name = $title, +) { + + if ! defined(Class[rclone]) { + fail('You must include the rclone base class before using any defined resources') + } + + $_operation = $ensure ? { + 'present' => 'create', + 'absent' => 'delete', + default => fail("Invalid ensure value '${ensure}'") + } + + $_options = $options.filter |$key, $val| { $val != undef }.convert_to(Array).flatten() + + exec { "rclone ${_operation} remote ${config_name} for user ${os_user}": + command => shell_join(['rclone', 'config', $_operation, $config_name] + + if ($_operation == 'create') { [$type] + $_options } else { [] }), + user => $os_user, + path => '/usr/bin', + require => Class[rclone], + } +} diff --git a/manifests/config/gdrive.pp b/manifests/config/gdrive.pp new file mode 100644 index 0000000..b99927b --- /dev/null +++ b/manifests/config/gdrive.pp @@ -0,0 +1,60 @@ +# @summary Google Drive configuration for Rclone. +# +# Ensures Drive Rclone configuration of given name and params. Include of `rclone` is required. +# Support only service account credentials (token authentication requires human interaction when setup). +# +# @example +# rclone::config::gdrive { 'drive_remote': +# os_user => 'my_user', +# client_id => 'SOME_CLIENT_ID', +# client_secret => 'SOME_CLIENT_SECRET', +# service_account_credentials => 'SERVICE_CREDENTIALS', +# } +# +# @param ensure +# configuration ensure +# @param os_user +# operating system user - used to execute rclone commands, effective configuration owner +# @param config_name +# configuration name - should be unique among Rclone configurations, defaults to title +# @param client_id +# Google drive client_id, maps to Rclone `client_id` property +# @param client_secret +# Google drive client_secret, maps to Rclone `client_secret` property +# @param service_account_credentials +# Google drive service_account_credentials, maps to Rclone `service_account_credentials` property +# @param scope +# Google drive access scope, maps to Rclone `scope` property +# @param root_folder_id +# Id of the drive root folder, maps to Rclone `root_folder_id` property +# @param team_drive +# Id of the team drive, maps to Rclone `team_drive` property +define rclone::config::gdrive ( + String $os_user, + String $client_id, + String $client_secret, + String $service_account_credentials, + Enum['present', 'absent'] $ensure = 'present', + String $config_name = $title, + String $scope = 'drive', + Optional[String] $root_folder_id = undef, + Optional[String] $team_drive = undef, +) { + + $_options = { + client_id => $client_id, + client_secret => $client_secret, + service_account_credentials => $service_account_credentials, + scope => $scope, + root_folder_id => $root_folder_id, + team_drive => $team_drive, + } + + rclone::config { $config_name: + ensure => $ensure, + os_user => $os_user, + type => 'drive', + options => $_options, + } + +} diff --git a/manifests/config/s3.pp b/manifests/config/s3.pp new file mode 100644 index 0000000..31cf731 --- /dev/null +++ b/manifests/config/s3.pp @@ -0,0 +1,75 @@ +# @summary S3 configuration for Rclone. +# +# Ensures S3 Rclone configuration of given name and params. Include of `rclone` is required. +# Currently only AWS provider is supported. +# +# @example +# rclone::config::s3 { 's3_remote': +# os_user => 'my_user', +# access_key_id => 'SOME_ACCESS_KEY', +# secret_access_key => 'SECRET_ACCESS_KEY', +# region => 'us-east-1', +# } +# +# @param ensure +# configuration ensure +# @param os_user +# operating system user - used to execute rclone commands, effective configuration owner +# @param config_name +# configuration name - should be unique among Rclone configurations, defaults to title +# @param access_key_id +# S3 provider's access_key_id, maps to Rclone `access_key_id` property +# @param secret_access_key +# S3 provider's secret_access_key, maps to Rclone `secret_access_key` property +# @param region +# S3 provider's region, maps to Rclone `region` property +# @param s3_provider +# S3 provider, maps to Rclone `provider` property +# @param canned_acl +# S3 canned ACL, maps to Rclone `acl` property +# @param endpoint +# S3 provider's endpoint, maps to Rclone `endpoint` property +# @param location_constraint +# S3 location_constraint, maps to Rclone `location_constraint` property +# @param location_constraint +# S3 location_constraint, maps to Rclone `location_constraint` property +# @param server_side_encryption +# S3 server_side_encryption, maps to Rclone `server_side_encryption` property +# @param storage_class +# S3 storage_class, maps to Rclone `storage_class` property +define rclone::config::s3 ( + String $os_user, + String $access_key_id, + String $secret_access_key, + String $region, + Enum['present', 'absent'] $ensure = 'present', + String $config_name = $title, + Enum['AWS'] $s3_provider = 'AWS', + Optional[String] $canned_acl = undef, + Optional[String] $endpoint = undef, + Optional[String] $location_constraint = undef, + Optional[String] $server_side_encryption = undef, + Optional[String] $storage_class = undef, +) { + + $_options = { + provider => $s3_provider, + env_auth => 'false', + access_key_id => $access_key_id, + secret_access_key => $secret_access_key, + region => $region, + acl => $canned_acl, + endpoint => $endpoint, + location_constraint => $location_constraint, + server_side_encryption => $server_side_encryption, + storage_class => $storage_class, + } + + rclone::config { $config_name: + ensure => $ensure, + os_user => $os_user, + type => 's3', + options => $_options, + } + +} diff --git a/metadata.json b/metadata.json index 4ffa834..85d1785 100644 --- a/metadata.json +++ b/metadata.json @@ -11,6 +11,10 @@ { "name": "puppet/archive", "version_requirement": ">= 4.4.0 < 5.0.0" + }, + { + "name": "puppetlabs/stdlib", + "version_requirement": ">= 6.0.0 < 7.0.0" } ], "operatingsystem_support": [ diff --git a/spec/defines/config/gdrive_spec.rb b/spec/defines/config/gdrive_spec.rb new file mode 100644 index 0000000..4445d19 --- /dev/null +++ b/spec/defines/config/gdrive_spec.rb @@ -0,0 +1,76 @@ +# frozen_string_literal: true + +require 'spec_helper' + +describe 'rclone::config::gdrive' do + let(:title) { 'test_r' } + let(:params) do + { + os_user: 'user', + client_id: 'gdrive_id', + client_secret: 'gdrive_secret', + service_account_credentials: 'SAC', + } + end + + context 'on missing rclone main class' do + it { is_expected.to compile.and_raise_error(%r{include the rclone base class}) } + end + + on_supported_os.each do |os, os_facts| + context "on #{os}" do + let(:facts) { os_facts } + let(:pre_condition) { 'include rclone' } + + it { is_expected.to compile } + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r drive ' \ + 'client_id gdrive_id client_secret gdrive_secret service_account_credentials SAC scope drive', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + + context 'with optional params' do + let(:params) do + super().merge( + root_folder_id: 'root_id', + team_drive: 'team_id', + ) + end + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r drive ' \ + 'client_id gdrive_id client_secret gdrive_secret service_account_credentials SAC scope drive ' \ + 'root_folder_id root_id team_drive team_id', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + + context 'with ensure => absent' do + let(:params) do + super().merge(ensure: 'absent') + end + + it { + is_expected.to contain_exec('rclone delete remote test_r for user user') + .with( + command: 'rclone config delete test_r', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + end + end +end diff --git a/spec/defines/config/s3_spec.rb b/spec/defines/config/s3_spec.rb new file mode 100644 index 0000000..32df209 --- /dev/null +++ b/spec/defines/config/s3_spec.rb @@ -0,0 +1,79 @@ +# frozen_string_literal: true + +require 'spec_helper' + +describe 'rclone::config::s3' do + let(:title) { 'test_r' } + let(:params) do + { + os_user: 'user', + access_key_id: 'aws_id', + secret_access_key: 'aws_key', + region: 'eu-west-1', + } + end + + context 'on missing rclone main class' do + it { is_expected.to compile.and_raise_error(%r{include the rclone base class}) } + end + + on_supported_os.each do |os, os_facts| + context "on #{os}" do + let(:facts) { os_facts } + let(:pre_condition) { 'include rclone' } + + it { is_expected.to compile } + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r s3 ' \ + 'provider AWS env_auth false access_key_id aws_id secret_access_key aws_key region eu-west-1', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + + context 'with optional params' do + let(:params) do + super().merge( + canned_acl: 'private', + endpoint: 'some-s3.com', + location_constraint: 'eu-west-1', + server_side_encryption: 'AES256', + storage_class: 'STANDARD', + ) + end + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r s3 ' \ + 'provider AWS env_auth false access_key_id aws_id secret_access_key aws_key region eu-west-1 ' \ + 'acl private endpoint some-s3.com location_constraint eu-west-1 server_side_encryption AES256 storage_class STANDARD', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + + context 'with ensure => absent' do + let(:params) do + super().merge(ensure: 'absent') + end + + it { + is_expected.to contain_exec('rclone delete remote test_r for user user') + .with( + command: 'rclone config delete test_r', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + end + end +end diff --git a/spec/defines/config_spec.rb b/spec/defines/config_spec.rb new file mode 100644 index 0000000..a6310c3 --- /dev/null +++ b/spec/defines/config_spec.rb @@ -0,0 +1,69 @@ +# frozen_string_literal: true + +require 'spec_helper' + +describe 'rclone::config' do + let(:title) { 'test_r' } + let(:params) do + { + os_user: 'user', + type: 'local', + options: {}, + } + end + + context 'on missing rclone main class' do + it { is_expected.to compile.and_raise_error(%r{include the rclone base class}) } + end + + on_supported_os.each do |os, os_facts| + context "on #{os}" do + let(:facts) { os_facts } + let(:pre_condition) { 'include rclone' } + + it { is_expected.to compile } + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r local', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + + context 'with options' do + let(:params) do + super().merge(options: { option_a: 'some_val', option_b: :undef, option_c: '\\a $"&' }) + end + + it { + is_expected.to contain_exec('rclone create remote test_r for user user') + .with( + command: 'rclone config create test_r local option_a some_val option_c \\\a\ \$\\"\&', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + + context 'with ensure => absent' do + let(:params) do + super().merge(ensure: 'absent') + end + + it { + is_expected.to contain_exec('rclone delete remote test_r for user user') + .with( + command: 'rclone config delete test_r', + user: 'user', + path: '/usr/bin', + ) + .that_requires('Class[rclone]') + } + end + end + end +end