diff --git a/.gitignore b/.gitignore index e680d89..948b8e6 100644 --- a/.gitignore +++ b/.gitignore @@ -17,4 +17,4 @@ Vagrantfile # Test provisioning -declarative.yaml \ No newline at end of file +*.yaml \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b7538d..0d44497 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -35,3 +35,14 @@ ansible_group_vars_path and ansible_host_vars_path are not supported anymore # 0.7.0 (November 02, 2016) * introduced support for declarative cluster definition + +# 0.7.1 (November 04, 2016) + +* Minor fix + +# 0.7.2 (November 16, 2016) + +* issues #11 Allow management of Ansible vars for all hosts +* issues #9 Create group_vars and host_vars directory only if necessary + +* breacking change: custom group of groups all_groups:children removed. Insteal use all (automatically created by ansible) diff --git a/doc/declarative.md b/doc/declarative.md index 8e6e6e3..94da3f9 100644 --- a/doc/declarative.md +++ b/doc/declarative.md @@ -295,9 +295,9 @@ kubernetes: ... ... ansible_group_vars: - etcd : + etcd: var1: ... - docker : + docker: var2: ... var3: ... ... @@ -308,6 +308,17 @@ Group vars can be set to literal value or to Jinja2 value generators, that will - **context_vars** see below - **nodes**, list of nodes in the ansible_group to which the group_vars belong +Additionally it is possible to set variables for all groups/all hosts, by setting vars for the pre-defined `all` group of groups: + +```yaml +kubernetes: + ... + ansible_group_vars: + all: + var1: ... + ... +``` + Ansible group vars will be stored into yaml files saved into `{cluster.ansible_playbook_path}\group_vars` folder. The variable `cluster.ansible_playbook_path` defaults to the current directory (the directory of the Vagrantfile) + `/provisioning`; this value can be changed like any other cluster attributes (see Defining cluster & cluster attributes). @@ -384,18 +395,42 @@ Context vars can be set to literal value or to Jinja2 value generators, that wil - nodes, list of nodes in the ansible_group to which the group_vars belong - > Context_vars generator are always executed before group_vars and host_vars generators; the resulting context, is given in input to group_vars and host_vars generators. +> In addition to context vars for groups, it is possible to create context_vars for all groups/all hosts, by setting vars for the pre-defined `all` group of groups; in this case, intuitively, the list of nodes whitin the context contains all the nodes. + Then, you can use the above context var when generating group_vars for host vars. +```yaml +kubernetes: + box: centos/7 + master: + ... + ansible_groups: + - kb8-master + minions: + ... + ansible_groups: + - kb8-minions + + ansible_context_vars: + all: + var0: "{{ nodes | count }}" + kb8-master: + var1: "{{ nodes | count }}" + + ansible_group_vars: + all: + var0_from_context: "{{ context['var0'] }}" + kb8-master: + var1_from_context: "{{ context['var1'] }}" +``` + ### Group of groups A useful ansible inventory feature is [group of groups](http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups). -By default vagrant-compose will generate a group named `[all_groups:children]` with all the ansible_groups defined in cluster configuration; however: -- you cannot rename all_groups -- you cannot exclude any ansible group from all_groups. +By default ansible has a group named `[all]` with all nodes in the cluster. If you need higher control on groups of groups you can simply add a new item to the variable `config.cluster.ansible_groups` before creating nodes. diff --git a/doc/programmatic.md b/doc/programmatic.md index 75a01af..8e6f973 100644 --- a/doc/programmatic.md +++ b/doc/programmatic.md @@ -302,9 +302,9 @@ end As you can see, `consul` and `docker` ansible_groups now include both nodes from `consul-agent` and `load-balancer` node set; vice versa, other groups like `registrator`, `consul-template`, `nginx` contain node only from one of the two nodes set. -Ansible playbook can leverage on groups for providing machines with the required software stacks. +Ansible playbook can additionally leverage on groups for providing machines with the required software stacks. -NB. you can see resulting ansible_groups by using `debug` command with `verbose` equal to `true`. +> NB. you can see resulting ansible_groups by using `debug` command with `verbose` equal to `true`. ### Defining group vars @@ -329,6 +329,18 @@ config.cluster.compose('test') do |c| end ``` +Additionally, it is possible to set variables for all groups/all hosts, by setting vars for the pre-defined `all` group of groups: + +``` ruby +config.cluster.compose('test') do |c| + ... + c.ansible_group_vars['all'] = lambda { |context, nodes| + return { 'var0' => nodes.length } + } + ... +end +``` + Ansible group vars will be stored into yaml files saved into `{cluster.ansible_playbook_path}\group_vars` folder. The variable `cluster.ansible_playbook_path` defaults to the current directory (the directory of the Vagrantfile) + `/provisioning`; this value can be changed like any other cluster attributes (see Defining cluster attributes). @@ -379,6 +391,8 @@ end > Context_vars generator are always executed before group_vars and host_vars generators; the resulting context, is given in input to group_vars and host_vars generators. +> In addition to context vars for groups, it is possible to create context_vars for all groups/all hosts, by setting vars for the pre-defined `all` group of groups; in this case, intuitively, the list of nodes whitin the context contains all the nodes. + Then, you can use the above context var when generating group_vars for nodes in the `consul-agent` group. ``` ruby @@ -397,9 +411,7 @@ end ### Group of groups A useful ansible inventory feature is [group of groups](http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups). -By default vagrant-compose will generate a group named `[all_groups:children]` with all the ansible_groups defined in cluster configuration; however: -- you cannot rename all_groups -- you cannot exclude any ansible group from all_groups. +By default ansible has a group named `[all]` with all the nodes defined in cluster configuration. If you need higher control on groups of groups you can simply add a new item to the variable `config.cluster.ansible_groups` before creating nodes. diff --git a/lib/vagrant/compose/config.rb b/lib/vagrant/compose/config.rb index 707af9c..5e000ac 100644 --- a/lib/vagrant/compose/config.rb +++ b/lib/vagrant/compose/config.rb @@ -62,8 +62,6 @@ def filterInventory(inventory) end end - ansible_groups['all_groups:children'] = ansible_groups.keys - return ansible_groups end diff --git a/lib/vagrant/compose/declarative/cluster.rb b/lib/vagrant/compose/declarative/cluster.rb index f92ff9b..5916c1e 100644 --- a/lib/vagrant/compose/declarative/cluster.rb +++ b/lib/vagrant/compose/declarative/cluster.rb @@ -24,27 +24,27 @@ class Cluster # It defaults to current directory/provisioning attr_accessor :ansible_playbook_path - # Costruttore di una istanza di cluster. + # Costruttore di una istanza di cluster. def initialize() - @multimachine_filter = "" + @multimachine_filter = "" end # Implements cluster creation from a playbook file - def from (file) - # calls vagrant-playbook utility for executing the playbook file. - playbook = YAML.load(pycompose (file)) + def from (file) + # calls vagrant-playbook utility for executing the playbook file. + playbook = YAML.load(pycompose (file)) - # extract cluster attributes - @name = playbook.keys[0] - @box = playbook[@name]['box'] - @domain = playbook[@name]['domain'] - @ansible_playbook_path = playbook[@name]['ansible_playbook_path'] + # extract cluster attributes + @name = playbook.keys[0] + @box = playbook[@name]['box'] + @domain = playbook[@name]['domain'] + @ansible_playbook_path = playbook[@name]['ansible_playbook_path'] # extract nodes nodes = [] playbook[@name]['nodes'].each do |node| - boxname = node.keys[0] + boxname = node.keys[0] box = node[boxname]['box'] hostname = node[boxname]['hostname'] @@ -64,74 +64,85 @@ def from (file) # extract ansible inventory, ansible_group_vars, ansible_host_vars ansible_groups = {} if playbook[@name].key?("ansible") - ansible = playbook[@name]['ansible'] - - # extract ansible inventory - ansible_groups = ansible['inventory'] - - # cleanup ansible_group_vars files - # TODO: make safe - ansible_group_vars_path = File.join(@ansible_playbook_path, 'group_vars') - - FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path) - Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")} - - #generazione ansible_group_vars file (NB. 1 group = 1 gruppo host ansible) - ansible['group_vars'].each do |group, vars| - # crea il file (se sono state generate delle variabili) - unless vars.empty? - # TODO: make safe - File.open(File.join(ansible_group_vars_path,"#{group}.yml") , 'w+') do |file| - file.puts YAML::dump(vars) - end - end - end - - # cleanup ansible_host_vars files (NB. 1 nodo = 1 host) - # TODO: make safe - ansible_host_vars_path = File.join(@ansible_playbook_path, 'host_vars') - - FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path) - Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")} - - #generazione ansible_host_vars file - ansible['host_vars'].each do |host, vars| - # crea il file (se sono state generate delle variabili) - unless vars.empty? - # TODO: make safe - File.open(File.join(ansible_host_vars_path,"#{host}.yml") , 'w+') do |file| - file.puts YAML::dump(vars) - end - end - end - end - return nodes, ansible_groups - end + ansible = playbook[@name]['ansible'] + + # extract ansible inventory + ansible_groups = ansible['inventory'] + + # cleanup ansible_group_vars files + # TODO: make safe + ansible_group_vars_path = File.join(@ansible_playbook_path, 'group_vars') + + if File.exists?(ansible_group_vars_path) + Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + end + + #generazione ansible_group_vars file (NB. 1 group = 1 gruppo host ansible) + if ansible.key?("group_vars") + ansible['group_vars'].each do |group, vars| + # crea il file (se sono state generate delle variabili) + unless vars.empty? + FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path) + # TODO: make safe + fileName = group.gsub(':', '_') + File.open(File.join(ansible_group_vars_path,"#{fileName}.yml") , 'w+') do |file| + file.puts YAML::dump(vars) + end + end + end + end + + # cleanup ansible_host_vars files (NB. 1 nodo = 1 host) + # TODO: make safe + ansible_host_vars_path = File.join(@ansible_playbook_path, 'host_vars') + + if File.exists?(ansible_host_vars_path) + Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + end + + #generazione ansible_host_vars file + if ansible.key?("host_vars") + ansible['host_vars'].each do |host, vars| + # crea il file (se sono state generate delle variabili) + unless vars.empty? + FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path) + + # TODO: make safe + File.open(File.join(ansible_host_vars_path,"#{host}.yml") , 'w+') do |file| + file.puts YAML::dump(vars) + end + end + end + end + end + return nodes, ansible_groups + end - # Executes pycompose command - def pycompose (file) - p_err = "" - p_out = "" - begin - p_status = Open4::popen4("vagrant-playbook -f #{file}") do |pid, stdin, stdout, stderr| - p_err = stderr.read.strip - p_out = stdout.read.strip - end - rescue Errno::ENOENT - raise VagrantPlugins::Compose::Errors::PyComposeMissing - rescue Exception => e - raise VagrantPlugins::Compose::Errors::PyComposeError, :message => e.message - end + # Executes pycompose command + def pycompose (file) + p_err = "" + p_out = "" + + begin + p_status = Open4::popen4("vagrant-playbook -f #{file}") do |pid, stdin, stdout, stderr| + p_err = stderr.read.strip + p_out = stdout.read.strip + end + rescue Errno::ENOENT + raise VagrantPlugins::Compose::Errors::PyComposeMissing + rescue Exception => e + raise VagrantPlugins::Compose::Errors::PyComposeError, :message => e.message + end - if p_status.exitstatus != 0 - raise VagrantPlugins::Compose::Errors::PyComposeError, :message => p_err - end + if p_status.exitstatus != 0 + raise VagrantPlugins::Compose::Errors::PyComposeError, :message => p_err + end - return p_out - end + return p_out + end end end diff --git a/lib/vagrant/compose/programmatic/cluster.rb b/lib/vagrant/compose/programmatic/cluster.rb index ee981cd..5e5a6cd 100644 --- a/lib/vagrant/compose/programmatic/cluster.rb +++ b/lib/vagrant/compose/programmatic/cluster.rb @@ -94,6 +94,7 @@ def compose ansible_groups[ansible_group] << node end end + extended_ansible_groups = ansible_groups.merge({'all' => nodes}) ## Fase2: Configurazione provisioning del cluster via Ansible # Ogni nodo diventerà una vm su cui sarà fatto il provisioning, ovvero un host nell'inventory di ansible @@ -120,7 +121,7 @@ def compose context = {} #genearazione context (NB. 1 group = 1 gruppo host ansible) - ansible_groups.each do |ansible_group, ansible_group_nodes| + extended_ansible_groups.each do |ansible_group, ansible_group_nodes| # genero le variabili per il group provisioners = @ansible_context_vars[ansible_group] unless provisioners.nil? @@ -145,11 +146,12 @@ def compose # TODO: make safe ansible_group_vars_path = File.join(@ansible_playbook_path, 'group_vars') - FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path) - Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + if File.exists?(ansible_group_vars_path) + Dir.foreach(ansible_group_vars_path) {|f| fn = File.join(ansible_group_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + end #generazione ansible_group_vars file (NB. 1 group = 1 gruppo host ansible) - ansible_groups.each do |ansible_group, ansible_group_nodes| + extended_ansible_groups.each do |ansible_group, ansible_group_nodes| ansible_group_vars = {} # genero le variabili per il group provisioners = @ansible_group_vars[ansible_group] @@ -171,8 +173,11 @@ def compose # crea il file (se sono state generate delle variabili) unless ansible_group_vars.empty? + FileUtils.mkdir_p(ansible_group_vars_path) unless File.exists?(ansible_group_vars_path) + # TODO: make safe - File.open(File.join(ansible_group_vars_path,"#{ansible_group}.yml") , 'w+') do |file| + fileName = ansible_group.gsub(':', '_') + File.open(File.join(ansible_group_vars_path,"#{fileName}.yml") , 'w+') do |file| file.puts YAML::dump(ansible_group_vars) end end @@ -182,8 +187,9 @@ def compose # TODO: make safe ansible_host_vars_path = File.join(@ansible_playbook_path, 'host_vars') - FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path) - Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + if File.exists?(ansible_host_vars_path) + Dir.foreach(ansible_host_vars_path) {|f| fn = File.join(ansible_host_vars_path, f); File.delete(fn) if f.end_with?(".yml")} + end #generazione ansible_host_vars file nodes.each do |node| @@ -211,6 +217,8 @@ def compose # crea il file (se sono state generate delle variabili) unless ansible_host_vars.empty? + FileUtils.mkdir_p(ansible_host_vars_path) unless File.exists?(ansible_host_vars_path) + # TODO: make safe File.open(File.join(ansible_host_vars_path,"#{node.hostname}.yml") , 'w+') do |file| file.puts YAML::dump(ansible_host_vars) diff --git a/lib/vagrant/compose/version.rb b/lib/vagrant/compose/version.rb index cb9ee78..95fa03d 100644 --- a/lib/vagrant/compose/version.rb +++ b/lib/vagrant/compose/version.rb @@ -1,5 +1,5 @@ module Vagrant module Compose - VERSION = "0.7.0" + VERSION = "0.7.2" end end