Skip to content
This repository has been archived by the owner on Jun 24, 2022. It is now read-only.

Commit

Permalink
Merge branch 'nullkarma-xpack'
Browse files Browse the repository at this point in the history
  • Loading branch information
Dale McDiarmid committed Aug 18, 2017
2 parents c0ae63d + 06cf45e commit 2c69fbc
Show file tree
Hide file tree
Showing 9 changed files with 205 additions and 11 deletions.
8 changes: 7 additions & 1 deletion .kitchen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,4 +104,10 @@ suites:
run_list:
attributes:
provisioner:
playbook: test/integration/xpack.yml
playbook: test/integration/xpack.yml
- name: xpack-standard-5x
run_list:
attributes:
provisioner:
playbook: test/integration/xpack-standard.yml
idempotency_test: true
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ The application of the elasticsearch role results in the installation of a node
The simplest configuration therefore consists of:

```
---
- name: Simple Example
hosts: localhost
roles:
Expand Down Expand Up @@ -79,7 +78,7 @@ The following illustrates applying configuration parameters to an Elasticsearch
es_heap_size: 1g
es_api_port: 9201
```
`

The role utilises Elasticsearch version defaults. The following should be set to ensure a successful cluster forms.

* ```es_config['http.port']``` - the http port for the node
Expand All @@ -95,7 +94,6 @@ The role makes no attempt to enforce the setting of these are requires users to
A more complex example:

```
---
- name: Elasticsearch with custom configuration
hosts: localhost
roles:
Expand Down Expand Up @@ -224,13 +222,15 @@ ansible-playbook -i hosts ./your-playbook.yml

X-Pack features, such as Security, are supported. This feature is currently experimental. To enable X-Pack set the parameter `es_enable_xpack` to true and list the required features in the parameter `es_xpack_features`.

The parameter `es_xpack_features` by default enables all features i.e. it defaults to ["alerting","monitoring","graph","security"]
The parameter `es_xpack_features` by default enables all features i.e. it defaults to ["alerting","monitoring","graph","security","ml"]

The following additional parameters allow X-Pack to be configured:

* ```es_message_auth_file``` System Key field to allow message authentication. This file should be placed in the 'files' directory.
* ```es_xpack_custom_url``` Url from which X-Pack can be downloaded. This can be used for installations in isolated environments where the elastic.co repo is not accessible. e.g. ```es_xpack_custom_url: "https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.5.1.zip"```
* ```es_role_mapping``` Role mappings file declared as yml as described [here](https://www.elastic.co/guide/en/x-pack/current/mapping-roles.html)


```
es_role_mapping:
power_user:
Expand Down Expand Up @@ -327,9 +327,9 @@ These can either be set to a user declared in the file based realm, with admin p

### Additional Configuration

Additional parameters to es_config allow the customization of the Java and Elasticsearch versions, in addition to role behaviour. Options include:
In addition to es_config, the following parameters allow the customization of the Java and Elasticsearch versions as well as the role behaviour. Options include:

* ```es_major_version``` (e.g. "5.1" ). Should be consistent with es_version. For versions >= 5.0 this must be "5.x".
* ```es_major_version``` Should be consistent with es_version. For versions >= 5.0 this must be "5.x".
* ```es_version``` (e.g. "5.1.2").
* ```es_api_host``` The host name used for actions requiring HTTP e.g. installing templates. Defaults to "localhost".
* ```es_api_port``` The port used for actions requiring HTTP e.g. installing templates. Defaults to 9200. **CHANGE IF THE HTTP PORT IS NOT 9200**
Expand Down
2 changes: 1 addition & 1 deletion defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ es_max_threads: 2048
es_max_map_count: 262144
es_allow_downgrades: false
es_enable_xpack: false
es_xpack_features: ["alerting","monitoring","graph","security"]
es_xpack_features: ["alerting","monitoring","graph","ml","security"]
#These are used for internal operations performed by ansible.
#They do not effect the current configuration
es_api_host: "localhost"
Expand Down
23 changes: 21 additions & 2 deletions tasks/xpack/elasticsearch-xpack-install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,32 @@


#Install plugin if not installed, or the es version has changed (so removed above), and its been requested
- name: Install x-pack plugin
- name: Download x-pack from url
get_url: url={{ es_xpack_custom_url }} dest=/tmp/x-pack-{{ es_version }}.zip
when: (x_pack_installed.rc == 1 or es_version_changed) and (es_enable_xpack and es_xpack_custom_url is defined)

- name: Install x-pack plugin from local
command: >
{{es_home}}/bin/elasticsearch-plugin install --silent --batch file:///tmp/x-pack-{{ es_version }}.zip
register: xpack_state
changed_when: xpack_state.rc == 0
when: (x_pack_installed.rc == 1 or es_version_changed) and (es_enable_xpack and es_xpack_custom_url is defined)
notify: restart elasticsearch
environment:
CONF_DIR: "{{ conf_dir }}"
ES_INCLUDE: "{{ instance_default_file }}"

- name: Delete x-pack zip file
file: dest=/tmp/x-pack-{{ es_version }}.zip state=absent
when: es_xpack_custom_url is defined

- name: Install x-pack plugin from elastic.co
command: >
{{es_home}}/bin/elasticsearch-plugin install --silent --batch x-pack
register: xpack_state
failed_when: "'ERROR' in xpack_state.stdout"
changed_when: xpack_state.rc == 0
when: (x_pack_installed.rc == 1 or es_version_changed) and es_enable_xpack
when: (x_pack_installed.rc == 1 or es_version_changed) and (es_enable_xpack and es_xpack_custom_url is not defined)
notify: restart elasticsearch
environment:
CONF_DIR: "{{ conf_dir }}"
Expand Down
10 changes: 9 additions & 1 deletion templates/elasticsearch.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,12 @@ xpack.monitoring.enabled: false

{% if not "alerting" in es_xpack_features %}
xpack.watcher.enabled: false
{% endif %}
{% endif %}

{% if not "ml" in es_xpack_features %}
xpack.ml.enabled: false
{% endif %}

{% if not "graph" in es_xpack_features %}
xpack.graph.enabled: false
{% endif %}
139 changes: 139 additions & 0 deletions test/integration/helpers/serverspec/xpack_standard_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
require 'spec_helper'

shared_examples 'xpack_standard::init' do |es_version,plugins|

describe user('elasticsearch') do
it { should exist }
end

describe service('security_node_elasticsearch') do
it { should be_running }
end

describe package('elasticsearch') do
it { should be_installed }
end

describe file('/etc/elasticsearch/security_node/elasticsearch.yml') do
it { should be_file }
it { should be_owned_by 'elasticsearch' }
end

describe file('/etc/elasticsearch/security_node/log4j2.properties') do
it { should be_file }
it { should be_owned_by 'elasticsearch' }
end

describe file('/etc/elasticsearch/security_node/elasticsearch.yml') do
it { should contain 'node.name: localhost-security_node' }
it { should contain 'cluster.name: elasticsearch' }
it { should contain 'path.conf: /etc/elasticsearch/security_node' }
it { should contain 'path.data: /var/lib/elasticsearch/localhost-security_node' }
it { should contain 'path.logs: /var/log/elasticsearch/localhost-security_node' }
it { should contain 'xpack.security.enabled: false' }
it { should contain 'xpack.watcher.enabled: false' }

end

describe 'Node listening' do
it 'listening in port 9200' do
expect(port 9200).to be_listening
end
end

describe 'version check' do
it 'should be reported as version '+es_version do
command = command('curl -s localhost:9200 | grep number')
expect(command.stdout).to match(es_version)
expect(command.exit_status).to eq(0)
end
end

describe file('/etc/init.d/elasticsearch') do
it { should_not exist }
end

describe file('/etc/default/elasticsearch') do
it { should_not exist }
end

describe file('/etc/sysconfig/elasticsearch') do
it { should_not exist }
end

describe file('/usr/lib/systemd/system/elasticsearch.service') do
it { should_not exist }
end

describe file('/etc/elasticsearch/elasticsearch.yml') do
it { should_not exist }
end

describe file('/etc/elasticsearch/logging.yml') do
it { should_not exist }
end

#Xpack specific tests
describe file('/usr/share/elasticsearch/plugins') do
it { should be_directory }
it { should be_owned_by 'elasticsearch' }
end

#Test if x-pack is activated
describe 'x-pack activation' do
it 'should be activated and valid' do
command = command('curl -s localhost:9200/_license?pretty=true')
expect(command.stdout).to match('"status" : "active"')
expect(command.exit_status).to eq(0)
end
end

describe file('/usr/share/elasticsearch/plugins/x-pack') do
it { should be_directory }
it { should be_owned_by 'elasticsearch' }
end

describe command('curl -s localhost:9200/_nodes/plugins?pretty=true -u es_admin:changeMeAgain | grep x-pack') do
its(:exit_status) { should eq 0 }
end

describe file('/etc/elasticsearch/security_node/x-pack') do
it { should be_directory }
it { should be_owned_by 'elasticsearch' }
end

describe file('/usr/share/elasticsearch/plugins/x-pack') do
it { should be_directory }
it { should be_owned_by 'elasticsearch' }
end

for plugin in plugins
describe file('/usr/share/elasticsearch/plugins/'+plugin) do
it { should be_directory }
it { should be_owned_by 'elasticsearch' }
end

describe command('curl -s localhost:9200/_nodes/plugins | grep \'"name":"'+plugin+'","version":"'+es_version+'"\'') do
its(:exit_status) { should eq 0 }
end
end

#Test users file, users_roles and roles.yml
describe file('/etc/elasticsearch/security_node/x-pack/users_roles') do
it { should be_owned_by 'elasticsearch' }
end

describe file('/etc/elasticsearch/security_node/x-pack/users') do
it { should be_owned_by 'elasticsearch' }
end

describe command('curl -s localhost:9200/_xpack') do
its(:stdout_as_json) { should include('features' => include('security' => include('enabled' => false))) }
its(:stdout_as_json) { should include('features' => include('watcher' => include('enabled' => false))) }
its(:stdout_as_json) { should include('features' => include('graph' => include('enabled' => true))) }
its(:stdout_as_json) { should include('features' => include('monitoring' => include('enabled' => true))) }
its(:stdout_as_json) { should include('features' => include('ml' => include('enabled' => true))) }
end

end

5 changes: 5 additions & 0 deletions test/integration/xpack-standard-5x/serverspec/default_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
require 'xpack_standard_spec'

describe 'Xpack Standard Tests v 5.x' do
include_examples 'xpack_standard::init', "5.5.1", []
end
2 changes: 2 additions & 0 deletions test/integration/xpack-standard-5x/xpack-standard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
- host: test-kitchen
15 changes: 15 additions & 0 deletions test/integration/xpack-standard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#Tests x-pack is idempotent and works when security is not enabled
---
- name: Elasticsearch Xpack tests - no security and manual download
hosts: localhost
roles:
- { role: elasticsearch, es_api_port: 9200, es_config: { "http.port": 9200, "transport.tcp.port":9300, discovery.zen.ping.unicast.hosts: "localhost:9300" }, es_instance_name: "security_node" }
vars:
es_version: "5.5.1"
es_heap_size: 2g
es_enable_xpack: true
es_xpack_custom_url: "https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.5.1.zip"
es_xpack_features:
- monitoring
- graph
- ml

0 comments on commit 2c69fbc

Please sign in to comment.