Skip to content
This repository has been archived by the owner. It is now read-only.

Commit

Permalink
Merge pull request #23 from HubbleStack/develop
Browse files Browse the repository at this point in the history 
Merge to master (prep for v2016.9.0)
  • Loading branch information
@basepi
basepi authored Aug 30, 2016
2 parents f696a9d + 77c15cc commit 04350fc
Show file tree
Hide file tree
Showing 12 changed files with 134 additions and 379 deletions.
62 changes: 34 additions & 28 deletions README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ You should now be able to sync the new modules to your minion(s) using the
salt \* saltutil.sync_beacons
Copy the ``hubblestack_pulsar.sls.orig`` into your Salt pillar, dropping the
``.orig`` extension and target it to selected minions.
Copy the ``pillar.example`` into your Salt pillar, renaming is as desired
(perhaps ``hubblestack_pulsar.sls``) and target it to selected minions.

.. code-block:: shell
Expand Down Expand Up @@ -131,6 +131,7 @@ the minions.
cd hubblestack-pulsar.git
mkdir -p /srv/salt/_beacons/
cp _beacons/pulsar.py /srv/salt/_beacons/
cp hubblestack_pulsar_config.yaml /srv/salt/
cp pillar.example /srv/pillar/hubblestack_pulsar.sls
salt \* saltutil.sync_beacons
Expand Down Expand Up @@ -159,27 +160,34 @@ with it. It simply runs quietly in the background and sends you alerts.
Configuration
=============

The default Pulsar configuration (found in ``<pillar/hubblestack_pulsar.sls>``
is meant to act as a template. Every environment will have different needs and
requirements, and we understand that, so we've designed Pulsar to be flexible.
The default Pulsar configuration (found in ``<pillar.example>``)
is meant to act as a template. It works in tandem with the
``<hubblestack_pulsar_config.yaml>`` file. Every environment will have
different needs and requirements, and we understand that, so we've designed
Pulsar to be flexible.

.. code-block:: yaml
# pillar.example
beacons:
pulsar:
/etc: { recurse: True, auto_add: True }
/bin: { recurse: True, auto_add: True }
/sbin: { recurse: True, auto_add: True }
/boot: { recurse: True, auto_add: True }
/usr/bin: { recurse: True, auto_add: True }
/usr/sbin: { recurse: True, auto_add: True }
/usr/local/bin: { recurse: True, auto_add: True }
/usr/local/sbin: { recurse: True, auto_add: True }
return: slack_pulsar
checksum: sha256
stats: True
batch: False
paths:
- salt://hubblestack_pulsar_config.yaml
# hubblestack_pulsar_config.yaml
/etc: { recurse: True, auto_add: True }
/bin: { recurse: True, auto_add: True }
/sbin: { recurse: True, auto_add: True }
/boot: { recurse: True, auto_add: True }
/usr/bin: { recurse: True, auto_add: True }
/usr/sbin: { recurse: True, auto_add: True }
/usr/local/bin: { recurse: True, auto_add: True }
/usr/local/sbin: { recurse: True, auto_add: True }
return: slack_pulsar
checksum: sha256
stats: True
batch: False
In order to receive Pulsar notifications you'll need to install the custom
returners found in the Quasar_ repository.
Expand Down Expand Up @@ -207,16 +215,14 @@ defined path.

.. code-block:: yaml
beacons:
pulsar:
/var:
recurse: True
auto_add: True
exclude:
- /var/log
- /var/spool
- /var/cache
- /var/lock
/var:
recurse: True
auto_add: True
exclude:
- /var/log
- /var/spool
- /var/cache
- /var/lock
.. _pulsar_under_the_hood:

Expand Down
130 changes: 50 additions & 80 deletions _beacons/pulsar.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import fnmatch
import os
import re
import yaml

# Import salt libs
import salt.ext.six
Expand Down Expand Up @@ -73,96 +74,43 @@ def _get_notifier():
return __context__['pulsar.notifier']


def validate(config):
'''
Validate the beacon configuration
'''

VALID_MASK = [
'access',
'attrib',
'close_nowrite',
'close_write',
'create',
'delete',
'delete_self',
'excl_unlink',
'ignored',
'modify',
'moved_from',
'moved_to',
'move_self',
'oneshot',
'onlydir',
'open',
'unmount'
]

# Configuration for pulsar beacon should be a dict of dicts
log.debug('config {0}'.format(config))
if not isinstance(config, dict):
return False, 'Configuration for pulsar beacon must be a dictionary.'
else:
for config_item in config:
if not isinstance(config[config_item], dict):
return False, ('Configuration for pulsar beacon must '
'be a dictionary of dictionaries.')
else:
if not any(j in ['mask', 'recurse', 'auto_add'] for j in config[config_item]):
return False, ('Configuration for pulsar beacon must '
'contain mask, recurse or auto_add items.')

if 'auto_add' in config[config_item]:
if not isinstance(config[config_item]['auto_add'], bool):
return False, ('Configuration for pulsar beacon '
'auto_add must be boolean.')

if 'recurse' in config[config_item]:
if not isinstance(config[config_item]['recurse'], bool):
return False, ('Configuration for pulsar beacon '
'recurse must be boolean.')

if 'mask' in config[config_item]:
if not isinstance(config[config_item]['mask'], list):
return False, ('Configuration for pulsar beacon '
'mask must be list.')
for mask in config[config_item]['mask']:
if mask not in VALID_MASK:
return False, ('Configuration for pulsar beacon '
'invalid mask option {0}.'.format(mask))
return True, 'Valid beacon configuration'


def beacon(config):
'''
Watch the configured files
Example Config
Example pillar config
.. code-block:: yaml
beacons:
pulsar:
/path/to/file/or/dir:
mask:
- open
- create
- close_write
recurse: True
auto_add: True
exclude:
- /path/to/file/or/dir/exclude1
- /path/to/file/or/dir/exclude2
- /path/to/file/or/dir/regex[\d]*$:
regex: True
return:
splunk:
batch: True
slack:
batch: False # overrides the global setting
checksum: sha256
stats: True
paths:
- salt://hubblestack_pulsar_config.yaml
Example yaml config on fileserver (targeted by pillar)
.. code-block:: yaml
/path/to/file/or/dir:
mask:
- open
- create
- close_write
recurse: True
auto_add: True
exclude:
- /path/to/file/or/dir/exclude1
- /path/to/file/or/dir/exclude2
- /path/to/file/or/dir/regex[\d]*$:
regex: True
return:
splunk:
batch: True
slack:
batch: False # overrides the global setting
checksum: sha256
stats: True
batch: True
Note that if `batch: True`, the configured returner must support receiving
a list of events, rather than single one-off events.
Expand Down Expand Up @@ -201,10 +149,32 @@ def beacon(config):
If pillar/grains/minion config key `hubblestack:pulsar:maintenance` is set to
True, then changes will be discarded.
'''
log.debug('Pulsar beacon called.')
log.trace('Pulsar beacon config from pillar:\n{0}'.format(config))
ret = []
notifier = _get_notifier()
wm = notifier._watch_manager

# Get config(s) from salt fileserver
new_config = config
if isinstance(config.get('paths'), list):
for path in config['paths']:
cpath = __salt__['cp.cache_file'](path)
if os.path.isfile(cpath):
with open(cpath, 'r') as f:
new_config = salt.utils.dictupdate.update(new_config,
yaml.safe_load(f),
recursive_update=True,
merge_lists=True)
else:
log.error('Path {0} does not exist or is not a file'.format(cpath))
else:
log.error('Pulsar beacon \'paths\' data improperly formatted. Should be list of salt:// paths')

config = new_config

log.trace('Pulsar beacon config (compiled from config list):\n{0}'.format(config))

# Read in existing events
if notifier.check_events(1):
notifier.read_events()
Expand Down
48 changes: 48 additions & 0 deletions hubblestack_pulsar_config.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/lib: { recurse: True, auto_add: True }
/bin: { recurse: True, auto_add: True }
/sbin: { recurse: True, auto_add: True }
/boot: { recurse: True, auto_add: True }
/lib64: { recurse: True, auto_add: True }
/usr/lib: { recurse: True, auto_add: True }
/usr/bin: { recurse: True, auto_add: True }
/usr/sbin: { recurse: True, auto_add: True }
/usr/lib64: { recurse: True, auto_add: True }
/usr/local/etc: { recurse: True, auto_add: True }
/usr/local/bin: { recurse: True, auto_add: True }
/usr/local/lib: { recurse: True, auto_add: True }
/usr/local/sbin: { recurse: True, auto_add: True }
/etc:
exclude:
- /etc/passwd.lock
- /etc/shadow.lock
- /etc/gshadow.lock
- /etc/group.lock
- /etc/passwd+
- /etc/passwd-
- /etc/shadow+
- /etc/shadow-
- /etc/group+
- /etc/group-
- /etc/gshadow+
- /etc/gshadow-
- /etc/cas/timestamp
- /etc/pki/nssdb/key4.db-journal
- /etc/pki/nssdb/cert9.db-journal
- /etc/salt/gpgkeys/random_seed
recurse: True
auto_add: True
/var:
exclude:
- /var/log
- /var/spool
- /var/cache
- /var/lock
- /var/lib/ntp
- /var/lib/mlocate
- /var/lib/logrotate.status
recurse: True
audo_add: True
return: slack_pulsar
checksum: sha256
stats: True
batch: True
50 changes: 2 additions & 48 deletions pillar.example
View file
Original file line number Diff line number Diff line change
@@ -1,50 +1,4 @@
beacons:
pulsar:
/lib: { recurse: True, auto_add: True }
/bin: { recurse: True, auto_add: True }
/sbin: { recurse: True, auto_add: True }
/boot: { recurse: True, auto_add: True }
/lib64: { recurse: True, auto_add: True }
/usr/lib: { recurse: True, auto_add: True }
/usr/bin: { recurse: True, auto_add: True }
/usr/sbin: { recurse: True, auto_add: True }
/usr/lib64: { recurse: True, auto_add: True }
/usr/local/etc: { recurse: True, auto_add: True }
/usr/local/bin: { recurse: True, auto_add: True }
/usr/local/lib: { recurse: True, auto_add: True }
/usr/local/sbin: { recurse: True, auto_add: True }
/etc:
exclude:
- /etc/passwd.lock
- /etc/shadow.lock
- /etc/gshadow.lock
- /etc/group.lock
- /etc/passwd+
- /etc/passwd-
- /etc/shadow+
- /etc/shadow-
- /etc/group+
- /etc/group-
- /etc/gshadow+
- /etc/gshadow-
- /etc/cas/timestamp
- /etc/pki/nssdb/key4.db-journal
- /etc/pki/nssdb/cert9.db-journal
- /etc/salt/gpgkeys/random_seed
recurse: True
auto_add: True
/var:
exclude:
- /var/log
- /var/spool
- /var/cache
- /var/lock
- /var/lib/ntp
- /var/lib/mlocate
- /var/lib/logrotate.status
recurse: True
audo_add: True
return: slack_pulsar
checksum: sha256
stats: True
batch: True
paths:
- salt://hubblestack_pulsar_config.yaml
Loading

0 comments on commit 04350fc

Please sign in to comment.