Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array handling #166

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Array handling #166

wants to merge 14 commits into from

Conversation

mkw
Copy link

@mkw mkw commented Dec 18, 2017

This is an attempt to provide support for arrays when expanding keys for JSON and Yaml.

It was motivated by using Consul for Spring Boot services, which allows for array configuration data with either comma-separated lists or property names ending with indexes in [] -- e.g. my/array/key[0]. While the existing loader provided fine support for loading Yaml files un-expanded, we wanted to expand keys in order to observe changes on a per-key level directly from Consul.

I believe that would address the now-closed #71 issue. It builds on the work done by @kapekost for better Yaml support, but adds two new properties to control how arrays are formatted.

  • array_format: Controls how primitive arrays are formatted. Options:
    • 'none': Ignore them (default and current behavior)
    • 'json': Output JSON-formatted value (the strategy employed in the work by @kapekost)
    • separator: Use the provided separator when joining the arrays to support things such as comma-separated lists.
  • array_key_format: Controls how arrays can be formatted as individual keys. Instead of picking a single format, this expects a format string using _ as a reserved-value for the array name and # as a reserved value for array "index". This allows arbitrary arrays to be formatted -- e.g. the JSON string {"my":{"array":{"key":[{"with_sub_obj":{"foo":"bar"}}]}}} becomes my/array/key/0/with_sub_obj/foo. The default behavior is to not output any keys, so the existing behavior is unchanged without additional configuration.

These values are described in the readme, and the code is isolated so it could easily be re-used in the discussed refactoring to allow for non-git configuration sources.

kapekost and others added 14 commits February 5, 2017 01:54
Current test suite always exits with zero because it's piped through
bunyan.  Since bunyan is always successful and it's the last command
on the pipe chain we get it's exit code.

By moving the command into a shell script and setting the pipefail
option for bash we will exit with the proper exit code.  This means
that Travis CI will correctly determine test failures.
Was trying to set the include_branch_name property on a variable that
didn't exist.  Which caused the test to fail with a syntax error.

Was using a property file but wasn't expecting the file extension to be
in the key path.
Part of moving from Cimpress to breser.
git 2.x seems to automatically have renames on and git2consul
doesn't know how to handle the R status.  Might be better
to handle all the statuses, but this appears to make it work with
newer versions of git without further changes so going with this.

Tested this with 1.9.1 and 2.14.1.

Noticed this since Travis CI now has 2.13.0
Fixed the cov command just like I'd done for the test command, so
that it will properly exit with an non-zero exit code on test failure.

Switched back to using the cov script for doing the tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

expand_keys support for JSON arrays
2 participants