Feature Proposal: Logger options in riak.conf

[martinsumner]

Background

The current riak.conf schema allows the log format to modified within logger, as well as the destination. The logger configuration that gets produced is not, though, backwards compatible with previous releases - i.e. crash.log etc.

There is also the issue of metric style logs, that are logged on a frequent heartbeat: these were introduced in-line with the observability strategy at one customer. Some components, especially the leveled backend, produce a significant number of logs when compared to other riak elements. Also it would be beneficial to stop misusing io:format and erlang.log for logging from application components.

There is a need for more flexible and configurable logging, and ways to divert relatively verbose logging components away from the main console.log.

Proposal

There exists riak_logger which will allow for a backwards compatible configuration to be provided. So it would be good if it were possible to produce this output from the default riak.conf (and for it to be the default).

The proposal is to extend riak_logger to allow for individual components, by setting a log_type as log metadata to then be subject to a policy of:

• log (log as normal to console.log);
• divert (log to another log file);
• ignore (just not be logged).

It should be configurable within riak to extend riak_logger through different configuration functions, so that different outputs can be produced from the configuration items.

Design

For the configuration the conversion of config items to logger config was managed through a translation function embedded in the riak.schema. There is now a configuration item, which will allow for a translation function to be selected from those available in the riak_logger_config module within the riak_logger application.

Alternative Design Ideas

There are some outstanding questions:

• Is the configurability of log format helpful? It may make support easier if the log format is fixed.
• Should it be possible to ignore backend and tictcaaae logs.
• How to test different configurations, is it enough to expand out eunit tests that confirm the configuration outputted from the function is as expected? It is hard to test anything driven by cuttlefish in riak_test.
• Leveled can have its log level set separately to the rest of Riak (https://github.com/martinsumner/leveled/blob/develop-3.4/priv/leveled.schema#L83-L96), but it also has the ability to change the level of individual log references (so that one can set the level to warn and promote interesting info log references to warn at startup). This isn't configurable though (really hard to do in cuttlefish) - should it be supported (even if only via advanced.config)?
• Should the log option to change the translation function be set to be nay arbitrary atom, so that new translation functions can be added by updating riak_logger, and without requiring a PR to riak.

Testing

...

Caveats

...

Pull Requests

OpenRiak/riak_logger#1
OpenRiak/riak#16

Planned Release for Inclusion

Riak 3.4.0

[martinsumner]

Pull requests:

OpenRiak/riak_logger#1

[tburghart]

There's a lengthier discussion to be had offline, after which we should update this.