HOME
These are aggregation plugins for fluentd(see document for detail). You can easy aggregate(sum group by) any key pairs. You need not to write any codes, only configuration.
The traditional approach such as hadoop is called "store-first, query-later". Shipping all the data from the servers consumes a great deal of bandwidth, so this puts a lot of stress on the network.
On the other hand, our approach performs aggregation on each server after some interval(per day, hour, minute and second). And we use MongoDB (or Redis) for aggregated data strage because they go well with messagepack or json format.
see configuration.
see example.