Plans for a support of custom, configurable loggers

[marcindulak]

I have questions related to logging in the upcoming bentoml 1.0

1. Are there plans for json logging?
   I'm using one of the log aggregators, which recognizes json logs out-of-the box, however requires writing custom parsers for text logs. For example, the following log emitted by bentoml appears the log aggregator as the content, without the message and INFO level identified separately

   [11:10:40] INFO     [boot] Starting production BentoServer from "."
   

   however, the json log below is ingested properly, with the "message" and "level" recognized

   {"message":"Starting production BentoServer from \".\"","level":"INFO"}
   

   I looks like the support for custom loggers has been removed following Unable to change prediction log format #1464,
   however there is a recent work on logging Service and Access Logging #2149.

2. How to reduce the amount of logged information?
   When starting a docker container

   git clone https://github.com/bentoml/gallery
   cd gallery/quickstart/
   git checkout c03e7604581265fc81247556a455cce11d03f686
   python3 -m venv venv
   source venv/bin/activate
   pip install bentoml==1.0.0a4 scikit-learn==1.0.2
   python train.py
   bentoml build
   bentoml containerize iris_classifier
   docker run -p 5000:5000 --rm --name bentoml-classifier iris_classifier:4bnlfuurs6tnewk4
   

   A failed request, generates a log with ~400 lines of traceback

   curl -X POST "http://localhost:5000/classify" -H "accept: */*" -H "Content-Type: application/json" -d '{}'
   

   [15:27:45] ERROR    [api_server] [187075731332202942763861365475404691678]      
                       [16646629043245606249] Exception on /classify [POST]        
                       ╭─────────── Traceback (most recent call last) ────────────╮
   ...
   

3. How to log a full timestamp in a specific format?
   The above examples use a [HH:MM:SS] time-only format, while a more explicit iso-8601, which includes a full timestamp may be preferred.

[ssheng]

First of all, thanks for the great feedback, @marcindulak. Let me try to address each of the points you raised.

1. Are there plans for json logging?

The primary goals behind the service and access logging were to create human readable logs written to stdout. While we can certainly add a configurable option to output JSON formatted logs, we'd first like to understand your use cases. The proposed JSON schema will have three high level fields, timestamp, level, and message. Is the log aggregator applying filtering logic or building indices on any of these fields? Are there additional fields, e.g. requests, responses, you'd like to include in the JSON logs?

A common use case we envisioned is logging the prediction input and output, which is tracked under prediction events (https://github.com/bentoml/BentoML/issues/2196).

2. How to reduce the amount of logged information?

The api_server and runner are current configured to log the full exception stack traces generated from runtime. Do you think the stack traces are useful? We can potentially add an option to omit stack traces in --production mode.

3. How to log a full timestamp in a specific format?

We will standardize timestamp format before the official 1.0 release.

We are happy to jump on a chat with you to learn your requirements on logging. Please let us know if any time works for you.

[marcindulak]

First of all, thanks for the great feedback, @marcindulak. Let me try to address each of the points you raised.

1. Are there plans for json logging?

The primary goals behind the service and access logging were to create human readable logs written to stdout. While we can certainly add a configurable option to output JSON formatted logs, we'd first like to understand your use cases. The proposed JSON schema will have three high level fields, timestamp, level, and message. Is the log aggregator applying filtering logic or building indices on any of these fields? Are there additional fields, e.g. requests, responses, you'd like to include in the JSON logs?

A common use case we envisioned is logging the prediction input and output, which is tracked under prediction events (#2196).

The use case I'm thinking about, is to have the log format of bentoml as close as possible to other, custom developed applications, which use bentoml as API endpoints. A customizable log format, like in bobbui/json-logging-python#13 would provide most flexibility. In my case a json log which contains timestamp,level, and message, plus some bentoml-specific fields would be probably by default ingested by the aggregator, but in general other field names like '@timestamp' may be needed by the aggregators.
The aggregator may index the contents of the message and other specified fields, and allow for filtering or faceting.

2. How to reduce the amount of logged information?

The api_server and runner are current configured to log the full exception stack traces generated from runtime. Do you think the stack traces are useful? We can potentially add an option to omit stack traces in --production mode.

A customizable setting of the log level would be more flexible. A lightly used production service or a service being debugged in production may support or require detailed debug logs.

3. How to log a full timestamp in a specific format?

We will standardize timestamp format before the official 1.0 release.

We are happy to jump on a chat with you to learn your requirements on logging. Please let us know if any time works for you.

[ssheng]

Standardized logging format in PR #2298.