How to get rid of charset from Content-Type header when Posting request to logstash using http sink

[msdotnet]

Step 1: Describe your environment

• Windows Server 2012

Step 2: Describe the problem

When sending Post request using http sink to logstash, data is being saved as string in the message field not as object. This is happening because of charset presence in the Content-Type header

Observed results

By default when Serilog is sending POST request to Logstash, it's adding "Content-type: application/json; charset=utf-8” to the request header. How can I just send just "Content-type: application/json” without charset

Expected results

Need to just get rid of charset from Content-Type header value when Posting request to logstash

[FantasticFiasco]

data is being saved as string in the message field not as object.

I don't follow you here. The message template and the rendered message is of type string, even in the serialized JSON. The JSON will also have a property called Properties holding the structured data. This is by design and has nothing to do with UTF-8 encoding of the request content.

Do you have any problems on the receiving side with UTF-8 as content encoding?

For a quick fix, you can always provide your own implementation of IHttpClient when creating the sink, there is already support for that.

[msdotnet]

Thanks for your valuable reply. Setting codec=> json in the logstash config file has resolved the issue.

[FantasticFiasco]

Now I understand what you meant, sorry that my answer wasn't more precise.

For a detailed example of sending log event to Elastic Stack, you can always take a look at my other repo called serilog-sinks-http-elastic-stack. It has the configuration you mentioned, and is why I apologize I didn't get that through to you almost two weeks ago.

[skasliwal-Liv]

@FantasticFiasco - I am facing the same issue while sending the data to Fluentbit . It is always adding the Charset in the content-type header and thats why its failing to send logs to Fluentbit Http input. is there any way to get rid of charset when sending the post request ?

[FantasticFiasco]

I think the runtime appends the charset to the header, because the code in this sink is setting the header to application/json (see source).

Since the sink isn't adding the charset, I'm at a loss on how to remove it.

[metalfork]

@FantasticFiasco the charset is being added via HttpHeaders. Opened an issue with fluent-bit