Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log output is not expected, need to remove some unused info #9847

Open
wwyhy opened this issue Jan 17, 2025 · 2 comments
Open

Log output is not expected, need to remove some unused info #9847

wwyhy opened this issue Jan 17, 2025 · 2 comments

Comments

@wwyhy
Copy link

wwyhy commented Jan 17, 2025

Bug Report

Describe the bug
Log output is not expected, (remove time kv, and message key, only keep value of message)

To Reproduce

  • Can not be reproduce by Rubular

Current behavior
2025-01-17 17:24:50.363
time="2025-01-17T09:24:50.363234193Z" message="{"level":"info","time":"2025-01-17T09:24:50.363146Z","scope":"xdsproxy","msg":"connected to delta upstream XDS server: istiod-canary.istio-system.svc:15012","id":539}"

Expected behavior
2025-01-17 17:24:50.363
{"level":"info","time":"2025-01-17T09:24:50.363146Z","scope":"xdsproxy","msg":"connected to delta upstream XDS server: istiod-canary.istio-system.svc:15012","id":539}

Screenshots

Pod Log:
Image

Loki Query result:
Image

Your Environment

  • Version used: 3.2.4
  • Environment name and version (Kubernetes 1.30.3):
  • Configuration, Filters and plugins:
Image

Thanks!

@patrick-stephens
Copy link
Contributor

Do not make your own docker or CRI parsers - use the built in ones particularly for K8S: https://chronosphere.io/learn/fluent-bit-kubernetes-filter/#Troubleshooting%20misconfiguration%20woes

Does it happen with the built-in CRI parser using the recommended configuration in the docs?
https://docs.fluentbit.io/manual/pipeline/inputs/tail#multiline-and-containers-v1.8

[INPUT]
    name              tail
    path              /var/log/containers/*.log
    multiline.parser  docker, cri

@wwyhy
Copy link
Author

wwyhy commented Jan 20, 2025

@patrick-stephens

Sorry for my incorrect description.

I upgraded fluentbit from v2.x to 3.2.4, then below content is added to each of line, now I want to remove them, only keep content of message. how to remove them? thanks!

time="2025-01-17T09:24:50.363234193Z" message=""

@wwyhy wwyhy changed the title a double quote " is appended on end of each line Log formart is not expected, need to remove some unused info Jan 20, 2025
@wwyhy wwyhy changed the title Log formart is not expected, need to remove some unused info Log output is not expected, need to remove some unused info Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants