Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filter_log_to_metrics: Add setting "label" #7739

Merged
merged 4 commits into from
Sep 23, 2023
Merged

filter_log_to_metrics: Add setting "label" #7739

merged 4 commits into from
Sep 23, 2023

Conversation

wndhydrnt
Copy link
Contributor

@wndhydrnt wndhydrnt commented Jul 23, 2023 •
edited
Loading

Add setting label to filter "Log to Metrics". It allows a user to define a custom label name and a key from which the value of the label is read.

Addresses #7737.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

fluent/fluent-bit-docs#1167

Backporting

  • [N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 16:59 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 16:59 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 16:59 — with GitHub Actions Inactive
@wndhydrnt
Copy link
Contributor Author

Configuration:

[SERVICE]
    Flush     1
    Log_Level debug

[INPUT]
    Name  dummy
    Dummy {"message":"This is a test","kubernetes":{"labels":{"app":"testservice"}}}

[FILTER]
    Name               log_to_metrics
    Match              *
    Tag                test_metric
    Label              app $kubernetes['labels']['app']
    Metric_mode        counter
    Metric_name        count_all_dummy_messages
    Metric_description This metric counts dummy messages

[OUTPUT]
    Name  stdout
    Match *

[OUTPUT]
    Name  prometheus_exporter
    Match test_metric
    Host  0.0.0.0
    Port  2021

@wndhydrnt
Copy link
Contributor Author

wndhydrnt commented Jul 23, 2023 •
edited
Loading

Debug output:

fluent-bit@35b0d2ad42e2:/workspaces/fluent-bit/build$ ./bin/fluent-bit -c /fluent-bit/etc/fluent-bit.conf
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/07/23 17:19:17] [ info] Configuration:
[2023/07/23 17:19:17] [ info]  flush time     | 1.000000 seconds
[2023/07/23 17:19:17] [ info]  grace          | 5 seconds
[2023/07/23 17:19:17] [ info]  daemon         | 0
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  inputs:
[2023/07/23 17:19:17] [ info]      dummy
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  filters:
[2023/07/23 17:19:17] [ info]      log_to_metrics.0
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  outputs:
[2023/07/23 17:19:17] [ info]      stdout.0
[2023/07/23 17:19:17] [ info]      prometheus_exporter.1
[2023/07/23 17:19:17] [ info] ___________
[2023/07/23 17:19:17] [ info]  collectors:
[2023/07/23 17:19:17] [ info] [fluent bit] version=2.1.8, commit=, pid=11199
[2023/07/23 17:19:17] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2023/07/23 17:19:17] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/23 17:19:17] [ info] [cmetrics] version=0.6.3
[2023/07/23 17:19:17] [ info] [ctraces ] version=0.3.1
[2023/07/23 17:19:17] [ info] [input:dummy:dummy.0] initializing
[2023/07/23 17:19:17] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/07/23 17:19:17] [debug] [dummy:dummy.0] created event channels: read=21 write=22
[2023/07/23 17:19:17] [ info] [input:emitter:emitter.1] initializing
[2023/07/23 17:19:17] [ info] [input:emitter:emitter.1] storage_strategy='memory' (memory only)
[2023/07/23 17:19:17] [debug] [emitter:emitter.1] created event channels: read=23 write=24
[2023/07/23 17:19:17] [debug] [stdout:stdout.0] created event channels: read=25 write=26
[2023/07/23 17:19:17] [debug] [prometheus_exporter:prometheus_exporter.1] created event channels: read=32 write=33
[2023/07/23 17:19:17] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/23 17:19:17] [ info] [output:prometheus_exporter:prometheus_exporter.1] listening iface=0.0.0.0 tcp_port=2021
[2023/07/23 17:19:17] [debug] [router] match rule dummy.0:stdout.0
[2023/07/23 17:19:17] [debug] [router] match rule emitter.1:stdout.0
[2023/07/23 17:19:17] [ info] [sp] stream processor started
[2023/07/23 17:19:18] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:18] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[0] dummy.0: [[1690132758.358001759, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:19] [debug] [task] created task=0xffff98038910 id=0 OK
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:19:19] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:19] [debug] [task] created task=0xffff98038990 id=1 OK
[2023/07/23 17:19:19] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:19:19] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:19] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:19] [debug] [task] destroy task=0xffff98038910 (task_id=0)
2023-07-23T17:19:18.358371342Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 1
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:19:19] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:19:19] [debug] [task] destroy task=0xffff98038990 (task_id=1)
[0] dummy.0: [[1690132759.358301218, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:20] [debug] [task] created task=0xffff98075ad0 id=0 OK
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=2
[2023/07/23 17:19:20] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:20] [debug] [task] created task=0xffff98075b50 id=1 OK
[2023/07/23 17:19:20] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
2023-07-23T17:19:19.358409093Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 2
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=3
[2023/07/23 17:19:20] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:20] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:20] [debug] [task] destroy task=0xffff98075ad0 (task_id=0)
[2023/07/23 17:19:20] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:19:20] [debug] [task] destroy task=0xffff98075b50 (task_id=1)
[2023/07/23 17:19:21] [debug] [task] created task=0xffff98045950 id=0 OK
[2023/07/23 17:19:21] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:21] [debug] [task] created task=0xffff980459d0 id=1 OK
[2023/07/23 17:19:21] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[0] dummy.0: [[1690132760.357341760, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:19:20.357457968Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 3
[2023/07/23 17:19:21] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=4
[2023/07/23 17:19:21] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=5
[2023/07/23 17:19:21] [debug] [out flush] cb_destroy coro_id=2
[2023/07/23 17:19:21] [debug] [task] destroy task=0xffff98045950 (task_id=0)
[2023/07/23 17:19:21] [debug] [task] destroy task=0xffff980459d0 (task_id=1)
[0] dummy.0: [[1690132761.355765760, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:22] [debug] [task] created task=0xffff980448d0 id=0 OK
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=6
[2023/07/23 17:19:22] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:22] [debug] [task] created task=0xffff980459d0 id=1 OK
[2023/07/23 17:19:22] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
2023-07-23T17:19:21.356098510Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 4
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=7
[2023/07/23 17:19:22] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:22] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:22] [debug] [task] destroy task=0xffff980448d0 (task_id=0)
[2023/07/23 17:19:22] [debug] [out flush] cb_destroy coro_id=3
[2023/07/23 17:19:22] [debug] [task] destroy task=0xffff980459d0 (task_id=1)
[2023/07/23 17:19:23] [debug] [task] created task=0xffff980459d0 id=0 OK
[2023/07/23 17:19:23] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] dummy.0: [[1690132762.358517552, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=8
2023-07-23T17:19:22.358604219Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 5
[2023/07/23 17:19:23] [debug] [task] created task=0xffff98045f20 id=1 OK
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=9
[2023/07/23 17:19:23] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:19:23] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:19:23] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:19:23] [debug] [task] destroy task=0xffff980459d0 (task_id=0)
[2023/07/23 17:19:23] [debug] [out flush] cb_destroy coro_id=4
[2023/07/23 17:19:23] [debug] [task] destroy task=0xffff98045f20 (task_id=1)
^C[2023/07/23 17:19:24] [engine] caught signal (SIGINT)
[2023/07/23 17:19:24] [debug] [task] created task=0xffff980459d0 id=0 OK
[2023/07/23 17:19:24] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:19:24] [debug] [task] created task=0xffff98045e70 id=1 OK
[2023/07/23 17:19:24] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[0] dummy.0: [[1690132763.360203886, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:19:23.360298136Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 6
[2023/07/23 17:19:24] [ warn] [engine] service will shutdown in max 5 seconds
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=10
[2023/07/23 17:19:24] [ info] [input] pausing dummy.0
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=11
[2023/07/23 17:19:24] [debug] [task] destroy task=0xffff980459d0 (task_id=0)
[2023/07/23 17:19:24] [debug] [out flush] cb_destroy coro_id=5
[2023/07/23 17:19:24] [debug] [task] destroy task=0xffff98045e70 (task_id=1)
[2023/07/23 17:19:24] [ info] [engine] service has stopped (0 pending tasks)
[2023/07/23 17:19:24] [ info] [input] pausing dummy.0
[2023/07/23 17:19:24] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/07/23 17:19:24] [ info] [output:stdout:stdout.0] thread worker #0 stopped

@wndhydrnt
Copy link
Contributor Author

Valgrind output:

$ valgrind --leak-check=full ./bin/fluent-bit -c ../.dev/fluent-bit.conf
==11206== Memcheck, a memory error detector
==11206== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==11206== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==11206== Command: ./bin/fluent-bit -c ../.dev/fluent-bit.conf
==11206==
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/07/23 17:24:23] [ info] Configuration:
[2023/07/23 17:24:23] [ info]  flush time     | 1.000000 seconds
[2023/07/23 17:24:23] [ info]  grace          | 5 seconds
[2023/07/23 17:24:23] [ info]  daemon         | 0
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  inputs:
[2023/07/23 17:24:23] [ info]      dummy
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  filters:
[2023/07/23 17:24:23] [ info]      log_to_metrics.0
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  outputs:
[2023/07/23 17:24:23] [ info]      stdout.0
[2023/07/23 17:24:23] [ info]      prometheus_exporter.1
[2023/07/23 17:24:23] [ info] ___________
[2023/07/23 17:24:23] [ info]  collectors:
[2023/07/23 17:24:23] [ info] [fluent bit] version=2.1.8, commit=, pid=11206
[2023/07/23 17:24:23] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2023/07/23 17:24:23] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/23 17:24:23] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/23 17:24:23] [ info] [cmetrics] version=0.6.3
[2023/07/23 17:24:23] [ info] [ctraces ] version=0.3.1
[2023/07/23 17:24:23] [ info] [input:dummy:dummy.0] initializing
[2023/07/23 17:24:23] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/07/23 17:24:23] [debug] [dummy:dummy.0] created event channels: read=21 write=22
[2023/07/23 17:24:23] [ info] [input:emitter:emitter.1] initializing
[2023/07/23 17:24:23] [ info] [input:emitter:emitter.1] storage_strategy='memory' (memory only)
[2023/07/23 17:24:23] [debug] [emitter:emitter.1] created event channels: read=23 write=24
[2023/07/23 17:24:23] [debug] [stdout:stdout.0] created event channels: read=25 write=26
[2023/07/23 17:24:23] [debug] [prometheus_exporter:prometheus_exporter.1] created event channels: read=32 write=33
[2023/07/23 17:24:24] [ info] [output:prometheus_exporter:prometheus_exporter.1] listening iface=0.0.0.0 tcp_port=2021
[2023/07/23 17:24:24] [debug] [router] match rule dummy.0:stdout.0
[2023/07/23 17:24:24] [debug] [router] match rule emitter.1:stdout.0
[2023/07/23 17:24:24] [ info] [sp] stream processor started
[2023/07/23 17:24:24] [debug] [input chunk] update output instances with new chunk size diff=282, records=0, input=emitter.1
[2023/07/23 17:24:24] [debug] [input chunk] update output instances with new chunk size diff=81, records=1, input=dummy.0
[2023/07/23 17:24:25] [debug] [task] created task=0x52d0a30 id=0 OK
[0] dummy.0: [[1690133064.382956803, {}], {"message"=>"This is a test", "kubernetes"=>{"labels"=>{"app"=>"testservice"}}}]
2023-07-23T17:24:24.424676637Z log_metric_counter_count_all_dummy_messages{app="testservice"} = 1
[2023/07/23 17:24:25] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:24:25] [debug] [task] created task=0x52d0e80 id=1 OK
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=1
[2023/07/23 17:24:25] [debug] [output:stdout:stdout.0] task_id=1 assigned to thread #0
[2023/07/23 17:24:25] [debug] [task] destroy task=0x52d0a30 (task_id=0)
[2023/07/23 17:24:25] [debug] [out flush] cb_destroy coro_id=0
[2023/07/23 17:24:25] [debug] [task] destroy task=0x52d0e80 (task_id=1)
^C[2023/07/23 17:24:28] [engine] caught signal (SIGINT)
[2023/07/23 17:24:28] [ warn] [engine] service will shutdown in max 5 seconds
[2023/07/23 17:24:28] [ info] [input] pausing dummy.0
[2023/07/23 17:24:29] [ info] [engine] service has stopped (0 pending tasks)
[2023/07/23 17:24:29] [ info] [input] pausing dummy.0
[2023/07/23 17:24:29] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/07/23 17:24:29] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==11206==
==11206== HEAP SUMMARY:
==11206==     in use at exit: 0 bytes in 0 blocks
==11206==   total heap usage: 2,582 allocs, 2,582 frees, 1,950,769 bytes allocated
==11206==
==11206== All heap blocks were freed -- no leaks are possible
==11206==
==11206== For lists of detected and suppressed errors, rerun with: -s
==11206== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

wndhydrnt added 4 commits July 23, 2023 19:27
@wndhydrnt
Fix formatting
d69485b 
Signed-off-by: Markus Meyer <[email protected]>
@wndhydrnt
Implement
bdbd88f 
Signed-off-by: Markus Meyer <[email protected]>
@wndhydrnt
Simplify test
18d60e1 
Signed-off-by: Markus Meyer <[email protected]>
@wndhydrnt
Fix issues highlighted by valgrind
ed60c66 
Signed-off-by: Markus Meyer <[email protected]>
@wndhydrnt wndhydrnt force-pushed the log_to_metrics-label-key branch from 5806941 to ed60c66 Compare July 23, 2023 17:27
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 17:27 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 17:27 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 17:27 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt temporarily deployed to pr July 23, 2023 17:53 — with GitHub Actions Inactive
@wndhydrnt wndhydrnt mentioned this pull request Jul 23, 2023
Merged
@wndhydrnt wndhydrnt marked this pull request as ready for review July 23, 2023 18:02
@edsiper
Copy link
Member

edsiper commented Sep 23, 2023

merging this PR, note that I will rename label option to add_label to be compliant with other plugins that performs the same operation

@edsiper edsiper merged commit b18587b into fluent:master Sep 23, 2023
@wndhydrnt wndhydrnt deleted the log_to_metrics-label-key branch September 24, 2023 10:10
@wndhydrnt wndhydrnt mentioned this pull request Sep 24, 2023
Closed
4 tasks
@wndhydrnt
Copy link
Contributor Author

Thank you for the review and the merge @edsiper. Unfortunately, some lines were missed in #7964. I have created #7967 to address the issue. Please take a look.

fluent/fluent-bit-docs#1167 has been updated as well to reflect the rename.

@Porkepix Porkepix mentioned this pull request Sep 28, 2023
Merged
leonardo-albertovich pushed a commit that referenced this pull request Oct 5, 2023
@wndhydrnt @leonardo-albertovich
filter_log_to_metrics: Add setting "label" (#7739)
43ebe08 
---------

Signed-off-by: Markus Meyer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edsiper edsiper Awaiting requested review from edsiper edsiper is a code owner

@leonardo-albertovich leonardo-albertovich Awaiting requested review from leonardo-albertovich leonardo-albertovich is a code owner

@fujimotos fujimotos Awaiting requested review from fujimotos fujimotos is a code owner

@koleini koleini Awaiting requested review from koleini koleini is a code owner

Assignees
No one assigned
Labels
docs-required
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wndhydrnt @edsiper