Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_docker: Implement a capability for cgroups V2 #8087

Merged
merged 1 commit into from
Oct 26, 2023

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Oct 26, 2023

Currently, in_docker does not provide cgroups v2.
In this PR, I added a capability to handle cgroups v2 metrics for Docker.

This is related for #5949.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
$  sudo bin/fluent-bit -i docker -o stdout 
  • Debug log output from testing the change

In other terminals:

$ docker run -it --rm ubuntu:jammy bash
$ docker run -it --memory 300MB --rm debian:bullseye bash

fluent-bit log:

Fluent Bit v2.2.0
* Copyright (C) 2015-2023 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/10/26 15:18:53] [ info] Configuration:
[2023/10/26 15:18:53] [ info]  flush time     | 1.000000 seconds
[2023/10/26 15:18:53] [ info]  grace          | 5 seconds
[2023/10/26 15:18:53] [ info]  daemon         | 0
[2023/10/26 15:18:53] [ info] ___________
[2023/10/26 15:18:53] [ info]  inputs:
[2023/10/26 15:18:53] [ info]      docker
[2023/10/26 15:18:53] [ info] ___________
[2023/10/26 15:18:53] [ info]  filters:
[2023/10/26 15:18:53] [ info] ___________
[2023/10/26 15:18:53] [ info]  outputs:
[2023/10/26 15:18:53] [ info]      stdout.0
[2023/10/26 15:18:53] [ info] ___________
[2023/10/26 15:18:53] [ info]  collectors:
[2023/10/26 15:18:53] [ info] [fluent bit] version=2.2.0, commit=4acd344399, pid=335975
[2023/10/26 15:18:53] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2023/10/26 15:18:53] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/10/26 15:18:53] [ info] [cmetrics] version=0.6.4
[2023/10/26 15:18:53] [ info] [ctraces ] version=0.3.1
[2023/10/26 15:18:53] [ info] [input:docker:docker.0] initializing
[2023/10/26 15:18:53] [ info] [input:docker:docker.0] storage_strategy='memory' (memory only)
[2023/10/26 15:18:53] [debug] [docker:docker.0] created event channels: read=21 write=22
[2023/10/26 15:18:53] [ info] [input:docker:docker.0] Detected cgroups v2
[2023/10/26 15:18:53] [debug] [stdout:stdout.0] created event channels: read=23 write=24
[2023/10/26 15:18:53] [ info] [sp] stream processor started
[2023/10/26 15:18:53] [ info] [output:stdout:stdout.0] worker #0 started
[2023/10/26 15:18:54] [debug] [input chunk] update output instances with new chunk size diff=109, records=1, input=docker.0
[2023/10/26 15:18:54] [debug] [input chunk] update output instances with new chunk size diff=112, records=1, input=docker.0
[2023/10/26 15:18:55] [debug] [task] created task=0x7f13e801ece0 id=0 OK
[2023/10/26 15:18:55] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] docker.0: [[1698301134.146797664, {}], {"id"=>"1a38a7677732", "name"=>"confident_snyder", "cpu_used"=>22631, "mem_used"=>823296, "mem_limit"=>314572800}]
[1] docker.0: [[1698301134.147009000, {}], {"id"=>"043b521509b6", "name"=>"objective_bassi", "cpu_used"=>53562, "mem_used"=>1208320, "mem_limit"=>18446744073709551615}]
[2023/10/26 15:18:55] [debug] [out flush] cb_destroy coro_id=0
[2023/10/26 15:18:55] [debug] [input chunk] update output instances with new chunk size diff=109, records=1, input=docker.0
[2023/10/26 15:18:55] [debug] [input chunk] update output instances with new chunk size diff=112, records=1, input=docker.0
[2023/10/26 15:18:55] [debug] [task] destroy task=0x7f13e801ece0 (task_id=0)
^C[2023/10/26 15:18:55] [engine] caught signal (SIGINT)
[2023/10/26 15:18:55] [debug] [task] created task=0x7f13e801f3e0 id=0 OK
[2023/10/26 15:18:55] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/10/26 15:18:55] [ warn] [engine] service will shutdown in max 5 seconds
[2023/10/26 15:18:55] [ info] [input] pausing docker.0
[0] docker.0: [[1698301135.146551719, {}], {"id"=>"1a38a7677732", "name"=>"confident_snyder", "cpu_used"=>22631, "mem_used"=>823296, "mem_limit"=>314572800}]
[1] docker.0: [[1698301135.146633514, {}], {"id"=>"043b521509b6", "name"=>"objective_bassi", "cpu_used"=>53562, "mem_used"=>1208320, "mem_limit"=>18446744073709551615}]
[2023/10/26 15:18:55] [debug] [out flush] cb_destroy coro_id=1
[2023/10/26 15:18:55] [debug] [task] destroy task=0x7f13e801f3e0 (task_id=0)
[2023/10/26 15:18:56] [ info] [engine] service has stopped (0 pending tasks)
[2023/10/26 15:18:56] [ info] [input] pausing docker.0
[2023/10/26 15:18:56] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/10/26 15:18:56] [ info] [output:stdout:stdout.0] thread worker #0 stopped
  • Attached Valgrind output that shows no leaks or memory corruption was found
==339923== 
==339923== HEAP SUMMARY:
==339923==     in use at exit: 0 bytes in 0 blocks
==339923==   total heap usage: 2,933 allocs, 2,933 frees, 838,012 bytes allocated
==339923== 
==339923== All heap blocks were freed -- no leaks are possible
==339923== 
==339923== For lists of detected and suppressed errors, rerun with: -s
==339923== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@cosmo0920 cosmo0920 temporarily deployed to pr October 26, 2023 06:30 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 26, 2023 06:30 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 26, 2023 06:30 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 26, 2023 07:01 — with GitHub Actions Inactive
@edsiper edsiper merged commit d98e273 into master Oct 26, 2023
42 of 44 checks passed
@edsiper edsiper deleted the cosmo0920-detect-and-use-cgroups-v2-in_docker branch October 26, 2023 18:06
@edsiper
Copy link
Member

edsiper commented Oct 26, 2023

@cosmo0920 thank you, let's add unit tests in a different PR, you can use a hard copy of sysfs/procfs and custom paths to them to test in a runtime environment

@cosmo0920 cosmo0920 restored the cosmo0920-detect-and-use-cgroups-v2-in_docker branch October 27, 2023 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants