Memory corruption in userland tool

[gnosek]

Freshly updated sysdig 0.1.87 (Ubuntu 12.04) has just failed on me with what looks like memory corruption. Happened just once, not sure if reproducible, though subsequent sysdig runs seem fine (this is a production machine I really don't want to kill in the middle of the day if the kernel part does something weird too).

No core dump or other info, sorry. dmesg looks clean (no suspicious messages from sysdig).

root@p21:~# sysdig proc.aname=dovecot and evt.buffer contains p2mail2
(a few dozen lines of legit output and then):

*** glibc detected *** sysdig: free(): invalid next size (normal): 0x0000000002cb3300 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f7d57512b96]
sysdig(_ZN20sinsp_thread_manager10add_threadER16sinsp_threadinfob+0x6b4)[0x462314]
sysdig(_ZN12sinsp_parser16parse_clone_exitEP9sinsp_evt+0x3fa)[0x45c95a]
sysdig(_ZN12sinsp_parser13process_eventEP9sinsp_evt+0x24b)[0x45ce0b]
sysdig(_ZN5sinsp4nextEPP9sinsp_evt+0x1a0)[0x463b20]
sysdig(_Z10do_inspectP5sinspmbbP12sinsp_filterPSt6vectorI19summary_table_entrySaIS4_EEP19sinsp_evt_formatter+0x7a)[0x4259aa]
sysdig(_Z11sysdig_initiPPc+0x190d)[0x427d4d]
sysdig(main+0x40)[0x41c7f0]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f7d574b576d]
sysdig[0x41cdb1]
======= Memory map: ========
00400000-0054c000 r-xp 00000000 09:03 1988643                            /usr/bin/sysdig
0074b000-0074d000 rw-p 0014b000 09:03 1988643                            /usr/bin/sysdig
012d6000-043ac000 rw-p 00000000 00:00 0                                  [heap]
7f7d55c58000-7f7d55e58000 r--s 00000000 00:05 6986041                    /dev/sysdig7
7f7d55e58000-7f7d56058000 r--s 00000000 00:05 6986038                    /dev/sysdig6
7f7d56058000-7f7d56258000 r--s 00000000 00:05 6986036                    /dev/sysdig5
7f7d56258000-7f7d56458000 r--s 00000000 00:05 6986034                    /dev/sysdig4
7f7d56458000-7f7d56658000 r--s 00000000 00:05 6986033                    /dev/sysdig3
7f7d56658000-7f7d56858000 r--s 00000000 00:05 6986032                    /dev/sysdig2
7f7d56858000-7f7d56a58000 r--s 00000000 00:05 6986030                    /dev/sysdig1
7f7d56a58000-7f7d56c58000 r--s 00000000 00:05 6986028                    /dev/sysdig0
7f7d56c58000-7f7d56c64000 r-xp 00000000 09:03 262611                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d56c64000-7f7d56e63000 ---p 0000c000 09:03 262611                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d56e63000-7f7d56e64000 r--p 0000b000 09:03 262611                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d56e64000-7f7d56e65000 rw-p 0000c000 09:03 262611                     /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d56e65000-7f7d56e6f000 r-xp 00000000 09:03 262615                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f7d56e6f000-7f7d5706f000 ---p 0000a000 09:03 262615                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f7d5706f000-7f7d57070000 r--p 0000a000 09:03 262615                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f7d57070000-7f7d57071000 rw-p 0000b000 09:03 262615                     /lib/x86_64-linux-gnu/libnss_nis-2.15.so
7f7d57071000-7f7d57088000 r-xp 00000000 09:03 262627                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f7d57088000-7f7d57287000 ---p 00017000 09:03 262627                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f7d57287000-7f7d57288000 r--p 00016000 09:03 262627                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f7d57288000-7f7d57289000 rw-p 00017000 09:03 262627                     /lib/x86_64-linux-gnu/libnsl-2.15.so
7f7d57289000-7f7d5728b000 rw-p 00000000 00:00 0 
7f7d5728b000-7f7d57293000 r-xp 00000000 09:03 262609                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f7d57293000-7f7d57492000 ---p 00008000 09:03 262609                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f7d57492000-7f7d57493000 r--p 00007000 09:03 262609                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f7d57493000-7f7d57494000 rw-p 00008000 09:03 262609                     /lib/x86_64-linux-gnu/libnss_compat-2.15.so
7f7d57494000-7f7d57649000 r-xp 00000000 09:03 262607                     /lib/x86_64-linux-gnu/libc-2.15.so
7f7d57649000-7f7d57849000 ---p 001b5000 09:03 262607                     /lib/x86_64-linux-gnu/libc-2.15.so
7f7d57849000-7f7d5784d000 r--p 001b5000 09:03 262607                     /lib/x86_64-linux-gnu/libc-2.15.so
7f7d5784d000-7f7d5784f000 rw-p 001b9000 09:03 262607                     /lib/x86_64-linux-gnu/libc-2.15.so
7f7d5784f000-7f7d57854000 rw-p 00000000 00:00 0 
7f7d57854000-7f7d57869000 r-xp 00000000 09:03 262223                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d57869000-7f7d57a68000 ---p 00015000 09:03 262223                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d57a68000-7f7d57a69000 r--p 00014000 09:03 262223                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d57a69000-7f7d57a6a000 rw-p 00015000 09:03 262223                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d57a6a000-7f7d57b65000 r-xp 00000000 09:03 262618                     /lib/x86_64-linux-gnu/libm-2.15.so
7f7d57b65000-7f7d57d64000 ---p 000fb000 09:03 262618                     /lib/x86_64-linux-gnu/libm-2.15.so
7f7d57d64000-7f7d57d65000 r--p 000fa000 09:03 262618                     /lib/x86_64-linux-gnu/libm-2.15.so
7f7d57d65000-7f7d57d66000 rw-p 000fb000 09:03 262618                     /lib/x86_64-linux-gnu/libm-2.15.so
7f7d57d66000-7f7d57e48000 r-xp 00000000 09:03 1966917                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f7d57e48000-7f7d58047000 ---p 000e2000 09:03 1966917                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f7d58047000-7f7d5804f000 r--p 000e1000 09:03 1966917                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f7d5804f000-7f7d58051000 rw-p 000e9000 09:03 1966917                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
7f7d58051000-7f7d58066000 rw-p 00000000 00:00 0 
7f7d58066000-7f7d58068000 r-xp 00000000 09:03 262622                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d58068000-7f7d58268000 ---p 00002000 09:03 262622                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d58268000-7f7d58269000 r--p 00002000 09:03 262622                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d58269000-7f7d5826a000 rw-p 00003000 09:03 262622                     /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d5826a000-7f7d5828c000 r-xp 00000000 09:03 262619                     /lib/x86_64-linux-gnu/ld-2.15.so
7f7d5847b000-7f7d5847d000 rw-p 00000000 00:00 0 
7f7d5847d000-7f7d5847e000 rw-s 28f064000 00:05 6986041                   /dev/sysdig7
7f7d5847e000-7f7d5847f000 rw-s 74a3aa000 00:05 6986038                   /dev/sysdig6
7f7d5847f000-7f7d58480000 rw-s 4c0abc000 00:05 6986036                   /dev/sysdig5
7f7d58480000-7f7d58485000 rw-p 00000000 00:00 0 
7f7d58485000-7f7d58486000 rw-s 7df7fd000 00:05 6986034                   /dev/sysdig4
7f7d58486000-7f7d58487000 rw-s 7bba0d000 00:05 6986033                   /dev/sysdig3Aborted

[ldegio]

Is this something that happens to you regularly?
I've recently done several fixes in that section of the code, so if you can replicate the issue, could you see if a build from master fixes it?

[gnosek]

Happened exactly once so far, not sure how to replicate it.

[arossert]

I have the same issue when using the sinsp class directly in my program.

this simple example replicate it:

#include <sinsp.h>
void main() {
    sinsp *inspector = new sinsp();
    delete inspector;
}

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.