Olivier Langlois's blog

When I start an application with glc-capture, the application become frozen and unresponsive.
It even ignores kill -11 signals!

With gdb, I have found that glc blocks inside a dlopen() call.


(gdb) where
#0 0xf76ff430 in __kernel_vsyscall ()
#1 0xf7591231 in __lll_lock_wait_private () from /usr/lib32/libc.so.6
#2 0xf750fd8a in _L_lock_6856 () from /usr/lib32/libc.so.6
#3 0xf750d77d in malloc () from /usr/lib32/libc.so.6
#4 0xf770d05b in _dl_map_object_deps () from /lib/ld-linux.so.2
#5 0xf7712f8b in dl_open_worker () from /lib/ld-linux.so.2
#6 0xf770ee1a in _dl_catch_error () from /lib/ld-linux.so.2
#7 0xf7712954 in _dl_open () from /lib/ld-linux.so.2
#8 0xf75bb67b in do_dlopen () from /usr/lib32/libc.so.6
#9 0xf770ee1a in _dl_catch_error () from /lib/ld-linux.so.2
#10 0xf75bb76b in dlerror_run () from /usr/lib32/libc.so.6
#11 0xf75bb7f1 in __libc_dlopen_mode () from /usr/lib32/libc.so.6
#12 0xf7591ad8 in init () from /usr/lib32/libc.so.6
#13 0xf769c0ee in pthread_once () from /usr/lib32/libpthread.so.0
#14 0xf7591d45 in backtrace () from /usr/lib32/libc.so.6
#15 0xf74adc63 in backtrace_and_maps () from /usr/lib32/libc.so.6
#16 0xf7504263 in __libc_message () from /usr/lib32/libc.so.6
#17 0xf750a3ca in malloc_printerr () from /usr/lib32/libc.so.6
#18 0xf750be11 in _int_malloc () from /usr/lib32/libc.so.6
#19 0xf750d788 in malloc () from /usr/lib32/libc.so.6
#20 0xf770d05b in _dl_map_object_deps () from /lib/ld-linux.so.2
#21 0xf7712f8b in dl_open_worker () from /lib/ld-linux.so.2
#22 0xf770ee1a in _dl_catch_error () from /lib/ld-linux.so.2
#23 0xf7712954 in _dl_open () from /lib/ld-linux.so.2
#24 0xf768bcbc in ?? () from /usr/lib32/libdl.so.2
#25 0xf770ee1a in _dl_catch_error () from /lib/ld-linux.so.2
#26 0xf768c37c in ?? () from /usr/lib32/libdl.so.2
#27 0xf768bd71 in dlopen () from /usr/lib32/libdl.so.2
#28 0xf76c6077 in get_real_alsa () at /home/lano1106/dev/lib32-glc/src/glc/src/hook/alsa.c:256
#29 0xf76c64d9 in alsa_init (glc=glc@entry=0xf76cd280 <mpriv>) at /home/lano1106/dev/lib32-glc/src/glc/src/hook/alsa.c:96
#30 0xf76c29ab in init_glc () at /home/lano1106/dev/lib32-glc/src/glc/src/hook/main.c:114
#31 0xf76c56f5 in __opengl_glXGetProcAddressARB (proc_name=0x928a8c2 "glDeleteFencesNV") at /home/lano1106/dev/lib32-glc/src/glc/src/hook/opengl.c:332
#32 0xf76ee80c in glXGetProcAddressARB () from /home/lano1106/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so


I was suspicious about malloc_printerr() call so I went to glibc malloc source code. BTW, I have found something else interesting to share in my next post concerning glibc heap management. My suspicion was right. When you see malloc_printerr() in a stack, it is pretty much game over as it means that malloc() has detected memory corruption in the heap.
Since it is happening even before main() invoked when shared libraries are loaded, even __libc_message() needs something to be loaded by the dynamic loader which will reenter malloc and auto self deadlock itself!

Fortunately, I am entering this state only when using a particular glc command line option, hence it should not be too hard to find the offending code.

Update: I have found the bug!
https://github.com/nullkey/glc/pull/62