Pin process crashes before opening a socket for gdb

Question

When I run Intel pin with my custom pin-tool, for some reason it crashes on a segfault, before even starting the application under test. It happens for one application, even though, the same setup works for another application.

Here is an example of a successful run:

$ unset HOME && TEST_FILE=test000001.test pin -appdebug -t /home/necto/pin-trace.so -- ./executable1 <args to the executable>
Application stopped until continued from debugger.
Start GDB, then issue this command at the (gdb) prompt:
  target remote :42312

( unset HOME is there for the application's sake) Here is an example of an unsuccessful run:

$ unset HOME && TEST_FILE=test000001.test pin -appdebug -t /home/necto/pin-trace.so -- ./executable2 <args to the executable>
C: Tool (or Pin) caused signal 11 at PC 0x000000000
Segmentation fault

Note, that it does not even open a socket for gdb to attach to.

When running it directly under gdb, it seems to fail differently (on SIGUSR1 ):

$ unset HOME && TEST_FILE=test000001.test gdb --args pin -appdebug -t /home/necto/pin-trace.so -- ./executable2 <args to the executable>
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from pin...(no debugging symbols found)...done.
(gdb) r
Starting program: /home/necto/pin/pin -appdebug -t /home/necto/pin-trace.so -- ./executable2 <args to the executable>
process 185838 is executing new program: /home/necto/pin/intel64/bin/pinbin

Program received signal SIGUSR1, User defined signal 1.
0x00007ffff7edba1b in OS_BARESYSCALL_DoCallAsmIntel64Linux () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
(gdb) bt
#0  0x00007ffff7edba1b in OS_BARESYSCALL_DoCallAsmIntel64Linux () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
#1  0x00007fffffffd3d0 in ?? ()
#2  0x00007ffff7edbb53 in OS_SyscallDo () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
#3  0x00007ffff7eda4a3 in OS_SendSignalToThread () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
#4  0x00007ffff7ed8f8a in OS_RaiseException () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
#5  0x00007ffff7e87dad in raise () from /home/necto/pin/intel64/runtime/pincrt/libc-dynamic.so
#6  0x00005555558e747e in ?? ()
#7  0x00005555558e757e in LEVEL_INJECTOR::DoSystemChecks() ()
#8  0x00005555558db0ae in LEVEL_INJECTOR::UNIX_INJECTOR::Run() ()
#9  0x00005555558e0695 in LEVEL_INJECTOR::PIN_UNIX_ENVIRONMENT::LaunchPin() ()
#10 0x00005555558c8be5 in LEVEL_INJECTOR::PIN_ENVIRONMENT::Main() ()
#11 0x0000555555657cf9 in main ()
(gdb) 

The backtrace looks like nothing familiar. How can I find out the cause of this segfault?

Edit

As per suggestion of @Employed Russian, I let gdb pass the SIGUSR1 to pin, which helped it advance, but not by far:

(gdb) handle SIGUSR1 nostop noprint pass
Signal        Stop      Print   Pass to program Description
SIGUSR1       No        No      Yes             User defined signal 1
(gdb) r
Starting program: /home/necto/pin/pin -appdebug -t /home/necto/pin-trace.so -- ./executable2 <args to the executable>
process 186041 is executing new program: /home/necto/pin/intel64/bin/pinbin
E: Attach to pid 186041 failed. 
E:   The Operating System configuration prevents Pin from using the default (parent) injection mode.
E:   To resolve this, either execute the following (as root):
E:   $ echo 0 > /proc/sys/kernel/yama/ptrace_scope
E:   Or use the "-injection child" option.
E:   For more information, regarding child injection, see Injection section in the Pin User Manual.
E: 

Edit2

The problem is in my pin tool. My pin tool pin-trace.so calls a function from the user code (from the application). This function fails on an assertion in the executable2 , which becomes an exception in pin and converts into a segfault, being unhandled.

Answer 1

When running it directly under gdb, it seems to fail differently (on SIGUSR1)

It looks like pin is trying to use SGIUSR1 internally. If you ask GDB to ignore this signal with handle SIGUSR1 nostop noprint pass , your GDB session will likely proceed further, hopefully to the crash on NULL pointer dereference.

In case it helps, this:

C: Tool (or Pin) caused signal 11 at PC 0x000000000

means that your pintool (or pin itself) called NULL function pointer.