how to know which statement the running process is executing

Question

I have a process which suddenly hanged and is not giving any core dump and is also not killed.i can see it still running using the ps command.

how can i know which statement it is currently executing inside the code.

basically i want to know where exactly it got hanged.

language is c++ and platform is solaris unix.

demos.283> cat test3.cc
#include<stdio.h>
#include<unistd.h>

int main()
{

sleep(100);
return 0;

}
demos.284> CC test3.cc 
demos.285> ./a.out &
[1] 2231
demos.286> ps -o "pid,wchan,comm"
  PID            WCHAN COMMAND
23420 fffffe86e9a5aff6 -tcsh
 2345                - ps
 2231 ffffffffb8ca3376 ./a.out
demos.290> ps
   PID TTY         TIME CMD
  3823 pts/36      0:00 ps
 23420 pts/36      0:00 tcsh
  3822 pts/36      0:00 a.out
demos.291> pstack 3822
3822:   ./a.out
 fed1a215 nanosleep (80478c0, 80478c8)
 080508ff main     (1, 8047920, 8047928, fed93ec0) + f
 0805085d _start   (1, 8047a4c, 0, 8047a54, 8047a67, 8047c05) + 7d
demos.292> 

Answer 1

You have several options: the easiest is to check the WCHAN wait channel that the process is sleeping on:

$ ps -o "pid,wchan,comm"
  PID WCHAN  COMMAND
 2350 wait   bash
20639 hrtime i3status
20640 poll_s dzen2
28821 -      ps

This can give you a good indication of what the process is doing and is very easy to get.

You can use ktruss and ktrace or DTrace to trace your process. (Sorry, no Solaris here, so no examples.)

You can also attach gdb(1) to your process:

# gdb -p 20640
GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
...
(gdb) bt
#0  0x00007fd1a99fd123 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:82
#1  0x0000000000405533 in ?? ()
#2  0x00007fd1a993deff in __libc_start_main (main=0x4043e3, argc=13, ubp_av=0x7fff25e7b478, 
...

The backtrace is often the single most useful error report you can get from a process, so it is worth installing gdb(1) if it isn't already installed. gdb(1) can do a lot more than just show you backtraces, but a full tutorial is well outside the scope of Stack Overflow.

Answer 2

you can try with pstack passing pid as parameter. You can use ps to get the process id (pid)

For example: pstack 1267