#include <sys/ptrace.h>
long ptrace(enum __ptrace_request request, pid_t pid,
void *addr, void *data);
A tracee first needs to be attached to the tracer. Attachment and subsequent commands are per thread: in a multithreaded process, every thread can be individually attached to a (potentially different) tracer, or left not attached and thus not debugged. Therefore, "tracee" always means "(one) thread", never "a (possibly multithreaded) process". Ptrace commands are always sent to a specific tracee using a call of the form
ptrace(PTRACE_foo, pid, ...)
where pid is the thread ID of the corresponding Linux thread.
(Note that in this page, a "multithreaded process" means a thread group consisting of threads created using the clone(2) CLONE_THREAD flag.)
A process can initiate a trace by calling fork(2) and having the resulting child do a PTRACE_TRACEME, followed (typically) by an execve(2). Alternatively, one process may commence tracing another process using PTRACE_ATTACH or PTRACE_SEIZE.
While being traced, the tracee will stop each time a signal is delivered, even if the signal is being ignored. (An exception is SIGKILL, which has its usual effect.) The tracer will be notified at its next call to waitpid(2) (or one of the related "wait" system calls); that call will return a status value containing information that indicates the cause of the stop in the tracee. While the tracee is stopped, the tracer can use various ptrace requests to inspect and modify the tracee. The tracer then causes the tracee to continue, optionally ignoring the delivered signal (or even delivering a different signal instead).
If the PTRACE_O_TRACEEXEC option is not in effect, all successful calls to execve(2) by the traced process will cause it to be sent a SIGTRAP signal, giving the parent a chance to gain control before the new program begins execution.
When the tracer is finished tracing, it can cause the tracee to continue executing in a normal, untraced mode via PTRACE_DETACH.
The value of request determines the action to be performed:
struct ptrace_peeksiginfo_args {
u64 off; /* Ordinal position in queue at which
to start copying signals */
u32 flags; /* PTRACE_PEEKSIGINFO_SHARED or 0 */
s32 nr; /* Number of signals to copy */
};
Currently, there is only one flag, PTRACE_PEEKSIGINFO_SHARED, for dumping signals from the process-wide signal queue. If this flag is not set, signals are read from the per-thread queue of the specified thread.
status>>8 == (SIGTRAP | (PTRACE_EVENT_CLONE<<8))
The PID of the new process can be retrieved with PTRACE_GETEVENTMSG.
status>>8 == (SIGTRAP | (PTRACE_EVENT_EXEC<<8))
If the execing thread is not a thread group leader, the thread ID is reset to thread group leader's ID before this stop. Since Linux 3.0, the former thread ID can be retrieved with PTRACE_GETEVENTMSG.
status>>8 == (SIGTRAP | (PTRACE_EVENT_EXIT<<8))
The tracee's exit status can be retrieved with PTRACE_GETEVENTMSG.
status>>8 == (SIGTRAP | (PTRACE_EVENT_FORK<<8))
The PID of the new process can be retrieved with PTRACE_GETEVENTMSG.
status>>8 == (SIGTRAP | (PTRACE_EVENT_VFORK<<8))
The PID of the new process can be retrieved with PTRACE_GETEVENTMSG.
status>>8 == (SIGTRAP | (PTRACE_EVENT_VFORK_DONE<<8))
The PID of the new process can (since Linux 2.6.18) be retrieved with PTRACE_GETEVENTMSG.
Note that the killing signal will first cause signal-delivery-stop (on one tracee only), and only after it is injected by the tracer (or after it was dispatched to a thread which isn't traced), will death from the signal happen on all tracees within a multithreaded process. (The term "signal-delivery-stop" is explained below.)
SIGKILL does not generate signal-delivery-stop and therefore the tracer can't suppress it. SIGKILL kills even within system calls (syscall-exit-stop is not generated prior to death by SIGKILL). The net effect is that SIGKILL always kills the process (all its threads), even if some threads of the process are ptraced.
When the tracee calls _exit(2), it reports its death to its tracer. Other threads are not affected.
When any thread executes exit_group(2), every tracee in its thread group reports its death to its tracer.
If the PTRACE_O_TRACEEXIT option is on, PTRACE_EVENT_EXIT will happen before actual death. This applies to exits via exit(2), exit_group(2), and signal deaths (except SIGKILL), and when threads are torn down on execve(2) in a multithreaded process.
The tracer cannot assume that the ptrace-stopped tracee exists. There are many scenarios when the tracee may die while stopped (such as SIGKILL). Therefore, the tracer must be prepared to handle an ESRCH error on any ptrace operation. Unfortunately, the same error is returned if the tracee exists but is not ptrace-stopped (for commands which require a stopped tracee), or if it is not traced by the process which issued the ptrace call. The tracer needs to keep track of the stopped/running state of the tracee, and interpret ESRCH as "tracee died unexpectedly" only if it knows that the tracee has been observed to enter ptrace-stop. Note that there is no guarantee that waitpid(WNOHANG) will reliably report the tracee's death status if a ptrace operation returned ESRCH. waitpid(WNOHANG) may return 0 instead. In other words, the tracee may be "not yet fully dead", but already refusing ptrace requests.
The tracer can't assume that the tracee always ends its life by reporting WIFEXITED(status) or WIFSIGNALED(status); there are cases where this does not occur. For example, if a thread other than thread group leader does an execve(2), it disappears; its PID will never be seen again, and any subsequent ptrace stops will be reported under the thread group leader's PID.
There are many kinds of states when the tracee is stopped, and in ptrace discussions they are often conflated. Therefore, it is important to use precise terms.
In this manual page, any stopped state in which the tracee is ready to accept ptrace commands from the tracer is called ptrace-stop. Ptrace-stops can be further subdivided into signal-delivery-stop, group-stop, syscall-stop, and so on. These stopped states are described in detail below.
When the running tracee enters ptrace-stop, it notifies its tracer using waitpid(2) (or one of the other "wait" system calls). Most of this manual page assumes that the tracer waits with:
pid = waitpid(pid_or_minus_1, &status, __WALL);
Ptrace-stopped tracees are reported as returns with pid greater than 0 and WIFSTOPPED(status) true.
The __WALL flag does not include the WSTOPPED and WEXITED flags, but implies their functionality.
Setting the WCONTINUED flag when calling waitpid(2) is not recommended: the "continued" state is per-process and consuming it can confuse the real parent of the tracee.
Use of the WNOHANG flag may cause waitpid(2) to return 0 ("no wait results available yet") even if the tracer knows there should be a notification. Example:
errno = 0;
ptrace(PTRACE_CONT, pid, 0L, 0L);
if (errno == ESRCH) {
/* tracee is dead */
r = waitpid(tracee, &status, __WALL | WNOHANG);
/* r can still be 0 here! */
}
The following kinds of ptrace-stops exist: signal-delivery-stops, group-stops, PTRACE_EVENT stops, syscall-stops. They all are reported by waitpid(2) with WIFSTOPPED(status) true. They may be differentiated by examining the value status>>8, and if there is ambiguity in that value, by querying PTRACE_GETSIGINFO. (Note: the WSTOPSIG(status) macro can't be used to perform this examination, because it returns the value (status>>8) & 0xff.)
Signal-delivery-stop is observed by the tracer as waitpid(2) returning with WIFSTOPPED(status) true, with the signal returned by WSTOPSIG(status). If the signal is SIGTRAP, this may be a different kind of ptrace-stop; see the "Syscall-stops" and "execve" sections below for details. If WSTOPSIG(status) returns a stopping signal, this may be a group-stop; see below.
ptrace(PTRACE_restart, pid, 0, sig)
where PTRACE_restart is one of the restarting ptrace requests. If sig is 0, then a signal is not delivered. Otherwise, the signal sig is delivered. This operation is called signal injection in this manual page, to distinguish it from signal-delivery-stop.
The sig value may be different from the WSTOPSIG(status) value: the tracer can cause a different signal to be injected.
Note that a suppressed signal still causes system calls to return prematurely. In this case, system calls will be restarted: the tracer will observe the tracee to reexecute the interrupted system call (or restart_syscall(2) system call for a few system calls which use a different mechanism for restarting) if the tracer uses PTRACE_SYSCALL. Even system calls (such as poll(2)) which are not restartable after signal are restarted after signal is suppressed; however, kernel bugs exist which cause some system calls to fail with EINTR even though no observable signal is injected to the tracee.
Restarting ptrace commands issued in ptrace-stops other than signal-delivery-stop are not guaranteed to inject a signal, even if sig is nonzero. No error is reported; a nonzero sig may simply be ignored. Ptrace users should not try to "create a new signal" this way: use tgkill(2) instead.
The fact that signal injection requests may be ignored when restarting the tracee after ptrace stops that are not signal-delivery-stops is a cause of confusion among ptrace users. One typical scenario is that the tracer observes group-stop, mistakes it for signal-delivery-stop, restarts the tracee with
ptrace(PTRACE_restart, pid, 0, stopsig)
with the intention of injecting stopsig, but stopsig gets ignored and the tracee continues to run.
The SIGCONT signal has a side effect of waking up (all threads of) a group-stopped process. This side effect happens before signal-delivery-stop. The tracer can't suppress this side effect (it can only suppress signal injection, which only causes the SIGCONT handler to not be executed in the tracee, if such a handler is installed). In fact, waking up from group-stop may be followed by signal-delivery-stop for signal(s) other than SIGCONT, if they were pending when SIGCONT was delivered. In other words, SIGCONT may be not the first signal observed by the tracee after it was sent.
Stopping signals cause (all threads of) a process to enter group-stop. This side effect happens after signal injection, and therefore can be suppressed by the tracer.
In Linux 2.4 and earlier, the SIGSTOP signal can't be injected.
PTRACE_GETSIGINFO can be used to retrieve a siginfo_t structure which corresponds to the delivered signal. PTRACE_SETSIGINFO may be used to modify it. If PTRACE_SETSIGINFO has been used to alter siginfo_t, the si_signo field and the sig parameter in the restarting command must match, otherwise the result is undefined.
Group-stop is observed by the tracer as waitpid(2) returning with WIFSTOPPED(status) true, with the stopping signal available via WSTOPSIG(status). The same result is returned by some other classes of ptrace-stops, therefore the recommended practice is to perform the call
ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo)
The call can be avoided if the signal is not SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU; only these four signals are stopping signals. If the tracer sees something else, it can't be a group-stop. Otherwise, the tracer needs to call PTRACE_GETSIGINFO. If PTRACE_GETSIGINFO fails with EINVAL, then it is definitely a group-stop. (Other failure codes are possible, such as ESRCH ("no such process") if a SIGKILL killed the tracee.)
If tracee was attached using PTRACE_SEIZE, group-stop is indicated by PTRACE_EVENT_STOP: status>>16 == PTRACE_EVENT_STOP. This allows detection of group-stops without requiring an extra PTRACE_GETSIGINFO call.
As of Linux 2.6.38, after the tracer sees the tracee ptrace-stop and until it restarts or kills it, the tracee will not run, and will not send notifications (except SIGKILL death) to the tracer, even if the tracer enters into another waitpid(2) call.
The kernel behavior described in the previous paragraph causes a problem with transparent handling of stopping signals. If the tracer restarts the tracee after group-stop, the stopping signal is effectively ignored---the tracee doesn't remain stopped, it runs. If the tracer doesn't restart the tracee before entering into the next waitpid(2), future SIGCONT signals will not be reported to the tracer; this would cause the SIGCONT signals to have no effect on the tracee.
Since Linux 3.4, there is a method to overcome this problem: instead of PTRACE_CONT, a PTRACE_LISTEN command can be used to restart a tracee in a way where it does not execute, but waits for a new event which it can report via waitpid(2) (such as when it is restarted by a SIGCONT).
PTRACE_EVENT stops are observed by the tracer as waitpid(2) returning with WIFSTOPPED(status), and WSTOPSIG(status) returns SIGTRAP. An additional bit is set in the higher byte of the status word: the value status>>8 will be
(SIGTRAP | PTRACE_EVENT_foo << 8).
The following events exist:
For all four stops described above, the stop occurs in the parent (i.e., the tracee), not in the newly created thread. PTRACE_GETEVENTMSG can be used to retrieve the new thread's ID.
PTRACE_GETSIGINFO on PTRACE_EVENT stops returns SIGTRAP in si_signo, with si_code set to (event<<8) | SIGTRAP.
Other possibilities are that the tracee may stop in a PTRACE_EVENT stop, exit (if it entered _exit(2) or exit_group(2)), be killed by SIGKILL, or die silently (if it is a thread group leader, the execve(2) happened in another thread, and that thread is not traced by the same tracer; this situation is discussed later).
Syscall-enter-stop and syscall-exit-stop are observed by the tracer as waitpid(2) returning with WIFSTOPPED(status) true, and WSTOPSIG(status) giving SIGTRAP. If the PTRACE_O_TRACESYSGOOD option was set by the tracer, then WSTOPSIG(status) will give the value (SIGTRAP | 0x80).
Syscall-stops can be distinguished from signal-delivery-stop with SIGTRAP by querying PTRACE_GETSIGINFO for the following cases:
However, syscall-stops happen very often (twice per system call), and performing PTRACE_GETSIGINFO for every syscall-stop may be somewhat expensive.
Some architectures allow the cases to be distinguished by examining registers. For example, on x86, rax == -ENOSYS in syscall-enter-stop. Since SIGTRAP (like any other signal) always happens after syscall-exit-stop, and at this point rax almost never contains -ENOSYS, the SIGTRAP looks like "syscall-stop which is not syscall-enter-stop"; in other words, it looks like a "stray syscall-exit-stop" and can be detected this way. But such detection is fragile and is best avoided.
Using the PTRACE_O_TRACESYSGOOD option is the recommended method to distinguish syscall-stops from other kinds of ptrace-stops, since it is reliable and does not incur a performance penalty.
Syscall-enter-stop and syscall-exit-stop are indistinguishable from each other by the tracer. The tracer needs to keep track of the sequence of ptrace-stops in order to not misinterpret syscall-enter-stop as syscall-exit-stop or vice versa. The rule is that syscall-enter-stop is always followed by syscall-exit-stop, PTRACE_EVENT stop or the tracee's death; no other kinds of ptrace-stop can occur in between.
If after syscall-enter-stop, the tracer uses a restarting command other than PTRACE_SYSCALL, syscall-exit-stop is not generated.
PTRACE_GETSIGINFO on syscall-stops returns SIGTRAP in si_signo, with si_code set to SIGTRAP or (SIGTRAP|0x80).
When the tracee is in ptrace-stop, the tracer can read and write data to the tracee using informational commands. These commands leave the tracee in ptrace-stopped state:
ptrace(PTRACE_PEEKTEXT/PEEKDATA/PEEKUSER, pid, addr, 0);
ptrace(PTRACE_POKETEXT/POKEDATA/POKEUSER, pid, addr, long_val);
ptrace(PTRACE_GETREGS/GETFPREGS, pid, 0, &struct);
ptrace(PTRACE_SETREGS/SETFPREGS, pid, 0, &struct);
ptrace(PTRACE_GETREGSET, pid, NT_foo, &iov);
ptrace(PTRACE_SETREGSET, pid, NT_foo, &iov);
ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo);
ptrace(PTRACE_SETSIGINFO, pid, 0, &siginfo);
ptrace(PTRACE_GETEVENTMSG, pid, 0, &long_var);
ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_flags);
Note that some errors are not reported. For example, setting signal information (siginfo) may have no effect in some ptrace-stops, yet the call may succeed (return 0 and not set errno); querying PTRACE_GETEVENTMSG may succeed and return some random value if current ptrace-stop is not documented as returning a meaningful event message.
The call
ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_flags);
affects one tracee. The tracee's current flags are replaced. Flags are inherited by new tracees created and "auto-attached" via active PTRACE_O_TRACEFORK, PTRACE_O_TRACEVFORK, or PTRACE_O_TRACECLONE options.
Another group of commands makes the ptrace-stopped tracee run. They have the form:
ptrace(cmd, pid, 0, sig);
where cmd is PTRACE_CONT, PTRACE_LISTEN, PTRACE_DETACH, PTRACE_SYSCALL, PTRACE_SINGLESTEP, PTRACE_SYSEMU, or PTRACE_SYSEMU_SINGLESTEP. If the tracee is in signal-delivery-stop, sig is the signal to be injected (if it is nonzero). Otherwise, sig may be ignored. (When restarting a tracee from a ptrace-stop other than signal-delivery-stop, recommended practice is to always pass 0 in sig.)
ptrace(PTRACE_ATTACH, pid, 0, 0);
or
ptrace(PTRACE_SEIZE, pid, 0, PTRACE_O_flags);
PTRACE_ATTACH sends SIGSTOP to this thread. If the tracer wants this SIGSTOP to have no effect, it needs to suppress it. Note that if other signals are concurrently sent to this thread during attach, the tracer may see the tracee enter signal-delivery-stop with other signal(s) first! The usual practice is to reinject these signals until SIGSTOP is seen, then suppress SIGSTOP injection. The design bug here is that a ptrace attach and a concurrently delivered SIGSTOP may race and the concurrent SIGSTOP may be lost.
Since attaching sends SIGSTOP and the tracer usually suppresses it, this may cause a stray EINTR return from the currently executing system call in the tracee, as described in the "Signal injection and suppression" section.
Since Linux 3.4, PTRACE_SEIZE can be used instead of PTRACE_ATTACH. PTRACE_SEIZE does not stop the attached process. If you need to stop it after attach (or at any other time) without sending it any signals, use PTRACE_INTERRUPT command.
The request
ptrace(PTRACE_TRACEME, 0, 0, 0);
turns the calling thread into a tracee. The thread continues to run (doesn't enter ptrace-stop). A common practice is to follow the PTRACE_TRACEME with
raise(SIGSTOP);
and allow the parent (which is our tracer now) to observe our signal-delivery-stop.
If the PTRACE_O_TRACEFORK, PTRACE_O_TRACEVFORK, or PTRACE_O_TRACECLONE options are in effect, then children created by, respectively, vfork(2) or clone(2) with the CLONE_VFORK flag, fork(2) or clone(2) with the exit signal set to SIGCHLD, and other kinds of clone(2), are automatically attached to the same tracer which traced their parent. SIGSTOP is delivered to the children, causing them to enter signal-delivery-stop after they exit the system call which created them.
Detaching of the tracee is performed by:
ptrace(PTRACE_DETACH, pid, 0, sig);
PTRACE_DETACH is a restarting operation; therefore it requires the tracee to be in ptrace-stop. If the tracee is in signal-delivery-stop, a signal can be injected. Otherwise, the sig parameter may be silently ignored.
If the tracee is running when the tracer wants to detach it, the usual solution is to send SIGSTOP (using tgkill(2), to make sure it goes to the correct thread), wait for the tracee to stop in signal-delivery-stop for SIGSTOP and then detach it (suppressing SIGSTOP injection). A design bug is that this can race with concurrent SIGSTOPs. Another complication is that the tracee may enter other ptrace-stops and needs to be restarted and waited for again, until SIGSTOP is seen. Yet another complication is to be sure that the tracee is not already ptrace-stopped, because no signal delivery happens while it is---not even SIGSTOP.
If the tracer dies, all tracees are automatically detached and restarted, unless they were in group-stop. Handling of restart from group-stop is currently buggy, but the "as planned" behavior is to leave tracee stopped and waiting for SIGCONT. If the tracee is restarted from signal-delivery-stop, the pending signal is injected.
All of the above effects are the artifacts of the thread ID change in the tracee.
The PTRACE_O_TRACEEXEC option is the recommended tool for dealing with this situation. First, it enables PTRACE_EVENT_EXEC stop, which occurs before execve(2) returns. In this stop, the tracer can use PTRACE_GETEVENTMSG to retrieve the tracee's former thread ID. (This feature was introduced in Linux 3.0). Second, the PTRACE_O_TRACEEXEC option disables legacy SIGTRAP generation on execve(2).
When the tracer receives PTRACE_EVENT_EXEC stop notification, it is guaranteed that except this tracee and the thread group leader, no other threads from the process are alive.
On receiving the PTRACE_EVENT_EXEC stop notification, the tracer should clean up all its internal data structures describing the threads of this process, and retain only one data structure---one which describes the single still running tracee, with
thread ID == thread group ID == process ID.
Example: two threads call execve(2) at the same time:
*** we get syscall-enter-stop in thread 1: **
PID1 execve("/bin/foo", "foo" <unfinished ...>
*** we issue PTRACE_SYSCALL for thread 1 **
*** we get syscall-enter-stop in thread 2: **
PID2 execve("/bin/bar", "bar" <unfinished ...>
*** we issue PTRACE_SYSCALL for thread 2 **
*** we get PTRACE_EVENT_EXEC for PID0, we issue PTRACE_SYSCALL **
*** we get syscall-exit-stop for PID0: **
PID0 <... execve resumed> ) = 0
If the PTRACE_O_TRACEEXEC option is not in effect for the execing tracee, the kernel delivers an extra SIGTRAP to the tracee after execve(2) returns. This is an ordinary signal (similar to one which can be generated by kill -TRAP), not a special kind of ptrace-stop. Employing PTRACE_GETSIGINFO for this signal returns si_code set to 0 (SI_USER). This signal may be blocked by signal mask, and thus may be delivered (much) later.
Usually, the tracer (for example, strace(1)) would not want to show this extra post-execve SIGTRAP signal to the user, and would suppress its delivery to the tracee (if SIGTRAP is set to SIG_DFL, it is a killing signal). However, determining which SIGTRAP to suppress is not easy. Setting the PTRACE_O_TRACEEXEC option and thus suppressing this extra SIGTRAP is the recommended approach.
Many of these bugs have been fixed, but as of Linux 2.6.38 several still exist; see BUGS below.
As of Linux 2.6.38, the following is believed to work correctly:
On error, all requests return -1, and errno is set appropriately. Since the value returned by a successful PTRACE_PEEK* request may be -1, the caller must clear errno before the call, and then check it afterward to determine whether or not an error occurred.
In Linux kernels before 2.6.26, init(8), the process with PID 1, may not be traced.
The layout of the contents of memory and the USER area are quite operating-system- and architecture-specific. The offset supplied, and the data returned, might not entirely match with the definition of struct user.
The size of a "word" is determined by the operating-system variant (e.g., for 32-bit Linux it is 32 bits).
This page documents the way the ptrace() call works currently in Linux. Its behavior differs significantly on other flavors of UNIX. In any case, use of ptrace() is highly specific to the operating system and architecture.
Group-stop notifications are sent to the tracer, but not to real parent. Last confirmed on 2.6.38.6.
If a thread group leader is traced and exits by calling _exit(2), a PTRACE_EVENT_EXIT stop will happen for it (if requested), but the subsequent WIFEXITED notification will not be delivered until all other threads exit. As explained above, if one of other threads calls execve(2), the death of the thread group leader will never be reported. If the execed thread is not traced by this tracer, the tracer will never know that execve(2) happened. One possible workaround is to PTRACE_DETACH the thread group leader instead of restarting it in this case. Last confirmed on 2.6.38.6.
A SIGKILL signal may still cause a PTRACE_EVENT_EXIT stop before actual signal death. This may be changed in the future; SIGKILL is meant to always immediately kill tasks even under ptrace. Last confirmed on 2.6.38.6.
Some system calls return with EINTR if a signal was sent to a tracee, but delivery was suppressed by the tracer. (This is very typical operation: it is usually done by debuggers on every attach, in order to not introduce a bogus SIGSTOP). As of Linux 3.2.9, the following system calls are affected (this list is likely incomplete): epoll_wait(2), and read(2) from an inotify(7) file descriptor. The usual symptom of this bug is that when you attach to a quiescent process with the command
strace -p <process-ID>
then, instead of the usual and expected one-line output such as
restart_syscall(<... resuming interrupted call ...>_
or
select(6, [5], NULL, [5], NULL_
('_' denotes the cursor position), you observe more than one line.
For example:
clock_gettime(CLOCK_MONOTONIC, {15370, 690928118}) = 0
epoll_wait(4,_
What is not visible here is that the process was blocked in
epoll_wait(2)
before
strace(1)
has attached to it.
Attaching caused
epoll_wait(2)
to return to user space with the error
EINTR.
In this particular case, the program reacted to
EINTR
by checking the current time, and then executing
epoll_wait(2)
again.
(Programs which do not expect such "stray"
EINTR
errors may behave in an unintended way upon an
strace(1)
attach.)