Discussion:
Monitoring thread specific variable using dtrace
r***@public.gmane.org
2010-09-01 06:54:25 UTC
Permalink
Hi All,

Using dtrace how we can print/monitor a thread specific variable which is getting traced by dtrace. In one multithreaded application we want to trace a thread specific variable and monitor how it is changing the value. Is it possible using dtrace? Please advise and point me to the link/doc that I can use. I have gone through 'Solaris Dynamic Tracing Guide' pdf doc but could not locate what I was needing.


Any help would be appreciated.



Regards
Sudip
Ryan Johnson
2010-09-01 08:05:21 UTC
Permalink
Post by r***@public.gmane.org
In one multithreaded application we want to trace a thread specific
variable and monitor how it is changing the value. Is it possible
using dtrace?
I'm not aware of any particular support for reading TLS variables easily
(or any other variable in userland, for that matter). However, that
needn't stop anyone ;)

You could attach with a debugger and use it to print the TLS address,
then pass that to a .d script as a parameter, but that would fix it to a
particular thread instance of a particular version of your binary.

You could also compute the address manually (see below***) which may let
you trace any thread's version of the TLS var for that particular
version of the binary. However, this only works in normal (non-PIC)
code. PIC code forces the compiler to call a function to look up the TLS
address (since it's dynamically loaded), so dtrace would be out of luck.

Combining the previous two, you could attach with a debugger, ask it for
any thread's address of the TLS var, then subtract off that thread's
value of %tls-reg to get OFFSET. Then pass the offset to dtrace and
recompute %tls-reg+OFFSET when needed. This would avoid mucking around
in assembler and would work even for PIC code as long as you don't
unload/reload the module which contains your variable of interest.
You're still stuck if the binary changes, though:

=== using dbx on sparcv9 for TLS of type int ===
export TLS_OFFSET=$(echo 'print &tls_var_of_interest - $g7' | dbx -p
pid-of-interest | tail -n2 | awk '{print $3}')
dtrace -n 'the:probe:of:interest { this->tls_val = *(int*)
copyin($1+uregs[REG_G7], sizeof(int)) }' $TLS_OFFSET
======

The most flexible way to tell dtrace about an address of interest
requires source code and a recompile. Create a hook function which
accepts a void* as its only parameter and does nothing. Have every
thread call that hook at startup with &tls_var_of_interest as the
argument. Then, in the .d script, trace the hook's entry and store the
address in self->tls_addr or some such. From then on, you can just
copyin() that address whenever you want to see its contents. Just make
sure dtrace is running before the thread of interest starts and don't
let the compiler inline the hook function. This works for any variable,
not just TLS (heap, stack, global).

=== c code ===
void give_address_to_dtrace(void* addr) { /* do nothing */ }
static __thread int tls_var = 10;
void* worker_thread_run(void* arg) {
give_address_to_dtrace(&tls_var);
....
}
======

=== d script ===
pid$target::give_address_to_dtrace:entry { self->tls_addr = args[0] }
the:probe:of:interest { this->tls_value = *(int*) copyin(self->tls_addr,
sizeof(int)) }
======

*** The manual way... thread-local storage is implemented by making some
register (%g7 on sparcv9) always point to the running thread's internal
data structure, which (among other things) stores TLS variables. The ABI
specification for your architecture will have all the hairy details
(disassembling toy programs also works pretty well). In a normal (not
PIC) app the address of a TLS variable is simply %tls-reg + OFFSET,
where OFFSET is some number assigned by the compiler. You should be able
to disassemble the app and figure out that OFFSET. On x86-64-gcc this is
fairly simple (2-3 instructions usually), but sparcv9-SunCC takes an
impressively convoluted approach to computing it...

Regards,
Ryan

Loading...