Discussion:
drops on CPU?
Kevin Colwell
2010-05-19 23:25:19 UTC
Permalink
In reviewing some dtrace output I'm confused because something I expect to see is missing. I see a process' thread enter a function but I never see it return. About a minute later I see the same tid active, but in other functions. I'm guessing that perhaps the trace entries were dropped due to buffer overflow, but I don't have the stderr from when it ran so I don't know for sure.

Are "drops on cpu" deterministic? Chronologically, would the buffer fill with the first traces and then drop subsequent events, or would it keep on tracing in a circular buffer, overwriting old events and reporting only the latest events that remain in the buffer? If the former, I would expect to see the function return because it should have been the first event of a high activity period.

Kevin
Michael Schuster
2010-05-20 07:26:48 UTC
Permalink
Post by Kevin Colwell
In reviewing some dtrace output I'm confused because something I expect
to see is missing. I see a process' thread enter a function but I never
see it return. About a minute later I see the same tid active, but in
other functions. I'm guessing that perhaps the trace entries were
dropped due to buffer overflow, but I don't have the stderr from when it
ran so I don't know for sure.
Are "drops on cpu" deterministic? Chronologically, would the buffer
fill with the first traces and then drop subsequent events, or would it
keep on tracing in a circular buffer, overwriting old events and
reporting only the latest events that remain in the buffer? If the
former, I would expect to see the function return because it should have
been the first event of a high activity period.
Dtrace cannot instrument all return functions - I think it's got to do with
the way the stack is set up (I think "leaf functions" is the operative word
here, I may be wrong though).

I'd suggest you peruse the archives, Adam Leventhal among others has
answered this question several times ;-)

HTH
Michael
--
michael.schuster-QHcLZuEGTsvQT0dZR+***@public.gmane.org http://blogs.sun.com/recursion
Recursion, n.: see 'Recursion'
Adam Leventhal
2010-05-20 14:42:04 UTC
Permalink
Michael,
Dtrace cannot instrument all return functions - I think it's got to do with the way the stack is set up (I think "leaf functions" is the operative word here, I may be wrong though).
I'd suggest you peruse the archives, Adam Leventhal among others has answered this question several times ;-)
DTrace can instrument all return functions. If you see functions that can't be instrumented or aren't properly instrumented, please let us know.
Post by Kevin Colwell
In reviewing some dtrace output I'm confused because something I expect
to see is missing. I see a process' thread enter a function but I never
see it return. About a minute later I see the same tid active, but in
other functions. I'm guessing that perhaps the trace entries were
dropped due to buffer overflow, but I don't have the stderr from when it
ran so I don't know for sure.
Is this on a system with multiple CPU cores? When presenting its data, DTrace does not correlate thread control flow as it moves between CPUs. You can do this as a post-processing step by adding a timestamp and sorting.
Post by Kevin Colwell
Are "drops on cpu" deterministic? Chronologically, would the buffer
fill with the first traces and then drop subsequent events, or would it
keep on tracing in a circular buffer, overwriting old events and
reporting only the latest events that remain in the buffer? If the
former, I would expect to see the function return because it should have
been the first event of a high activity period.
DTrace will not discard data without reporting it. They are not deterministic per se since the amount of data being recorded on a given CPU for a given time period is affected by the scheduler and other system load for example.

Adam

--
Adam Leventhal, Fishworks http://blogs.sun.com/ahl
Loading...