tcptop and tcpsnoop

[Charles Meo]

A reprise on what seems to be a long story...

I've just downloaded dtrace toolkit 0.99 and, like many others over the
years, I'm having trouble with tcptop and tcpsnoop

sunray# ./tcpsnoop
dtrace: failed to compile script /dev/fd/11: line 173: failed to resolve
: Unknown variable name

sunray# ./tcptop
dtrace: failed to compile script /dev/fd/11: line 166: failed to resolve
`tcp_g_q: Unknown symbol name

sunray# ./tcpsnoop_snv
dtrace: failed to compile script /dev/fd/11: line 205: probe description
fbt:ip:tcp_xchg:entry does not match any probes

sunray# ./tcptop_snv
dtrace: failed to compile script /dev/fd/11: line 198: probe description
fbt:ip:tcp_xchg:entry does not match any probes

sunray# ./tcpsnoop.d
dtrace: failed to compile script ./tcpsnoop.d: line 210: failed to
resolve `tcp_g_q: Unknown symbol name

sunray# ./tcpsnoop_snv.d
dtrace: failed to compile script ./tcpsnoop_snv.d: line 243: probe
description fbt:ip:tcp_xchg:entry does not match any probes

I've spent half a day poking around in the forums and trying various of
the suggested hacks to get it working. All I've done is encountered all
the other error
messages that everyone else has seen doing the same things.

I gather that this all hinges on the rather suspect provider and there
are issues from solaris u5 onwards or thereabouts.

So, is anyone working on this right now, and, the age-old question, are
there working versions of these things around anywhere?
There doesn't seem to be any activity on this for quite a while, or else
I've not found it.

For the record:

sunray# uname -a
SunOS sunray 5.10 Generic_141445-09 i86pc i386 i86pc

sunray# cat /etc/release
Solaris 10 10/09 s10x_u8wos_08a X86
Copyright 2009 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 16 September 2009
--
::
Charles Meo
Solution Architect
Remora Technologies Pty Ltd
www.remora.com.au

Level 6, 685 Burke Road, Camberwell VIC 3124
E charles.meo-***@public.gmane.org
T 03 8080 5777
F 03 8080 5778
M 0409 258 471
::

[Peter Memishian]

Post by Charles Meo
A reprise on what seems to be a long story...
I've just downloaded dtrace toolkit 0.99 and, like many others over the
years, I'm having trouble with tcptop and tcpsnoop

I can't speak to Solaris 10, but as for OpenSolaris (which is the subject
of this alias), PSARC/2010/106 ("DTrace TCP and UDP providers") was
recently approved, and with it will come a set of stable TCP and UDP probe
points, along with a full set of DTrace Toolkit scripts using those stable
providers (including tcpsnoop and tcptop). This work is currently
targeting build 141.

--
meem

[Charles Meo]

Post by Peter Memishian

Post by Charles Meo
A reprise on what seems to be a long story...
I've just downloaded dtrace toolkit 0.99 and, like many others over the
years, I'm having trouble with tcptop and tcpsnoop

I can't speak to Solaris 10, but as for OpenSolaris (which is the subject
of this alias), PSARC/2010/106 ("DTrace TCP and UDP providers") was
recently approved, and with it will come a set of stable TCP and UDP probe
points, along with a full set of DTrace Toolkit scripts using those stable
providers (including tcpsnoop and tcptop). This work is currently
targeting build 141.
--
meem

Thanks Peter, but nearly everyone in customerland is using commercial
Solaris and that's the answer I really need...

I'll just have to wing it in the meantime.
--
::
Charles Meo
Solution Architect
Remora Technologies Pty Ltd
www.remora.com.au

Level 6, 685 Burke Road, Camberwell VIC 3124
E charles.meo-***@public.gmane.org
T 03 8080 5777
F 03 8080 5778
M 0409 258 471
::

[James Carlson]

Post by Charles Meo
Thanks Peter, but nearly everyone in customerland is using commercial
Solaris and that's the answer I really need...
I'll just have to wing it in the meantime.

On Solaris 10, those dtrace scripts are built on top of unstable
implementation artifacts, which means that they can (and do) break when
patches are released. And, unfortunately, the folks maintaining the
kernel bits aren't necessarily the same ones maintaining the dtrace
"examples." So breakage of the sort you've seen is unfortunately common.

Since you're talking about S10 rather than OpenSolaris, your best bet is
to contact either the dtrace toolkit's author or Oracle's customer
support group for help. It's not that S10 is "unknown" on these
OpenSolaris lists, but so much has changed since then that discussion
here is sometimes less than helpful.

[Charles Meo]

Post by James Carlson

Post by Charles Meo
Thanks Peter, but nearly everyone in customerland is using commercial
Solaris and that's the answer I really need...
I'll just have to wing it in the meantime.

On Solaris 10, those dtrace scripts are built on top of unstable
implementation artifacts, which means that they can (and do) break when
patches are released. And, unfortunately, the folks maintaining the
kernel bits aren't necessarily the same ones maintaining the dtrace
"examples." So breakage of the sort you've seen is unfortunately common.
Since you're talking about S10 rather than OpenSolaris, your best bet is
to contact either the dtrace toolkit's author or Oracle's customer
support group for help. It's not that S10 is "unknown" on these
OpenSolaris lists, but so much has changed since then that discussion
here is sometimes less than helpful.

OK, this is the impression I got from the what I've seen in the forums
etc. For the time being, then,
these tools are not available. So it goes...

Thanks for the info!
--
::
Charles Meo
Solution Architect
Remora Technologies Pty Ltd
www.remora.com.au

Level 6, 685 Burke Road, Camberwell VIC 3124
E charles.meo-***@public.gmane.org
T 03 8080 5777
F 03 8080 5778
M 0409 258 471
::

[Brendan Gregg - Sun Microsystems]

G'Day,

Post by Peter Memishian

Post by Charles Meo
A reprise on what seems to be a long story...
I've just downloaded dtrace toolkit 0.99 and, like many others over the
years, I'm having trouble with tcptop and tcpsnoop

I can't speak to Solaris 10, but as for OpenSolaris (which is the subject
of this alias), PSARC/2010/106 ("DTrace TCP and UDP providers") was
recently approved, and with it will come a set of stable TCP and UDP probe
points, along with a full set of DTrace Toolkit scripts using those
stable providers (including tcpsnoop and tcptop). This work is currently
targeting build 141.

Apologies for the broken scripts, and if it appears that no one is trying to
fix it; many of us have been, the outcome of which the PSARC case Peter
mentioned. And as Peter mentioned, this includes stable versions of tcpsnoop
and tcptop - at last!

The first goal was to come up with a stable TCP provider so that a robust (and
final) version of the scripts can be written. This stops tcpsnoop and tcptop
breakage in subsequent OpenSolaris versions (and Solaris when it appears
there), which stops the problem getting worse.

Then we can tackle the problem on hand - how many builds of OpenSolaris and
Solaris 10 do the current DTraceToolkit scripts not work for. Then fix them
one by one (some solutions have been posted here over the years).

Brendan

[Charles Meo]

Post by Brendan Gregg - Sun Microsystems
G'Day,

Post by Charles Meo
A reprise on what seems to be a long story..

Apologies for the broken scripts, and if it appears that no one is trying to
fix it; many of us have been, the outcome of which the PSARC case Peter
mentioned. And as Peter mentioned, this includes stable versions of tcpsnoop
and tcptop - at last!
[snip]
Brendan

Brendan,

Apologies definitely not needed. I think everyone appreciates the
massive amount of work involved and it's hardly your
fault that S'noracle has shall we say lost focus over the last few
months, and not just on engineering issues. I work for a Sun
partner and the ride has been bumpy to say the least. Things seem to be
improving across the board though.

The fixes will turn up in due course, in the meantime there is plenty of
good stuff to be getting on with. I've added various items from
the top 10 tools page to the DTrace Tools distribution and that will be
fine for now.
--
::
Charles Meo
Solution Architect
Remora Technologies Pty Ltd
www.remora.com.au

Level 6, 685 Burke Road, Camberwell VIC 3124
E charles.meo-***@public.gmane.org
T 03 8080 5777
F 03 8080 5778
M 0409 258 471
::

[James Carlson]

Post by Charles Meo
Apologies definitely not needed. I think everyone appreciates the
massive amount of work involved and it's hardly your
fault that S'noracle has shall we say lost focus over the last few
months, and not just on engineering issues. I work for a Sun
partner and the ride has been bumpy to say the least. Things seem to be
improving across the board though.

Uh, no, it's not that sort of thing.

It's great that Brendan Gregg is offering to clean up the scripts that
exist, and I wish him luck in tracking down all the patch levels
required to do that, but the truth is that the scripts were originally
written to use internal interfaces that are _intentionally_ not stable.
That internal interfaces are not stable has been true since the
beginning of time and is still true now for those same interfaces in
OpenSolaris.

What's actually changed in OpenSolaris is that *NEW* stable interfaces
were introduced on which scripts like those could safely be (re)written.
It just was not safe to write those scripts before there were stable
interfaces that could be used, and that lack of safety is what's showing up.

Blaming the problem on a lack of "focus" or something having to do with
Oracle is to completely miss the point. The problem was an
inappropriate (albeit unavoidable) reliance on undocumented
implementation artifacts, and the issues you experienced are the
unavoidable outcome of that sort of design.

This sort of problem has happened many times in the past, and will
almost certainly happen many times again in the future. As long as
there are folks writing independently delivered and maintained software
that depends on undocumented and unstable interfaces, you're going to
have field breakage and unhappy users.

For what it's worth, that's exactly why Sun has run an architecture
review board for 20 years, and why Oracle continues to do so.