Why does java not stop() when I tell it to stop()?

[Jim Fiori]

Hi,

I'm using the rexit/stop()/prun() dance to capture process exit in order to grab various procfs commands on exit. Works fine for simple program like "sleep". But for Java (and other threaded customer apps) it runs one or 2 of the p-commands, but the process goes away before I can get all I want:

$ cat exit.d
#!/usr/sbin/dtrace -qws

syscall::rexit:entry
/execname == "sleep" || execname == "java"/
{
stop();
system("pargs %d",pid);
system ("pldd %d",pid);
system ("pfiles %d",pid);
system ("pstack %d",pid);

system ("prun %d",pid);
}

$ ./exit.d&
[1] 22487

OK, now test a "sleep" exit:

bash-3.00$ sleep 5
22488: sleep 5
argv[0]: sleep
argv[1]: 5
22488: sleep 5
/lib/libc.so.1
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
22488: sleep 5
Current rlimit: 256 file descriptors
0: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
/devices/pseudo/***@0:18
1: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
/devices/pseudo/***@0:18
2: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
/devices/pseudo/***@0:18
22488: sleep 5
feea4d18 ecvt (2, 8047290, 8047296, 0, 8047298, 8047306)

Looks like all the p-commands completed. But for java:

0$ /usr/java/bin/java -version
java version "1.5.0_20"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_20-b02)
Java HotSpot(TM) Client VM (build 1.5.0_20-b02, mixed mode, sharing)
22513: /usr/java/bin/java -version
argv[0]: /usr/java/bin/java
argv[1]: -version
22513: /usr/java/bin/java -version
/lib/libthread.so.1
/lib/libdl.so.1
/lib/libc.so.1
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/client/libjvm.so
/lib/libsocket.so.1
/usr/lib/libsched.so.1
/usr/lib/libCrun.so.1
/lib/libm.so.1
/lib/libnsl.so.1
/lib/libm.so.2
/lib/libscf.so.1
/lib/libdoor.so.1
/lib/libuutil.so.1
/lib/libgen.so.1
/lib/libmd.so.1
/lib/libmp.so.2
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/native_threads/libhpi.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libverify.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libjava.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libzip.so
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
bash-3.00$ pfiles: cannot examine 22513: no such process
pstack: cannot examine 22513: no such process or core file
prun: cannot control 22513: no such process


Any ideas? Something is telling the process to continue.

Jim

[Adam Leventhal]

Hey Jim,

I don't have any ideas off the top of my head, but you can certainly use DTrace to figure out what's giving java the order to continue...

Adam

Post by Jim Fiori
Hi,
$ cat exit.d
#!/usr/sbin/dtrace -qws
syscall::rexit:entry
/execname == "sleep" || execname == "java"/
{
stop();
system("pargs %d",pid);
system ("pldd %d",pid);
system ("pfiles %d",pid);
system ("pstack %d",pid);
system ("prun %d",pid);
}
$ ./exit.d&
[1] 22487
bash-3.00$ sleep 5
22488: sleep 5
argv[0]: sleep
argv[1]: 5
22488: sleep 5
/lib/libc.so.1
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
22488: sleep 5
Current rlimit: 256 file descriptors
0: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
1: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
2: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667 gid:7 rdev:24,18
O_RDWR
22488: sleep 5
feea4d18 ecvt (2, 8047290, 8047296, 0, 8047298, 8047306)
0$ /usr/java/bin/java -version
java version "1.5.0_20"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_20-b02)
Java HotSpot(TM) Client VM (build 1.5.0_20-b02, mixed mode, sharing)
22513: /usr/java/bin/java -version
argv[0]: /usr/java/bin/java
argv[1]: -version
22513: /usr/java/bin/java -version
/lib/libthread.so.1
/lib/libdl.so.1
/lib/libc.so.1
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/client/libjvm.so
/lib/libsocket.so.1
/usr/lib/libsched.so.1
/usr/lib/libCrun.so.1
/lib/libm.so.1
/lib/libnsl.so.1
/lib/libm.so.2
/lib/libscf.so.1
/lib/libdoor.so.1
/lib/libuutil.so.1
/lib/libgen.so.1
/lib/libmd.so.1
/lib/libmp.so.2
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/native_threads/libhpi.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libverify.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libjava.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libzip.so
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
bash-3.00$ pfiles: cannot examine 22513: no such process
pstack: cannot examine 22513: no such process or core file
prun: cannot control 22513: no such process
Any ideas? Something is telling the process to continue.
Jim
--
This message posted from opensolaris.org
_______________________________________________
dtrace-discuss mailing list

--
Adam Leventhal, Fishworks http://blogs.sun.com/ahl

[Jim Fiori]

Thanks Adam - I've already started down this path.

Jim

Post by Adam Leventhal
Hey Jim,
I don't have any ideas off the top of my head, but
you can certainly use DTrace to figure out what's
giving java the order to continue...
Adam

Post by Jim Fiori
Hi,
I'm using the rexit/stop()/prun() dance to capture

process exit in order to grab various procfs commands
on exit. Works fine for simple program like "sleep".
But for Java (and other threaded customer apps) it
runs one or 2 of the p-commands, but the process goes

Post by Jim Fiori
$ cat exit.d
#!/usr/sbin/dtrace -qws
syscall::rexit:entry
/execname == "sleep" || execname == "java"/
{
stop();
system("pargs %d",pid);
system ("pldd %d",pid);
system ("pfiles %d",pid);
system ("pstack %d",pid);
system ("prun %d",pid);
}
$ ./exit.d&
[1] 22487
bash-3.00$ sleep 5
22488: sleep 5
argv[0]: sleep
argv[1]: 5
22488: sleep 5
/lib/libc.so.1

/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3

Post by Jim Fiori
22488: sleep 5
Current rlimit: 256 file descriptors
0: S_IFCHR mode:0620 dev:287,0 ino:12582952

uid:5667 gid:7 rdev:24,18

Post by Jim Fiori
O_RDWR
1: S_IFCHR mode:0620 dev:287,0 ino:12582952

uid:5667 gid:7 rdev:24,18

Post by Jim Fiori
O_RDWR
2: S_IFCHR mode:0620 dev:287,0 ino:12582952

uid:5667 gid:7 rdev:24,18

Post by Jim Fiori
O_RDWR
22488: sleep 5
feea4d18 ecvt (2, 8047290, 8047296, 0, 8047298,

8047306)

Post by Jim Fiori
Looks like all the p-commands completed. But for
0$ /usr/java/bin/java -version
java version "1.5.0_20"
Java(TM) 2 Runtime Environment, Standard Edition

(build 1.5.0_20-b02)

Post by Jim Fiori
Java HotSpot(TM) Client VM (build 1.5.0_20-b02,

mixed mode, sharing)

Post by Jim Fiori
22513: /usr/java/bin/java -version
argv[0]: /usr/java/bin/java
argv[1]: -version
22513: /usr/java/bin/java -version
/lib/libthread.so.1
/lib/libdl.so.1
/lib/libc.so.1

/usr/jdk/instances/jdk1.5.0/jre/lib/i386/client/libjvm
.so

Post by Jim Fiori
/lib/libsocket.so.1
/usr/lib/libsched.so.1
/usr/lib/libCrun.so.1
/lib/libm.so.1
/lib/libnsl.so.1
/lib/libm.so.2
/lib/libscf.so.1
/lib/libdoor.so.1
/lib/libuutil.so.1
/lib/libgen.so.1
/lib/libmd.so.1
/lib/libmp.so.2

/usr/jdk/instances/jdk1.5.0/jre/lib/i386/native_thread
s/libhpi.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libverify.so

Post by Jim Fiori
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libjava.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libzip.so

/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3

Post by Jim Fiori
bash-3.00$ pfiles: cannot examine 22513: no such

process

Post by Jim Fiori
pstack: cannot examine 22513: no such process or

core file

Post by Jim Fiori
prun: cannot control 22513: no such process
Any ideas? Something is telling the process to

continue.

Post by Jim Fiori
Jim
--
This message posted from opensolaris.org
_______________________________________________
dtrace-discuss mailing list

--
Adam Leventhal, Fishworks
http://blogs.sun.com/ahl
_________________________
dtrace-discuss mailing list

[Jim Fiori]

Well it turns out that various proc commands (pldd, pstack to name a few)
send process control messages to the process by writing to /proc/PID/ctl.
One of those commands is PCRUN which makes the process runnable.

I noticed though that if I stopped the process outside of DTrace with pstop,
the process stayed stopped even after running these procfs commands.

So all I had to do to work around this was add a pstop command after the DTrace stop():

stop();
system("pstop %d",pid)
... collect till my heart's content
system("prun %d",pid);

Jim

Post by Jim Fiori
Hi,
I'm using the rexit/stop()/prun() dance to capture
process exit in order to grab various procfs commands
on exit. Works fine for simple program like "sleep".
But for Java (and other threaded customer apps) it
runs one or 2 of the p-commands, but the process goes
$ cat exit.d
#!/usr/sbin/dtrace -qws
syscall::rexit:entry
/execname == "sleep" || execname == "java"/
{
stop();
system("pargs %d",pid);
system ("pldd %d",pid);
system ("pfiles %d",pid);
system ("pstack %d",pid);
system ("prun %d",pid);
$ ./exit.d&
[1] 22487
bash-3.00$ sleep 5
22488: sleep 5
argv[0]: sleep
argv[1]: 5
22488: sleep 5
/lib/libc.so.1
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
22488: sleep 5
Current rlimit: 256 file descriptors
0: S_IFCHR mode:0620 dev:287,0 ino:12582952 uid:5667
gid:7 rdev:24,18
O_RDWR
1: S_IFCHR mode:0620 dev:287,0 ino:12582952
uid:5667 gid:7 rdev:24,18
O_RDWR
2: S_IFCHR mode:0620 dev:287,0 ino:12582952
uid:5667 gid:7 rdev:24,18
O_RDWR
2488: sleep 5
feea4d18 ecvt (2, 8047290, 8047296, 0, 8047298,
8047306)
Looks like all the p-commands completed. But for
0$ /usr/java/bin/java -version
java version "1.5.0_20"
Java(TM) 2 Runtime Environment, Standard Edition
(build 1.5.0_20-b02)
Java HotSpot(TM) Client VM (build 1.5.0_20-b02, mixed
mode, sharing)
22513: /usr/java/bin/java -version
argv[0]: /usr/java/bin/java
argv[1]: -version
22513: /usr/java/bin/java -version
/lib/libthread.so.1
/lib/libdl.so.1
/lib/libc.so.1
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/client/libjvm
.so
/lib/libsocket.so.1
/usr/lib/libsched.so.1
/usr/lib/libCrun.so.1
/lib/libm.so.1
/lib/libnsl.so.1
/lib/libm.so.2
/lib/libscf.so.1
/lib/libdoor.so.1
/lib/libuutil.so.1
/lib/libgen.so.1
/lib/libmd.so.1
/lib/libmp.so.2
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/native_thread
s/libhpi.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libverify.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libjava.so
/usr/jdk/instances/jdk1.5.0/jre/lib/i386/libzip.so
/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
bash-3.00$ pfiles: cannot examine 22513: no such
process
pstack: cannot examine 22513: no such process or core
file
prun: cannot control 22513: no such process
Any ideas? Something is telling the process to
continue.
Jim