dtrace nfs requests on a zfs filesystem

[wessels]

Hi,

I'm issuing the following statement on a ONNV_104 (I know a bit old
but very stable) NFS server:
# dtrace -n 'nfsv3:::op-read-start,nfsv3:::op-write-start
{@[probefunc,args[1]->noi_curpath]=count(); }'

which works fine...most of the time but not always.
Usually it resolve's the filenames on which the I/O are done but not
always. It displays <unknown> as filename. None of the unknown
filenames are created and/or deleted during the dtrace. They are all
VM images, which only get modified. zpool scrub doesn't show any
errors.
Looking in the code I think that zfs_vget in zfs_vfsops.c gets called
for filename resolution. But I can't figure out why this fails for
certain files. Allways the same files get resolved.

Thank for looking into this.

Frederik

[David Pacheco]

Post by wessels
I'm issuing the following statement on a ONNV_104 (I know a bit old
# dtrace -n 'nfsv3:::op-read-start,nfsv3:::op-write-start
which works fine...most of the time but not always.
Usually it resolve's the filenames on which the I/O are done but not
always. It displays <unknown> as filename.

There's this:

http://mail.opensolaris.org/pipermail/dtrace-discuss/2010-February/008527.html

-- Dave

[Kyle Hailey]

I've had issues like this as well. Fortunately Eric Schrock and Adam
Leventhal work down the all from me and I ran this past them. Eric had the
suggestion to simply do a "find" from the directory that I'm interested in
getting file names for and that did the trick. For example all of my tracing
is on

/domain0

so I simply do

$ cd /domain0
$ find . > /dev/null

and then do my tracing. Since adding this trick, I haven't had problems with
"unknown"

- Kyle Hailey

O: +1.415.341.3430
F: +1.650.494.1676
275 Middlefield Road, Suite 50
Menlo Park, CA 94025
http://www.delphix.com

Post by David Pacheco

Post by wessels
I'm issuing the following statement on a ONNV_104 (I know a bit old
# dtrace -n 'nfsv3:::op-read-start,nfsv3:::op-write-start
which works fine...most of the time but not always.
Usually it resolve's the filenames on which the I/O are done but not
always. It displays <unknown> as filename.

http://mail.opensolaris.org/pipermail/dtrace-discuss/2010-February/008527.html
-- Dave
_______________________________________________
dtrace-discuss mailing list

[wessels]

I was aware of that suggestion but found it so ugly that I never tried it. I
ditched pride and embraced pragmatism. And yes running find did resolve all
names. Could you be so kind and trouble the gentleman down the hall and ask
them what's happening and why? Perhaps something can be done about it in
code. With some pointers I might look into it.

Thanks again.

Post by Kyle Hailey
I've had issues like this as well. Fortunately Eric Schrock and Adam
Leventhal work down the all from me and I ran this past them. Eric had the
suggestion to simply do a "find" from the directory that I'm interested in
getting file names for and that did the trick. For example all of my tracing
is on
/domain0
so I simply do
$ cd /domain0
$ find . > /dev/null
and then do my tracing. Since adding this trick, I haven't had problems
with "unknown"
- Kyle Hailey
O: +1.415.341.3430
F: +1.650.494.1676
275 Middlefield Road, Suite 50
Menlo Park, CA 94025
http://www.delphix.com

Post by David Pacheco

Post by wessels
I'm issuing the following statement on a ONNV_104 (I know a bit old
# dtrace -n 'nfsv3:::op-read-start,nfsv3:::op-write-start
which works fine...most of the time but not always.
Usually it resolve's the filenames on which the I/O are done but not
always. It displays <unknown> as filename.

http://mail.opensolaris.org/pipermail/dtrace-discuss/2010-February/008527.html
-- Dave
_______________________________________________
dtrace-discuss mailing list

[C***@oracle.com]

Post by wessels
I was aware of that suggestion but found it so ugly that I never tried it. I
ditched pride and embraced pragmatism. And yes running find did resolve all
names. Could you be so kind and trouble the gentleman down the hall and ask
them what's happening and why? Perhaps something can be done about it in
code. With some pointers I might look into it.

The pathnames are cached with the vnode but they are only created when the
filename is looked up (open, stat, etc). Find will look up all the files;
but without that, it won't know the name. NFSv3 is stateless and the
look up could have been done a long time ago (the server might be
rebooted).

Casper