hitting scratch space limit?

[Mark Phalan]

I'm getting an "out of scratch space in action" error and I don't know
why. The amount of data being copied around is very small:


# dtrace -n 'simple$target:::start {printf("%s\n",
args[0]->data_string);}' -c "./simple"
dtrace: description 'simple$target:::start ' matched 1 probe
dtrace: pid 1601 has exited
dtrace: error on enabled probe ID 1 (ID 66120:
simple1601:simple:main:start): out of scratch space in action #1 at DIF
offset 148


# cat simple.d
typedef struct _my_data {
uint32_t length; /* length of data buffer */
uint32_t data; /* Pointer to data */
} my_data;

typedef struct data {
string data_string;
} data_t;

translator data_t < my_data *P > {
data_string = strjoin("@",
stringof(copyin((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t))))));
};


# cat simple.c
typedef struct _my_data {
unsigned int length;
char *data;
} my_data;

#include "s.h"

int main() {
my_data data;

data.length = strlen("TEST");
data.data = malloc(data.length);
memcpy(data.data, "TEST", data.length);

SIMPLE_START(&data);
}


If I remove the "strjoin" in the translator it works fine.

-M

[Angelo Rajadurai]

The error message is a little misleading. The limitation is not scratch space but the size of the strings(strsize) in DTrace. This is 256 bytes by default. If your strjoin is bigger than 256 bytes then you would get this error. See

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3863

The way to fix it is to set strsize

#pragma D option strsize=1024 // make sure your string can fit in!

-Angelo

Post by Mark Phalan
I'm getting an "out of scratch space in action" error and I don't know
# dtrace -n 'simple$target:::start {printf("%s\n",
args[0]->data_string);}' -c "./simple"
dtrace: description 'simple$target:::start ' matched 1 probe
dtrace: pid 1601 has exited
simple1601:simple:main:start): out of scratch space in action #1 at DIF
offset 148
# cat simple.d
typedef struct _my_data {
uint32_t length; /* length of data buffer */
uint32_t data; /* Pointer to data */
} my_data;
typedef struct data {
string data_string;
} data_t;
translator data_t < my_data *P > {
stringof(copyin((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t))))));
};
# cat simple.c
typedef struct _my_data {
unsigned int length;
char *data;
} my_data;
#include "s.h"
int main() {
my_data data;
data.length = strlen("TEST");
data.data = malloc(data.length);
memcpy(data.data, "TEST", data.length);
SIMPLE_START(&data);
}
If I remove the "strjoin" in the translator it works fine.
-M
_______________________________________________
dtrace-discuss mailing list

[Mark Phalan]

Post by Angelo Rajadurai
The error message is a little misleading. The limitation is not scratch
space but the size of the strings(strsize) in DTrace. This is 256
bytes by default. If your strjoin is bigger than 256 bytes then you
would get this error. See
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3863
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
The size of the string is just 5 characters "TEST" + "@".

I'll file a bug.

Thanks,

-M

[Mark Phalan]

Post by Mark Phalan

Post by Angelo Rajadurai
The error message is a little misleading. The limitation is not scratch
space but the size of the strings(strsize) in DTrace. This is 256
bytes by default. If your strjoin is bigger than 256 bytes then you
would get this error. See
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3863
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
I'll file a bug.

6955097 "out of scratch space in action" error with small amounts of
data

-M

[Chad Mynhier]

Post by Mark Phalan

Post by Mark Phalan

Post by Angelo Rajadurai
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
I'll file a bug.

6955097 "out of scratch space in action" error with small amounts of
data

I don't think this is a bug in DTrace, I think this is just an
off-by-one error in your original code. Try adding 1 to data.length.

Even though "string" is a separate type in DTrace, a string is still
just stored as a null-terminated sequence of characters. stringof()
isn't doing anything to null-terminate what you give it, it's just
assuming that you're giving it something that's null-terminated. In
this case, it appears that there's enough garbage after the copied-in
string to ...

Oh. I know what's happening. Because there's no null-termination on
the string, strjoin() actually ends up generating the garbage that
makes the copied-in string appear infinitely long.

+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
| T | E | S | T | @ | T | E | S | T | @ | T | E | S | T | ...
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
^ ^
| |
A B

strjoin() is copying from pointer A to pointer B. Pointer A never
hits a '\0', you just end up running out of scratch space.

Chad

[Mark Phalan]

Post by Chad Mynhier

Post by Mark Phalan

Post by Mark Phalan

Post by Angelo Rajadurai
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
I'll file a bug.

6955097 "out of scratch space in action" error with small amounts of
data

I don't think this is a bug in DTrace, I think this is just an
off-by-one error in your original code. Try adding 1 to data.length.

In the simple example I included the string is NULL-terminated. This
isn't the case in reality. strings are represented by a pointer (data)
and a string length (length).

Anyway, I changed the C to look like this:

int main() {
my_data data;

data.length = strlen("TEST") + 1;
data.data = "TEST";

SIMPLE_START(&data);
}

and I see exactly the same error.

Post by Chad Mynhier
Even though "string" is a separate type in DTrace, a string is still
just stored as a null-terminated sequence of characters. stringof()
isn't doing anything to null-terminate what you give it, it's just
assuming that you're giving it something that's null-terminated. In
this case, it appears that there's enough garbage after the copied-in
string to ...

I thought that stringof() was being clever. I see this example:

"To print only as much of the string as the caller intended, use the
copyin() subroutine, which takes a size as its second argument:


syscall::write:entry
{
printf("%s", stringof(copyin(arg1, arg2)));
}"

from: http://docs.sun.com/app/docs/doc/817-6223/chp-user?a=view

which seems to imply that stringof() should NULL-terminate.

Is the above example incorrect?

Post by Chad Mynhier
Oh. I know what's happening. Because there's no null-termination on
the string, strjoin() actually ends up generating the garbage that
makes the copied-in string appear infinitely long.
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
^ ^
| |
A B
strjoin() is copying from pointer A to pointer B. Pointer A never
hits a '\0', you just end up running out of scratch space.

Any ideas how I can workaround this when the string being copied in
isn't null terminated?

Thanks,

-M

[Mark Phalan]

Post by Mark Phalan

Post by Chad Mynhier

Post by Mark Phalan

Post by Mark Phalan

Post by Angelo Rajadurai
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
I'll file a bug.

6955097 "out of scratch space in action" error with small amounts of
data

I don't think this is a bug in DTrace, I think this is just an
off-by-one error in your original code. Try adding 1 to data.length.

In the simple example I included the string is NULL-terminated. This
isn't the case in reality. strings are represented by a pointer (data)
and a string length (length).
int main() {
my_data data;
data.length = strlen("TEST") + 1;
data.data = "TEST";
SIMPLE_START(&data);
}
and I see exactly the same error.

Post by Chad Mynhier
Even though "string" is a separate type in DTrace, a string is still
just stored as a null-terminated sequence of characters. stringof()
isn't doing anything to null-terminate what you give it, it's just
assuming that you're giving it something that's null-terminated. In
this case, it appears that there's enough garbage after the copied-in
string to ...

"To print only as much of the string as the caller intended, use the
syscall::write:entry
{
printf("%s", stringof(copyin(arg1, arg2)));
}"
from: http://docs.sun.com/app/docs/doc/817-6223/chp-user?a=view
which seems to imply that stringof() should NULL-terminate.
Is the above example incorrect?

Post by Chad Mynhier
Oh. I know what's happening. Because there's no null-termination on
the string, strjoin() actually ends up generating the garbage that
makes the copied-in string appear infinitely long.
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
^ ^
| |
A B
strjoin() is copying from pointer A to pointer B. Pointer A never
hits a '\0', you just end up running out of scratch space.

Another data point. This works fine:

data_string = lltostr(strlen(stringof(copyin((uintptr_t)(*((uint32_t
*)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t))) -1))));

i.e. strlen seems to work fine on the result of stringof() and returns
"3".


-M

[Chad Mynhier]

  data_string = lltostr(strlen(stringof(copyin((uintptr_t)(*((uint32_t
*)
      copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
      copyin((uintptr_t)&P->length, sizeof (uint32_t))) -1))));
i.e. strlen seems to work fine on the result of stringof() and returns
"3".

Yep. Again, you're copying data into what's currently a zeroed area
of memory, so you'll get a null-terminated string as long as you're
not writing anything more into scratch space. When you do the
strjoin() in the other example, you overwrite the nulls that existed
there by default.

Chad

[Mark Phalan]

Post by Mark Phalan

Post by Chad Mynhier

Post by Mark Phalan

Post by Mark Phalan

Post by Angelo Rajadurai
The way to fix it is to set strsize
#pragma D option strsize=1024 // make sure your string can fit in!

Thanks for the idea but it doesn't seem to make any difference even if I
increase strsize up to 2097152 (above which I get drops).
I'll file a bug.

6955097 "out of scratch space in action" error with small amounts of
data

I don't think this is a bug in DTrace, I think this is just an
off-by-one error in your original code. Try adding 1 to data.length.

In the simple example I included the string is NULL-terminated. This
isn't the case in reality. strings are represented by a pointer (data)
and a string length (length).
int main() {
my_data data;
data.length = strlen("TEST") + 1;
data.data = "TEST";
SIMPLE_START(&data);
}
and I see exactly the same error.

I take that back. If I simply add one to the length so the NULL char is
copied in it works as expected.

-M

[Mark Phalan]

On Tue, 2010-05-25 at 11:13 +0200, Mark Phalan wrote:
...

Post by Mark Phalan
Any ideas how I can workaround this when the string being copied in
isn't null terminated?

I've managed to find a solution:

data_string = strjoin("@",
substr(copyin((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t)))), 0,
*((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t)))));


I'm open to suggestions for something simpler but at least this works.

Thanks,

-M

[Chad Mynhier]

Post by Mark Phalan

Post by Chad Mynhier
Even though "string" is a separate type in DTrace, a string is still
just stored as a null-terminated sequence of characters.  stringof()
isn't doing anything to null-terminate what you give it, it's just
assuming that you're giving it something that's null-terminated.  In
this case, it appears that there's enough garbage after the copied-in
string to ...

"To print only as much of the string as the caller intended, use the
syscall::write:entry
{
       printf("%s", stringof(copyin(arg1, arg2)));
}"
from: http://docs.sun.com/app/docs/doc/817-6223/chp-user?a=view
which seems to imply that stringof() should NULL-terminate.
Is the above example incorrect?

Nope, the above example is mostly working by sheer luck. In this
case, the destination of the copyin() just happens to be zeroed, so
the string it copies in will necessarily be null-terminated.

Note that stringof() doesn't actually _do_ anything. stringof() isn't
a subroutine that walks what you give it to verify that it's
null-terminated. (And it _can't_ do that, as it doesn't have a length
argument and thus has no way of determining the proper length of that
string.) stringof() isn't much more than a type cast.

OTOH, copyinstr() _does_ take a second argument that specifies a max
length, so the workaround you're looking for is to use that:

data_string = strjoin("@",
copyinstr((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t)))));

copyinstr() will null-terminate the resulting string. You can see
this here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3174.

Chad

[Mark Phalan]

Post by Chad Mynhier

Post by Mark Phalan

Post by Chad Mynhier
Even though "string" is a separate type in DTrace, a string is still
just stored as a null-terminated sequence of characters. stringof()
isn't doing anything to null-terminate what you give it, it's just
assuming that you're giving it something that's null-terminated. In
this case, it appears that there's enough garbage after the copied-in
string to ...

"To print only as much of the string as the caller intended, use the
syscall::write:entry
{
printf("%s", stringof(copyin(arg1, arg2)));
}"
from: http://docs.sun.com/app/docs/doc/817-6223/chp-user?a=view
which seems to imply that stringof() should NULL-terminate.
Is the above example incorrect?

Nope, the above example is mostly working by sheer luck. In this
case, the destination of the copyin() just happens to be zeroed, so
the string it copies in will necessarily be null-terminated.

Ok. I guess its a doc bug then.

Post by Chad Mynhier
Note that stringof() doesn't actually _do_ anything. stringof() isn't
a subroutine that walks what you give it to verify that it's
null-terminated. (And it _can't_ do that, as it doesn't have a length
argument and thus has no way of determining the proper length of that
string.) stringof() isn't much more than a type cast.

Ok.

Post by Chad Mynhier
OTOH, copyinstr() _does_ take a second argument that specifies a max
copyinstr((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t)))));
copyinstr() will null-terminate the resulting string. You can see
this here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3174.

copyinstr() will work when the source is NULL terminated. It mostly
won't be in the real-world case I'll need this for. I think my
workaround with substr() (see other mail) should work though.

Thanks for the help,

-M

[Chad Mynhier]

Post by Mark Phalan

Post by Chad Mynhier
OTOH, copyinstr() _does_ take a second argument that specifies a max
      copyinstr((uintptr_t)(*((uint32_t *)
      copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
      copyin((uintptr_t)&P->length, sizeof (uint32_t)))));
copyinstr() will null-terminate the resulting string.  You can see
this here:  http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3174.

copyinstr() will work when the source is NULL terminated. It mostly
won't be in the real-world case I'll need this for. I think my
workaround with substr() (see other mail) should work though.

I'm confused as to why you think that copyinstr() won't work here. If
you give copyinstr() a length as the second argument, it will behave
exactly like a copyin(), but it will also guarantee to null-terminate
the resulting string. I've tested this, and it works.

Chad

[Mark Phalan]

Post by Chad Mynhier

Post by Mark Phalan

Post by Chad Mynhier
OTOH, copyinstr() _does_ take a second argument that specifies a max
copyinstr((uintptr_t)(*((uint32_t *)
copyin((uintptr_t)&P->data, sizeof (uint32_t)))), *((uint32_t *)
copyin((uintptr_t)&P->length, sizeof (uint32_t)))));
copyinstr() will null-terminate the resulting string. You can see
this here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c#3174.

copyinstr() will work when the source is NULL terminated. It mostly
won't be in the real-world case I'll need this for. I think my
workaround with substr() (see other mail) should work though.

I'm confused as to why you think that copyinstr() won't work here. If
you give copyinstr() a length as the second argument, it will behave
exactly like a copyin(), but it will also guarantee to null-terminate
the resulting string. I've tested this, and it works.

Sorry, it was somehow stuck in my brain that copyinstr() took a single
argument which was null-terminated. I should have looked more closely at
your example.

You're quite right, copyinstr() works fine with the maxlength argument.

Thanks again,

-M