Should I never use stdbuf's fully buffered mode?
up vote
2
down vote
favorite
On my system (a recently updated Arch Linux), the manpage for stdbuf
has the following in the "BUGS" section:
On GLIBC platforms, specifying a buffer size, i.e., using fully buffered mode will result in undefined operation.
Apart of being mildly curious why this is the case and what 'undefined operation' means, I'm mostly worried whether or not that means that I should never specify buffer sizes for the command, and whether or not it can explode in my face if I do.
coreutils
New contributor
add a comment |
up vote
2
down vote
favorite
On my system (a recently updated Arch Linux), the manpage for stdbuf
has the following in the "BUGS" section:
On GLIBC platforms, specifying a buffer size, i.e., using fully buffered mode will result in undefined operation.
Apart of being mildly curious why this is the case and what 'undefined operation' means, I'm mostly worried whether or not that means that I should never specify buffer sizes for the command, and whether or not it can explode in my face if I do.
coreutils
New contributor
1
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
2
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given viasetvbuf
. But this a) doesn't seem to affectstdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.
– mosvy
yesterday
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
On my system (a recently updated Arch Linux), the manpage for stdbuf
has the following in the "BUGS" section:
On GLIBC platforms, specifying a buffer size, i.e., using fully buffered mode will result in undefined operation.
Apart of being mildly curious why this is the case and what 'undefined operation' means, I'm mostly worried whether or not that means that I should never specify buffer sizes for the command, and whether or not it can explode in my face if I do.
coreutils
New contributor
On my system (a recently updated Arch Linux), the manpage for stdbuf
has the following in the "BUGS" section:
On GLIBC platforms, specifying a buffer size, i.e., using fully buffered mode will result in undefined operation.
Apart of being mildly curious why this is the case and what 'undefined operation' means, I'm mostly worried whether or not that means that I should never specify buffer sizes for the command, and whether or not it can explode in my face if I do.
coreutils
coreutils
New contributor
New contributor
New contributor
asked 2 days ago
Michail
1134
1134
New contributor
New contributor
1
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
2
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given viasetvbuf
. But this a) doesn't seem to affectstdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.
– mosvy
yesterday
add a comment |
1
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
2
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given viasetvbuf
. But this a) doesn't seem to affectstdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.
– mosvy
yesterday
1
1
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
2
2
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given via
setvbuf
. But this a) doesn't seem to affect stdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.– mosvy
yesterday
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given via
setvbuf
. But this a) doesn't seem to affect stdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.– mosvy
yesterday
add a comment |
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
It looks like that's just a left-over from a previous version of stdbuf
, and does not correspond to the reality anymore.
In a comment from stdbuf
's source code, it says that:
/* Note currently for glibc (2.3.5) the following call does not change
the buffer size, and more problematically does not give any indication
that the new size request was ignored:
setvbuf (stdout, (char*)NULL, _IOFBF, 8192);
But the actual code goes on and (reluctantly?) allocates the
buffer itself instead of relying on the C library to do it, completely bypassing that problematic behavior.
if (size > 0)
{
if (!(buf = malloc (size))) /* will be freed by fclose() */
Also this (from the same comment) doesn't seem to be the case anymore:
Another issue is that on glibc-2.7 the following doesn't buffer
the first write if it's greater than 1 byte.
setvbuf(stdout,buf,_IOFBF,127);
No matter what arguments I give to the -i
and -o
options, it seems to handle them fine. Example:
$ echo 'int main(){ int c; while((c=getchar()) != EOF) putchar(c); }' | cc -include stdio.h -x c - -o slowcat
$ strace -s5 -e trace=read,write stdbuf -i143 -o127 2>&1 ./slowcat </dev/zero >/dev/null | awk '/ELF/{next}1;++n>5{exit}'
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
It looks like that's just a left-over from a previous version of stdbuf
, and does not correspond to the reality anymore.
In a comment from stdbuf
's source code, it says that:
/* Note currently for glibc (2.3.5) the following call does not change
the buffer size, and more problematically does not give any indication
that the new size request was ignored:
setvbuf (stdout, (char*)NULL, _IOFBF, 8192);
But the actual code goes on and (reluctantly?) allocates the
buffer itself instead of relying on the C library to do it, completely bypassing that problematic behavior.
if (size > 0)
{
if (!(buf = malloc (size))) /* will be freed by fclose() */
Also this (from the same comment) doesn't seem to be the case anymore:
Another issue is that on glibc-2.7 the following doesn't buffer
the first write if it's greater than 1 byte.
setvbuf(stdout,buf,_IOFBF,127);
No matter what arguments I give to the -i
and -o
options, it seems to handle them fine. Example:
$ echo 'int main(){ int c; while((c=getchar()) != EOF) putchar(c); }' | cc -include stdio.h -x c - -o slowcat
$ strace -s5 -e trace=read,write stdbuf -i143 -o127 2>&1 ./slowcat </dev/zero >/dev/null | awk '/ELF/{next}1;++n>5{exit}'
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
add a comment |
up vote
2
down vote
accepted
It looks like that's just a left-over from a previous version of stdbuf
, and does not correspond to the reality anymore.
In a comment from stdbuf
's source code, it says that:
/* Note currently for glibc (2.3.5) the following call does not change
the buffer size, and more problematically does not give any indication
that the new size request was ignored:
setvbuf (stdout, (char*)NULL, _IOFBF, 8192);
But the actual code goes on and (reluctantly?) allocates the
buffer itself instead of relying on the C library to do it, completely bypassing that problematic behavior.
if (size > 0)
{
if (!(buf = malloc (size))) /* will be freed by fclose() */
Also this (from the same comment) doesn't seem to be the case anymore:
Another issue is that on glibc-2.7 the following doesn't buffer
the first write if it's greater than 1 byte.
setvbuf(stdout,buf,_IOFBF,127);
No matter what arguments I give to the -i
and -o
options, it seems to handle them fine. Example:
$ echo 'int main(){ int c; while((c=getchar()) != EOF) putchar(c); }' | cc -include stdio.h -x c - -o slowcat
$ strace -s5 -e trace=read,write stdbuf -i143 -o127 2>&1 ./slowcat </dev/zero >/dev/null | awk '/ELF/{next}1;++n>5{exit}'
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
It looks like that's just a left-over from a previous version of stdbuf
, and does not correspond to the reality anymore.
In a comment from stdbuf
's source code, it says that:
/* Note currently for glibc (2.3.5) the following call does not change
the buffer size, and more problematically does not give any indication
that the new size request was ignored:
setvbuf (stdout, (char*)NULL, _IOFBF, 8192);
But the actual code goes on and (reluctantly?) allocates the
buffer itself instead of relying on the C library to do it, completely bypassing that problematic behavior.
if (size > 0)
{
if (!(buf = malloc (size))) /* will be freed by fclose() */
Also this (from the same comment) doesn't seem to be the case anymore:
Another issue is that on glibc-2.7 the following doesn't buffer
the first write if it's greater than 1 byte.
setvbuf(stdout,buf,_IOFBF,127);
No matter what arguments I give to the -i
and -o
options, it seems to handle them fine. Example:
$ echo 'int main(){ int c; while((c=getchar()) != EOF) putchar(c); }' | cc -include stdio.h -x c - -o slowcat
$ strace -s5 -e trace=read,write stdbuf -i143 -o127 2>&1 ./slowcat </dev/zero >/dev/null | awk '/ELF/{next}1;++n>5{exit}'
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
It looks like that's just a left-over from a previous version of stdbuf
, and does not correspond to the reality anymore.
In a comment from stdbuf
's source code, it says that:
/* Note currently for glibc (2.3.5) the following call does not change
the buffer size, and more problematically does not give any indication
that the new size request was ignored:
setvbuf (stdout, (char*)NULL, _IOFBF, 8192);
But the actual code goes on and (reluctantly?) allocates the
buffer itself instead of relying on the C library to do it, completely bypassing that problematic behavior.
if (size > 0)
{
if (!(buf = malloc (size))) /* will be freed by fclose() */
Also this (from the same comment) doesn't seem to be the case anymore:
Another issue is that on glibc-2.7 the following doesn't buffer
the first write if it's greater than 1 byte.
setvbuf(stdout,buf,_IOFBF,127);
No matter what arguments I give to the -i
and -o
options, it seems to handle them fine. Example:
$ echo 'int main(){ int c; while((c=getchar()) != EOF) putchar(c); }' | cc -include stdio.h -x c - -o slowcat
$ strace -s5 -e trace=read,write stdbuf -i143 -o127 2>&1 ./slowcat </dev/zero >/dev/null | awk '/ELF/{next}1;++n>5{exit}'
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
read(0, ""..., 143) = 143
write(1, ""..., 127) = 127
edited yesterday
answered yesterday
mosvy
5,2341323
5,2341323
add a comment |
add a comment |
Michail is a new contributor. Be nice, and check out our Code of Conduct.
Michail is a new contributor. Be nice, and check out our Code of Conduct.
Michail is a new contributor. Be nice, and check out our Code of Conduct.
Michail is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486960%2fshould-i-never-use-stdbufs-fully-buffered-mode%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
"Undefined" means, just that. Anything could happen, including nasal demons flying out of your nose - catb.org/jargon/html/N/nasal-demons.html :-)
– Stephen Harris
2 days ago
2
You can read an explanation in a comment from stdbuf's source.
– mosvy
2 days ago
@mosvy Thanks for the link, it is most helpful! Do I understand it correctly that it means that no actual undefined behaviour is involved, but the command will simply have no effect?
– Michail
2 days ago
The only 'undefined behavior' was that glibc was deciding the buffer size itself when allocating the buffer, instead of honoring the size given via
setvbuf
. But this a) doesn't seem to affectstdbuf
anymore, see answer b) wasn't nasal daemon material in the 1st place.– mosvy
yesterday