How to POSIX-ly count the number of lines in a string variable?
up vote
10
down vote
favorite
I know I can do this in Bash:
wc -l <<< "${string_variable}"
Basically, everything I found involved <<<
Bash operator.
But in POSIX shell, <<<
is undefined, and I have been unable to find an alternative approach for hours. I am quite sure there is a simple solution to this, but unfortunately, I didn't find it so far.
shell-script variable string posix
add a comment |
up vote
10
down vote
favorite
I know I can do this in Bash:
wc -l <<< "${string_variable}"
Basically, everything I found involved <<<
Bash operator.
But in POSIX shell, <<<
is undefined, and I have been unable to find an alternative approach for hours. I am quite sure there is a simple solution to this, but unfortunately, I didn't find it so far.
shell-script variable string posix
add a comment |
up vote
10
down vote
favorite
up vote
10
down vote
favorite
I know I can do this in Bash:
wc -l <<< "${string_variable}"
Basically, everything I found involved <<<
Bash operator.
But in POSIX shell, <<<
is undefined, and I have been unable to find an alternative approach for hours. I am quite sure there is a simple solution to this, but unfortunately, I didn't find it so far.
shell-script variable string posix
I know I can do this in Bash:
wc -l <<< "${string_variable}"
Basically, everything I found involved <<<
Bash operator.
But in POSIX shell, <<<
is undefined, and I have been unable to find an alternative approach for hours. I am quite sure there is a simple solution to this, but unfortunately, I didn't find it so far.
shell-script variable string posix
shell-script variable string posix
asked 2 days ago
Vlastimil
7,3971156132
7,3971156132
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
up vote
10
down vote
accepted
The simple answer is that wc -l <<< "${string_variable}"
is a ksh/bash/zsh shortcut for printf "%sn" "${string_variable}" | wc -l
.
There are actually differences in the way <<<
and a pipe work: <<<
creates a temporary file that is passed as input to the command, whereas |
creates a pipe. In bash and pdksh/mksh (but not in ksh93 or zsh), the command on right-hand side of the pipe runs in a subshell. But these differences don't matter in this particular case.
Note that in terms of counting lines, this assumes that the variable is not empty and does not end with a newline. Not ending with a newline is the case when the variable is the result of a command substitution, so you'll get the right result in most cases, but you'll get 1 for the empty string.
There are two differences between var=$(somecommand); wc -l <<<"$var"
and somecommand | wc -l
: using a command substitution and a temporary variable strips away blank lines at the end, forgets whether the last line of output ended in a newline or not (it always does if the command outputs a valid nonempty text file), and overcounts by one if the output is empty. If you want to both preserve the result and count lines, you can do it by appending some known text and stripping it off at the end:
output=$(somecommand; echo .)
line_count=$(($(printf "%sn" "$output" | wc -l) - 1))
printf "The exact output is:n%s" "${output%.}"
1
@Inian Keepingwc -l
is exactly equivalent to the original:<<<$foo
adds a newline to the value of$foo
(even if$foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.
– Gilles
2 days ago
add a comment |
up vote
2
down vote
Not conforming to shell built-ins, using external utilities like grep
and awk
with POSIX compliant options,
string_variable="one
two
three
four"
Doing with grep
to match start of lines
printf '%s' "${string_variable}" | grep -c '^'
4
And with awk
printf '%s' "${string_variable}" | awk 'BEGIN { count=0 } NF { count++ } END { print count }'
Note that some of the GNU tools, especially, GNU grep
does not respect POSIXLY_CORRECT=1
option to run the POSIX version of the tool. In grep
the only behavior affected by setting the variable will be the difference in processing of the order of the command line flags. From the documentation (GNU grep
manual), it seems that
POSIXLY_CORRECT
If set, grep behaves as POSIX requires; otherwise,
grep
behaves more like other GNU programs. POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options.
See How to use POSIXLY_CORRECT in grep?
2
Surelywc -l
is still viable here?
– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use withprintf
, e.g.printf '%s' "${string_variable}" | wc -l
might not work as expected but<<<
would because of the trailingn
appended by the herestring
– Inian
2 days ago
1
That was whatprintf '%sn'
was doing, before you took it out...
– Michael Homer
2 days ago
add a comment |
up vote
0
down vote
The here-string <<<
is pretty much a one-line version of the here-document <<
. The former isn't a standard feature, but the latter is. You can use <<
too in this case. These should be equivalent:
wc -l <<< "$somevar"
wc -l << EOF
$somevar
EOF
Though do note that both add an extra newline at the end of $somevar
, e.g. this prints 6
, even though the variable only has five lines :
s=$'foonnnbarnn'
wc -l <<< "$s"
With printf
, you could decide if you want the additional newline or not:
printf "%sn" "$s" | wc -l # 6
printf "%s" "$s" | wc -l # 5
But then, do note that wc
only counts complete lines (or the number of newline characters in the string). grep -c ^
should also count the final line fragment.
s='foo'
printf "%s" "$s" | wc -l # 0 !
printf "%s" "$s" | grep -c ^ # 1
(Of course you could also count the lines entirely in the shell by using the ${var%...}
expansion to remove them one at a time in a loop...)
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
10
down vote
accepted
The simple answer is that wc -l <<< "${string_variable}"
is a ksh/bash/zsh shortcut for printf "%sn" "${string_variable}" | wc -l
.
There are actually differences in the way <<<
and a pipe work: <<<
creates a temporary file that is passed as input to the command, whereas |
creates a pipe. In bash and pdksh/mksh (but not in ksh93 or zsh), the command on right-hand side of the pipe runs in a subshell. But these differences don't matter in this particular case.
Note that in terms of counting lines, this assumes that the variable is not empty and does not end with a newline. Not ending with a newline is the case when the variable is the result of a command substitution, so you'll get the right result in most cases, but you'll get 1 for the empty string.
There are two differences between var=$(somecommand); wc -l <<<"$var"
and somecommand | wc -l
: using a command substitution and a temporary variable strips away blank lines at the end, forgets whether the last line of output ended in a newline or not (it always does if the command outputs a valid nonempty text file), and overcounts by one if the output is empty. If you want to both preserve the result and count lines, you can do it by appending some known text and stripping it off at the end:
output=$(somecommand; echo .)
line_count=$(($(printf "%sn" "$output" | wc -l) - 1))
printf "The exact output is:n%s" "${output%.}"
1
@Inian Keepingwc -l
is exactly equivalent to the original:<<<$foo
adds a newline to the value of$foo
(even if$foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.
– Gilles
2 days ago
add a comment |
up vote
10
down vote
accepted
The simple answer is that wc -l <<< "${string_variable}"
is a ksh/bash/zsh shortcut for printf "%sn" "${string_variable}" | wc -l
.
There are actually differences in the way <<<
and a pipe work: <<<
creates a temporary file that is passed as input to the command, whereas |
creates a pipe. In bash and pdksh/mksh (but not in ksh93 or zsh), the command on right-hand side of the pipe runs in a subshell. But these differences don't matter in this particular case.
Note that in terms of counting lines, this assumes that the variable is not empty and does not end with a newline. Not ending with a newline is the case when the variable is the result of a command substitution, so you'll get the right result in most cases, but you'll get 1 for the empty string.
There are two differences between var=$(somecommand); wc -l <<<"$var"
and somecommand | wc -l
: using a command substitution and a temporary variable strips away blank lines at the end, forgets whether the last line of output ended in a newline or not (it always does if the command outputs a valid nonempty text file), and overcounts by one if the output is empty. If you want to both preserve the result and count lines, you can do it by appending some known text and stripping it off at the end:
output=$(somecommand; echo .)
line_count=$(($(printf "%sn" "$output" | wc -l) - 1))
printf "The exact output is:n%s" "${output%.}"
1
@Inian Keepingwc -l
is exactly equivalent to the original:<<<$foo
adds a newline to the value of$foo
(even if$foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.
– Gilles
2 days ago
add a comment |
up vote
10
down vote
accepted
up vote
10
down vote
accepted
The simple answer is that wc -l <<< "${string_variable}"
is a ksh/bash/zsh shortcut for printf "%sn" "${string_variable}" | wc -l
.
There are actually differences in the way <<<
and a pipe work: <<<
creates a temporary file that is passed as input to the command, whereas |
creates a pipe. In bash and pdksh/mksh (but not in ksh93 or zsh), the command on right-hand side of the pipe runs in a subshell. But these differences don't matter in this particular case.
Note that in terms of counting lines, this assumes that the variable is not empty and does not end with a newline. Not ending with a newline is the case when the variable is the result of a command substitution, so you'll get the right result in most cases, but you'll get 1 for the empty string.
There are two differences between var=$(somecommand); wc -l <<<"$var"
and somecommand | wc -l
: using a command substitution and a temporary variable strips away blank lines at the end, forgets whether the last line of output ended in a newline or not (it always does if the command outputs a valid nonempty text file), and overcounts by one if the output is empty. If you want to both preserve the result and count lines, you can do it by appending some known text and stripping it off at the end:
output=$(somecommand; echo .)
line_count=$(($(printf "%sn" "$output" | wc -l) - 1))
printf "The exact output is:n%s" "${output%.}"
The simple answer is that wc -l <<< "${string_variable}"
is a ksh/bash/zsh shortcut for printf "%sn" "${string_variable}" | wc -l
.
There are actually differences in the way <<<
and a pipe work: <<<
creates a temporary file that is passed as input to the command, whereas |
creates a pipe. In bash and pdksh/mksh (but not in ksh93 or zsh), the command on right-hand side of the pipe runs in a subshell. But these differences don't matter in this particular case.
Note that in terms of counting lines, this assumes that the variable is not empty and does not end with a newline. Not ending with a newline is the case when the variable is the result of a command substitution, so you'll get the right result in most cases, but you'll get 1 for the empty string.
There are two differences between var=$(somecommand); wc -l <<<"$var"
and somecommand | wc -l
: using a command substitution and a temporary variable strips away blank lines at the end, forgets whether the last line of output ended in a newline or not (it always does if the command outputs a valid nonempty text file), and overcounts by one if the output is empty. If you want to both preserve the result and count lines, you can do it by appending some known text and stripping it off at the end:
output=$(somecommand; echo .)
line_count=$(($(printf "%sn" "$output" | wc -l) - 1))
printf "The exact output is:n%s" "${output%.}"
edited yesterday
ilkkachu
53.8k781146
53.8k781146
answered 2 days ago
Gilles
521k12610401570
521k12610401570
1
@Inian Keepingwc -l
is exactly equivalent to the original:<<<$foo
adds a newline to the value of$foo
(even if$foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.
– Gilles
2 days ago
add a comment |
1
@Inian Keepingwc -l
is exactly equivalent to the original:<<<$foo
adds a newline to the value of$foo
(even if$foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.
– Gilles
2 days ago
1
1
@Inian Keeping
wc -l
is exactly equivalent to the original: <<<$foo
adds a newline to the value of $foo
(even if $foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.– Gilles
2 days ago
@Inian Keeping
wc -l
is exactly equivalent to the original: <<<$foo
adds a newline to the value of $foo
(even if $foo
was empty). I explain in my answer why this may not have been what was wanted, but it's what was asked.– Gilles
2 days ago
add a comment |
up vote
2
down vote
Not conforming to shell built-ins, using external utilities like grep
and awk
with POSIX compliant options,
string_variable="one
two
three
four"
Doing with grep
to match start of lines
printf '%s' "${string_variable}" | grep -c '^'
4
And with awk
printf '%s' "${string_variable}" | awk 'BEGIN { count=0 } NF { count++ } END { print count }'
Note that some of the GNU tools, especially, GNU grep
does not respect POSIXLY_CORRECT=1
option to run the POSIX version of the tool. In grep
the only behavior affected by setting the variable will be the difference in processing of the order of the command line flags. From the documentation (GNU grep
manual), it seems that
POSIXLY_CORRECT
If set, grep behaves as POSIX requires; otherwise,
grep
behaves more like other GNU programs. POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options.
See How to use POSIXLY_CORRECT in grep?
2
Surelywc -l
is still viable here?
– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use withprintf
, e.g.printf '%s' "${string_variable}" | wc -l
might not work as expected but<<<
would because of the trailingn
appended by the herestring
– Inian
2 days ago
1
That was whatprintf '%sn'
was doing, before you took it out...
– Michael Homer
2 days ago
add a comment |
up vote
2
down vote
Not conforming to shell built-ins, using external utilities like grep
and awk
with POSIX compliant options,
string_variable="one
two
three
four"
Doing with grep
to match start of lines
printf '%s' "${string_variable}" | grep -c '^'
4
And with awk
printf '%s' "${string_variable}" | awk 'BEGIN { count=0 } NF { count++ } END { print count }'
Note that some of the GNU tools, especially, GNU grep
does not respect POSIXLY_CORRECT=1
option to run the POSIX version of the tool. In grep
the only behavior affected by setting the variable will be the difference in processing of the order of the command line flags. From the documentation (GNU grep
manual), it seems that
POSIXLY_CORRECT
If set, grep behaves as POSIX requires; otherwise,
grep
behaves more like other GNU programs. POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options.
See How to use POSIXLY_CORRECT in grep?
2
Surelywc -l
is still viable here?
– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use withprintf
, e.g.printf '%s' "${string_variable}" | wc -l
might not work as expected but<<<
would because of the trailingn
appended by the herestring
– Inian
2 days ago
1
That was whatprintf '%sn'
was doing, before you took it out...
– Michael Homer
2 days ago
add a comment |
up vote
2
down vote
up vote
2
down vote
Not conforming to shell built-ins, using external utilities like grep
and awk
with POSIX compliant options,
string_variable="one
two
three
four"
Doing with grep
to match start of lines
printf '%s' "${string_variable}" | grep -c '^'
4
And with awk
printf '%s' "${string_variable}" | awk 'BEGIN { count=0 } NF { count++ } END { print count }'
Note that some of the GNU tools, especially, GNU grep
does not respect POSIXLY_CORRECT=1
option to run the POSIX version of the tool. In grep
the only behavior affected by setting the variable will be the difference in processing of the order of the command line flags. From the documentation (GNU grep
manual), it seems that
POSIXLY_CORRECT
If set, grep behaves as POSIX requires; otherwise,
grep
behaves more like other GNU programs. POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options.
See How to use POSIXLY_CORRECT in grep?
Not conforming to shell built-ins, using external utilities like grep
and awk
with POSIX compliant options,
string_variable="one
two
three
four"
Doing with grep
to match start of lines
printf '%s' "${string_variable}" | grep -c '^'
4
And with awk
printf '%s' "${string_variable}" | awk 'BEGIN { count=0 } NF { count++ } END { print count }'
Note that some of the GNU tools, especially, GNU grep
does not respect POSIXLY_CORRECT=1
option to run the POSIX version of the tool. In grep
the only behavior affected by setting the variable will be the difference in processing of the order of the command line flags. From the documentation (GNU grep
manual), it seems that
POSIXLY_CORRECT
If set, grep behaves as POSIX requires; otherwise,
grep
behaves more like other GNU programs. POSIX requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options.
See How to use POSIXLY_CORRECT in grep?
edited 2 days ago
answered 2 days ago
Inian
3,775824
3,775824
2
Surelywc -l
is still viable here?
– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use withprintf
, e.g.printf '%s' "${string_variable}" | wc -l
might not work as expected but<<<
would because of the trailingn
appended by the herestring
– Inian
2 days ago
1
That was whatprintf '%sn'
was doing, before you took it out...
– Michael Homer
2 days ago
add a comment |
2
Surelywc -l
is still viable here?
– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use withprintf
, e.g.printf '%s' "${string_variable}" | wc -l
might not work as expected but<<<
would because of the trailingn
appended by the herestring
– Inian
2 days ago
1
That was whatprintf '%sn'
was doing, before you took it out...
– Michael Homer
2 days ago
2
2
Surely
wc -l
is still viable here?– Michael Homer
2 days ago
Surely
wc -l
is still viable here?– Michael Homer
2 days ago
@MichaelHomer: From what I've observed,
wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use with printf
, e.g. printf '%s' "${string_variable}" | wc -l
might not work as expected but <<<
would because of the trailing n
appended by the herestring– Inian
2 days ago
@MichaelHomer: From what I've observed,
wc -l
needs a proper newline delimited stream (having a trailing 'n` at the end to count properly). One cannot use a simple FIFO to use with printf
, e.g. printf '%s' "${string_variable}" | wc -l
might not work as expected but <<<
would because of the trailing n
appended by the herestring– Inian
2 days ago
1
1
That was what
printf '%sn'
was doing, before you took it out...– Michael Homer
2 days ago
That was what
printf '%sn'
was doing, before you took it out...– Michael Homer
2 days ago
add a comment |
up vote
0
down vote
The here-string <<<
is pretty much a one-line version of the here-document <<
. The former isn't a standard feature, but the latter is. You can use <<
too in this case. These should be equivalent:
wc -l <<< "$somevar"
wc -l << EOF
$somevar
EOF
Though do note that both add an extra newline at the end of $somevar
, e.g. this prints 6
, even though the variable only has five lines :
s=$'foonnnbarnn'
wc -l <<< "$s"
With printf
, you could decide if you want the additional newline or not:
printf "%sn" "$s" | wc -l # 6
printf "%s" "$s" | wc -l # 5
But then, do note that wc
only counts complete lines (or the number of newline characters in the string). grep -c ^
should also count the final line fragment.
s='foo'
printf "%s" "$s" | wc -l # 0 !
printf "%s" "$s" | grep -c ^ # 1
(Of course you could also count the lines entirely in the shell by using the ${var%...}
expansion to remove them one at a time in a loop...)
add a comment |
up vote
0
down vote
The here-string <<<
is pretty much a one-line version of the here-document <<
. The former isn't a standard feature, but the latter is. You can use <<
too in this case. These should be equivalent:
wc -l <<< "$somevar"
wc -l << EOF
$somevar
EOF
Though do note that both add an extra newline at the end of $somevar
, e.g. this prints 6
, even though the variable only has five lines :
s=$'foonnnbarnn'
wc -l <<< "$s"
With printf
, you could decide if you want the additional newline or not:
printf "%sn" "$s" | wc -l # 6
printf "%s" "$s" | wc -l # 5
But then, do note that wc
only counts complete lines (or the number of newline characters in the string). grep -c ^
should also count the final line fragment.
s='foo'
printf "%s" "$s" | wc -l # 0 !
printf "%s" "$s" | grep -c ^ # 1
(Of course you could also count the lines entirely in the shell by using the ${var%...}
expansion to remove them one at a time in a loop...)
add a comment |
up vote
0
down vote
up vote
0
down vote
The here-string <<<
is pretty much a one-line version of the here-document <<
. The former isn't a standard feature, but the latter is. You can use <<
too in this case. These should be equivalent:
wc -l <<< "$somevar"
wc -l << EOF
$somevar
EOF
Though do note that both add an extra newline at the end of $somevar
, e.g. this prints 6
, even though the variable only has five lines :
s=$'foonnnbarnn'
wc -l <<< "$s"
With printf
, you could decide if you want the additional newline or not:
printf "%sn" "$s" | wc -l # 6
printf "%s" "$s" | wc -l # 5
But then, do note that wc
only counts complete lines (or the number of newline characters in the string). grep -c ^
should also count the final line fragment.
s='foo'
printf "%s" "$s" | wc -l # 0 !
printf "%s" "$s" | grep -c ^ # 1
(Of course you could also count the lines entirely in the shell by using the ${var%...}
expansion to remove them one at a time in a loop...)
The here-string <<<
is pretty much a one-line version of the here-document <<
. The former isn't a standard feature, but the latter is. You can use <<
too in this case. These should be equivalent:
wc -l <<< "$somevar"
wc -l << EOF
$somevar
EOF
Though do note that both add an extra newline at the end of $somevar
, e.g. this prints 6
, even though the variable only has five lines :
s=$'foonnnbarnn'
wc -l <<< "$s"
With printf
, you could decide if you want the additional newline or not:
printf "%sn" "$s" | wc -l # 6
printf "%s" "$s" | wc -l # 5
But then, do note that wc
only counts complete lines (or the number of newline characters in the string). grep -c ^
should also count the final line fragment.
s='foo'
printf "%s" "$s" | wc -l # 0 !
printf "%s" "$s" | grep -c ^ # 1
(Of course you could also count the lines entirely in the shell by using the ${var%...}
expansion to remove them one at a time in a loop...)
edited yesterday
answered yesterday
ilkkachu
53.8k781146
53.8k781146
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f482893%2fhow-to-posix-ly-count-the-number-of-lines-in-a-string-variable%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown