“/usr/bin/stat: Argument list too long” error
up vote
5
down vote
favorite
I need to get the list of files(.log/.lst
) present in a directory($logfolder
) whose creation date is in a specific $year
and $month
stat --format='%y %n' $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
This doesn't work when I query the command to a folder where there are too many files. I get the following error:
-bash: /usr/bin/stat: Argument list too long
shell
add a comment |
up vote
5
down vote
favorite
I need to get the list of files(.log/.lst
) present in a directory($logfolder
) whose creation date is in a specific $year
and $month
stat --format='%y %n' $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
This doesn't work when I query the command to a folder where there are too many files. I get the following error:
-bash: /usr/bin/stat: Argument list too long
shell
Yourgrep
will catch both files with the date in the name and thestat
date.
– Matt
Jun 25 '14 at 16:39
Yourawk
will miss files with spaces in the name.
– Matt
Jun 25 '14 at 16:40
add a comment |
up vote
5
down vote
favorite
up vote
5
down vote
favorite
I need to get the list of files(.log/.lst
) present in a directory($logfolder
) whose creation date is in a specific $year
and $month
stat --format='%y %n' $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
This doesn't work when I query the command to a folder where there are too many files. I get the following error:
-bash: /usr/bin/stat: Argument list too long
shell
I need to get the list of files(.log/.lst
) present in a directory($logfolder
) whose creation date is in a specific $year
and $month
stat --format='%y %n' $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
This doesn't work when I query the command to a folder where there are too many files. I get the following error:
-bash: /usr/bin/stat: Argument list too long
shell
shell
edited Nov 21 at 21:03
Rui F Ribeiro
38.2k1475125
38.2k1475125
asked Jun 25 '14 at 15:44
DCL
262
262
Yourgrep
will catch both files with the date in the name and thestat
date.
– Matt
Jun 25 '14 at 16:39
Yourawk
will miss files with spaces in the name.
– Matt
Jun 25 '14 at 16:40
add a comment |
Yourgrep
will catch both files with the date in the name and thestat
date.
– Matt
Jun 25 '14 at 16:39
Yourawk
will miss files with spaces in the name.
– Matt
Jun 25 '14 at 16:40
Your
grep
will catch both files with the date in the name and the stat
date.– Matt
Jun 25 '14 at 16:39
Your
grep
will catch both files with the date in the name and the stat
date.– Matt
Jun 25 '14 at 16:39
Your
awk
will miss files with spaces in the name.– Matt
Jun 25 '14 at 16:40
Your
awk
will miss files with spaces in the name.– Matt
Jun 25 '14 at 16:40
add a comment |
4 Answers
4
active
oldest
votes
up vote
3
down vote
For a funny possibility, if your find
handles -newerXY
, use it! For example to get the files from year 1977 and month October:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"
Done!
As you already have the variables year
and month
it's straightforward to write as:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"
Only one find
command! amazing!
1
+1 Since the OP'sstat
is of the GNU variant, hisfind
more than likely supports-newermt
as well.
– Stéphane Chazelas
Jun 25 '14 at 18:44
add a comment |
up vote
3
down vote
I would do it like this:
find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
Explanation
By grouping the two -name
calls in parentheses, you can combine them with the -o
(or) flag. This will make find
look for either .log
or .lst
files. The -printf
(a GNU extension) prints the file's modification month (%TB
), then its modification year (%TY
) and then its path (%p
), with a tab (t
) between each field.
The awk
simply checks that the 1st field (the month) is the same as $month
and the second is the same as $year
.
The cut
removes the first two fields (the month and year) and prints everything from the 3rd field on.
I tested the above by creating files modified in December 2012 (and set $month
to "December" and $year
to 2012):
$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst
(note that it assumes file and directory names don't contain newline characters).
add a comment |
up vote
1
down vote
Try this:
find $logfolder -type f -exec stat --format='%y %n' "{}" + |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
grep "^$year-$month-"
so you don't the string in file names
– Matt
Jun 25 '14 at 16:29
@mtm: I only usefind
to fix the errorArgument list too long
. It's very hard to know exactly what the OP want.
– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
awk
can dogrep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
– Stéphane Chazelas
Jun 25 '14 at 18:50
add a comment |
up vote
0
down vote
ls -lh *.log *.lst logfolder | grep year | grep month
add a comment |
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
For a funny possibility, if your find
handles -newerXY
, use it! For example to get the files from year 1977 and month October:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"
Done!
As you already have the variables year
and month
it's straightforward to write as:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"
Only one find
command! amazing!
1
+1 Since the OP'sstat
is of the GNU variant, hisfind
more than likely supports-newermt
as well.
– Stéphane Chazelas
Jun 25 '14 at 18:44
add a comment |
up vote
3
down vote
For a funny possibility, if your find
handles -newerXY
, use it! For example to get the files from year 1977 and month October:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"
Done!
As you already have the variables year
and month
it's straightforward to write as:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"
Only one find
command! amazing!
1
+1 Since the OP'sstat
is of the GNU variant, hisfind
more than likely supports-newermt
as well.
– Stéphane Chazelas
Jun 25 '14 at 18:44
add a comment |
up vote
3
down vote
up vote
3
down vote
For a funny possibility, if your find
handles -newerXY
, use it! For example to get the files from year 1977 and month October:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"
Done!
As you already have the variables year
and month
it's straightforward to write as:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"
Only one find
command! amazing!
For a funny possibility, if your find
handles -newerXY
, use it! For example to get the files from year 1977 and month October:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"
Done!
As you already have the variables year
and month
it's straightforward to write as:
find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"
Only one find
command! amazing!
answered Jun 25 '14 at 18:13
gniourf_gniourf
1,409612
1,409612
1
+1 Since the OP'sstat
is of the GNU variant, hisfind
more than likely supports-newermt
as well.
– Stéphane Chazelas
Jun 25 '14 at 18:44
add a comment |
1
+1 Since the OP'sstat
is of the GNU variant, hisfind
more than likely supports-newermt
as well.
– Stéphane Chazelas
Jun 25 '14 at 18:44
1
1
+1 Since the OP's
stat
is of the GNU variant, his find
more than likely supports -newermt
as well.– Stéphane Chazelas
Jun 25 '14 at 18:44
+1 Since the OP's
stat
is of the GNU variant, his find
more than likely supports -newermt
as well.– Stéphane Chazelas
Jun 25 '14 at 18:44
add a comment |
up vote
3
down vote
I would do it like this:
find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
Explanation
By grouping the two -name
calls in parentheses, you can combine them with the -o
(or) flag. This will make find
look for either .log
or .lst
files. The -printf
(a GNU extension) prints the file's modification month (%TB
), then its modification year (%TY
) and then its path (%p
), with a tab (t
) between each field.
The awk
simply checks that the 1st field (the month) is the same as $month
and the second is the same as $year
.
The cut
removes the first two fields (the month and year) and prints everything from the 3rd field on.
I tested the above by creating files modified in December 2012 (and set $month
to "December" and $year
to 2012):
$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst
(note that it assumes file and directory names don't contain newline characters).
add a comment |
up vote
3
down vote
I would do it like this:
find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
Explanation
By grouping the two -name
calls in parentheses, you can combine them with the -o
(or) flag. This will make find
look for either .log
or .lst
files. The -printf
(a GNU extension) prints the file's modification month (%TB
), then its modification year (%TY
) and then its path (%p
), with a tab (t
) between each field.
The awk
simply checks that the 1st field (the month) is the same as $month
and the second is the same as $year
.
The cut
removes the first two fields (the month and year) and prints everything from the 3rd field on.
I tested the above by creating files modified in December 2012 (and set $month
to "December" and $year
to 2012):
$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst
(note that it assumes file and directory names don't contain newline characters).
add a comment |
up vote
3
down vote
up vote
3
down vote
I would do it like this:
find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
Explanation
By grouping the two -name
calls in parentheses, you can combine them with the -o
(or) flag. This will make find
look for either .log
or .lst
files. The -printf
(a GNU extension) prints the file's modification month (%TB
), then its modification year (%TY
) and then its path (%p
), with a tab (t
) between each field.
The awk
simply checks that the 1st field (the month) is the same as $month
and the second is the same as $year
.
The cut
removes the first two fields (the month and year) and prints everything from the 3rd field on.
I tested the above by creating files modified in December 2012 (and set $month
to "December" and $year
to 2012):
$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst
(note that it assumes file and directory names don't contain newline characters).
I would do it like this:
find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
Explanation
By grouping the two -name
calls in parentheses, you can combine them with the -o
(or) flag. This will make find
look for either .log
or .lst
files. The -printf
(a GNU extension) prints the file's modification month (%TB
), then its modification year (%TY
) and then its path (%p
), with a tab (t
) between each field.
The awk
simply checks that the 1st field (the month) is the same as $month
and the second is the same as $year
.
The cut
removes the first two fields (the month and year) and prints everything from the 3rd field on.
I tested the above by creating files modified in December 2012 (and set $month
to "December" and $year
to 2012):
$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst
(note that it assumes file and directory names don't contain newline characters).
edited Jun 25 '14 at 18:46
Stéphane Chazelas
294k54555898
294k54555898
answered Jun 25 '14 at 17:20
terdon♦
126k31243418
126k31243418
add a comment |
add a comment |
up vote
1
down vote
Try this:
find $logfolder -type f -exec stat --format='%y %n' "{}" + |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
grep "^$year-$month-"
so you don't the string in file names
– Matt
Jun 25 '14 at 16:29
@mtm: I only usefind
to fix the errorArgument list too long
. It's very hard to know exactly what the OP want.
– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
awk
can dogrep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
– Stéphane Chazelas
Jun 25 '14 at 18:50
add a comment |
up vote
1
down vote
Try this:
find $logfolder -type f -exec stat --format='%y %n' "{}" + |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
grep "^$year-$month-"
so you don't the string in file names
– Matt
Jun 25 '14 at 16:29
@mtm: I only usefind
to fix the errorArgument list too long
. It's very hard to know exactly what the OP want.
– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
awk
can dogrep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
– Stéphane Chazelas
Jun 25 '14 at 18:50
add a comment |
up vote
1
down vote
up vote
1
down vote
Try this:
find $logfolder -type f -exec stat --format='%y %n' "{}" + |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
Try this:
find $logfolder -type f -exec stat --format='%y %n' "{}" + |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive
answered Jun 25 '14 at 15:53
cuonglm
101k23196297
101k23196297
grep "^$year-$month-"
so you don't the string in file names
– Matt
Jun 25 '14 at 16:29
@mtm: I only usefind
to fix the errorArgument list too long
. It's very hard to know exactly what the OP want.
– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
awk
can dogrep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
– Stéphane Chazelas
Jun 25 '14 at 18:50
add a comment |
grep "^$year-$month-"
so you don't the string in file names
– Matt
Jun 25 '14 at 16:29
@mtm: I only usefind
to fix the errorArgument list too long
. It's very hard to know exactly what the OP want.
– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
awk
can dogrep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
– Stéphane Chazelas
Jun 25 '14 at 18:50
grep "^$year-$month-"
so you don't the string in file names– Matt
Jun 25 '14 at 16:29
grep "^$year-$month-"
so you don't the string in file names– Matt
Jun 25 '14 at 16:29
@mtm: I only use
find
to fix the error Argument list too long
. It's very hard to know exactly what the OP want.– cuonglm
Jun 25 '14 at 16:33
@mtm: I only use
find
to fix the error Argument list too long
. It's very hard to know exactly what the OP want.– cuonglm
Jun 25 '14 at 16:33
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
No worries.. will comment on OP
– Matt
Jun 25 '14 at 16:39
1
1
awk
can do grep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.– Stéphane Chazelas
Jun 25 '14 at 18:50
awk
can do grep
's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.– Stéphane Chazelas
Jun 25 '14 at 18:50
add a comment |
up vote
0
down vote
ls -lh *.log *.lst logfolder | grep year | grep month
add a comment |
up vote
0
down vote
ls -lh *.log *.lst logfolder | grep year | grep month
add a comment |
up vote
0
down vote
up vote
0
down vote
ls -lh *.log *.lst logfolder | grep year | grep month
ls -lh *.log *.lst logfolder | grep year | grep month
answered Jun 25 '14 at 15:48
schaiba
5,39912028
5,39912028
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f139168%2fusr-bin-stat-argument-list-too-long-error%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Your
grep
will catch both files with the date in the name and thestat
date.– Matt
Jun 25 '14 at 16:39
Your
awk
will miss files with spaces in the name.– Matt
Jun 25 '14 at 16:40