“/usr/bin/stat: Argument list too long” error











up vote
5
down vote

favorite












I need to get the list of files(.log/.lst) present in a directory($logfolder) whose creation date is in a specific $year and $month



stat  --format='%y %n'  $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive


This doesn't work when I query the command to a folder where there are too many files. I get the following error:



-bash: /usr/bin/stat: Argument list too long









share|improve this question
























  • Your grep will catch both files with the date in the name and the stat date.
    – Matt
    Jun 25 '14 at 16:39










  • Your awk will miss files with spaces in the name.
    – Matt
    Jun 25 '14 at 16:40















up vote
5
down vote

favorite












I need to get the list of files(.log/.lst) present in a directory($logfolder) whose creation date is in a specific $year and $month



stat  --format='%y %n'  $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive


This doesn't work when I query the command to a folder where there are too many files. I get the following error:



-bash: /usr/bin/stat: Argument list too long









share|improve this question
























  • Your grep will catch both files with the date in the name and the stat date.
    – Matt
    Jun 25 '14 at 16:39










  • Your awk will miss files with spaces in the name.
    – Matt
    Jun 25 '14 at 16:40













up vote
5
down vote

favorite









up vote
5
down vote

favorite











I need to get the list of files(.log/.lst) present in a directory($logfolder) whose creation date is in a specific $year and $month



stat  --format='%y %n'  $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive


This doesn't work when I query the command to a folder where there are too many files. I get the following error:



-bash: /usr/bin/stat: Argument list too long









share|improve this question















I need to get the list of files(.log/.lst) present in a directory($logfolder) whose creation date is in a specific $year and $month



stat  --format='%y %n'  $logfolder/* |
grep "$year-$month-"|
awk -F' ' '{print $4}'|
grep 'log$|lst$' > $archivepath/filesToArchive


This doesn't work when I query the command to a folder where there are too many files. I get the following error:



-bash: /usr/bin/stat: Argument list too long






shell






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 21 at 21:03









Rui F Ribeiro

38.2k1475125




38.2k1475125










asked Jun 25 '14 at 15:44









DCL

262




262












  • Your grep will catch both files with the date in the name and the stat date.
    – Matt
    Jun 25 '14 at 16:39










  • Your awk will miss files with spaces in the name.
    – Matt
    Jun 25 '14 at 16:40


















  • Your grep will catch both files with the date in the name and the stat date.
    – Matt
    Jun 25 '14 at 16:39










  • Your awk will miss files with spaces in the name.
    – Matt
    Jun 25 '14 at 16:40
















Your grep will catch both files with the date in the name and the stat date.
– Matt
Jun 25 '14 at 16:39




Your grep will catch both files with the date in the name and the stat date.
– Matt
Jun 25 '14 at 16:39












Your awk will miss files with spaces in the name.
– Matt
Jun 25 '14 at 16:40




Your awk will miss files with spaces in the name.
– Matt
Jun 25 '14 at 16:40










4 Answers
4






active

oldest

votes

















up vote
3
down vote













For a funny possibility, if your find handles -newerXY, use it! For example to get the files from year 1977 and month October:



find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"


Done!



As you already have the variables year and month it's straightforward to write as:



find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"


Only one find command! amazing!






share|improve this answer

















  • 1




    +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
    – Stéphane Chazelas
    Jun 25 '14 at 18:44


















up vote
3
down vote













I would do it like this:



find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-


Explanation



By grouping the two -name calls in parentheses, you can combine them with the -o (or) flag. This will make find look for either .log or .lst files. The -printf (a GNU extension) prints the file's modification month (%TB), then its modification year (%TY) and then its path (%p), with a tab (t) between each field.



The awk simply checks that the 1st field (the month) is the same as $month and the second is the same as $year.



The cut removes the first two fields (the month and year) and prints everything from the 3rd field on.



I tested the above by creating files modified in December 2012 (and set $month to "December" and $year to 2012):



$ touch -d "December 13 2012" {a,b,c}{.lst,.log}
$ touch c.lst a.log ## c.lst and a.log now have today's modification date.
$ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
./b.log
./c.log
./b.lst
./a.lst


(note that it assumes file and directory names don't contain newline characters).






share|improve this answer






























    up vote
    1
    down vote













    Try this:



    find $logfolder -type f -exec stat --format='%y %n' "{}" + |
    grep "$year-$month-"|
    awk -F' ' '{print $4}'|
    grep 'log$|lst$' > $archivepath/filesToArchive





    share|improve this answer





















    • grep "^$year-$month-" so you don't the string in file names
      – Matt
      Jun 25 '14 at 16:29










    • @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
      – cuonglm
      Jun 25 '14 at 16:33










    • No worries.. will comment on OP
      – Matt
      Jun 25 '14 at 16:39






    • 1




      awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
      – Stéphane Chazelas
      Jun 25 '14 at 18:50


















    up vote
    0
    down vote













    ls -lh *.log *.lst logfolder | grep year | grep month






    share|improve this answer





















      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "106"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














       

      draft saved


      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f139168%2fusr-bin-stat-argument-list-too-long-error%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      4 Answers
      4






      active

      oldest

      votes








      4 Answers
      4






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      3
      down vote













      For a funny possibility, if your find handles -newerXY, use it! For example to get the files from year 1977 and month October:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"


      Done!



      As you already have the variables year and month it's straightforward to write as:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"


      Only one find command! amazing!






      share|improve this answer

















      • 1




        +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
        – Stéphane Chazelas
        Jun 25 '14 at 18:44















      up vote
      3
      down vote













      For a funny possibility, if your find handles -newerXY, use it! For example to get the files from year 1977 and month October:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"


      Done!



      As you already have the variables year and month it's straightforward to write as:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"


      Only one find command! amazing!






      share|improve this answer

















      • 1




        +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
        – Stéphane Chazelas
        Jun 25 '14 at 18:44













      up vote
      3
      down vote










      up vote
      3
      down vote









      For a funny possibility, if your find handles -newerXY, use it! For example to get the files from year 1977 and month October:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"


      Done!



      As you already have the variables year and month it's straightforward to write as:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"


      Only one find command! amazing!






      share|improve this answer












      For a funny possibility, if your find handles -newerXY, use it! For example to get the files from year 1977 and month October:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "1977-10-01" ! -newermt "1977-10-01 +1 month"


      Done!



      As you already have the variables year and month it's straightforward to write as:



      find "$logfolder" ( -name '*.log' -o -name '*.lst' ) -newermt "$year-$month-01" ! -newermt "$year-$month-01 +1 month"


      Only one find command! amazing!







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Jun 25 '14 at 18:13









      gniourf_gniourf

      1,409612




      1,409612








      • 1




        +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
        – Stéphane Chazelas
        Jun 25 '14 at 18:44














      • 1




        +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
        – Stéphane Chazelas
        Jun 25 '14 at 18:44








      1




      1




      +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
      – Stéphane Chazelas
      Jun 25 '14 at 18:44




      +1 Since the OP's stat is of the GNU variant, his find more than likely supports -newermt as well.
      – Stéphane Chazelas
      Jun 25 '14 at 18:44












      up vote
      3
      down vote













      I would do it like this:



      find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
      awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-


      Explanation



      By grouping the two -name calls in parentheses, you can combine them with the -o (or) flag. This will make find look for either .log or .lst files. The -printf (a GNU extension) prints the file's modification month (%TB), then its modification year (%TY) and then its path (%p), with a tab (t) between each field.



      The awk simply checks that the 1st field (the month) is the same as $month and the second is the same as $year.



      The cut removes the first two fields (the month and year) and prints everything from the 3rd field on.



      I tested the above by creating files modified in December 2012 (and set $month to "December" and $year to 2012):



      $ touch -d "December 13 2012" {a,b,c}{.lst,.log}
      $ touch c.lst a.log ## c.lst and a.log now have today's modification date.
      $ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
      awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
      ./b.log
      ./c.log
      ./b.lst
      ./a.lst


      (note that it assumes file and directory names don't contain newline characters).






      share|improve this answer



























        up vote
        3
        down vote













        I would do it like this:



        find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
        awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-


        Explanation



        By grouping the two -name calls in parentheses, you can combine them with the -o (or) flag. This will make find look for either .log or .lst files. The -printf (a GNU extension) prints the file's modification month (%TB), then its modification year (%TY) and then its path (%p), with a tab (t) between each field.



        The awk simply checks that the 1st field (the month) is the same as $month and the second is the same as $year.



        The cut removes the first two fields (the month and year) and prints everything from the 3rd field on.



        I tested the above by creating files modified in December 2012 (and set $month to "December" and $year to 2012):



        $ touch -d "December 13 2012" {a,b,c}{.lst,.log}
        $ touch c.lst a.log ## c.lst and a.log now have today's modification date.
        $ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
        awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
        ./b.log
        ./c.log
        ./b.lst
        ./a.lst


        (note that it assumes file and directory names don't contain newline characters).






        share|improve this answer

























          up vote
          3
          down vote










          up vote
          3
          down vote









          I would do it like this:



          find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
          awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-


          Explanation



          By grouping the two -name calls in parentheses, you can combine them with the -o (or) flag. This will make find look for either .log or .lst files. The -printf (a GNU extension) prints the file's modification month (%TB), then its modification year (%TY) and then its path (%p), with a tab (t) between each field.



          The awk simply checks that the 1st field (the month) is the same as $month and the second is the same as $year.



          The cut removes the first two fields (the month and year) and prints everything from the 3rd field on.



          I tested the above by creating files modified in December 2012 (and set $month to "December" and $year to 2012):



          $ touch -d "December 13 2012" {a,b,c}{.lst,.log}
          $ touch c.lst a.log ## c.lst and a.log now have today's modification date.
          $ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
          awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
          ./b.log
          ./c.log
          ./b.lst
          ./a.lst


          (note that it assumes file and directory names don't contain newline characters).






          share|improve this answer














          I would do it like this:



          find "$logfolder" ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
          awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-


          Explanation



          By grouping the two -name calls in parentheses, you can combine them with the -o (or) flag. This will make find look for either .log or .lst files. The -printf (a GNU extension) prints the file's modification month (%TB), then its modification year (%TY) and then its path (%p), with a tab (t) between each field.



          The awk simply checks that the 1st field (the month) is the same as $month and the second is the same as $year.



          The cut removes the first two fields (the month and year) and prints everything from the 3rd field on.



          I tested the above by creating files modified in December 2012 (and set $month to "December" and $year to 2012):



          $ touch -d "December 13 2012" {a,b,c}{.lst,.log}
          $ touch c.lst a.log ## c.lst and a.log now have today's modification date.
          $ find $logfolder ( -name '*.log' -o -name '*lst' ) -printf "%TBt%TYt%pn" |
          awk '$1==m && $2==y' m="$month" y="$year" | cut -f 3-
          ./b.log
          ./c.log
          ./b.lst
          ./a.lst


          (note that it assumes file and directory names don't contain newline characters).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jun 25 '14 at 18:46









          Stéphane Chazelas

          294k54555898




          294k54555898










          answered Jun 25 '14 at 17:20









          terdon

          126k31243418




          126k31243418






















              up vote
              1
              down vote













              Try this:



              find $logfolder -type f -exec stat --format='%y %n' "{}" + |
              grep "$year-$month-"|
              awk -F' ' '{print $4}'|
              grep 'log$|lst$' > $archivepath/filesToArchive





              share|improve this answer





















              • grep "^$year-$month-" so you don't the string in file names
                – Matt
                Jun 25 '14 at 16:29










              • @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
                – cuonglm
                Jun 25 '14 at 16:33










              • No worries.. will comment on OP
                – Matt
                Jun 25 '14 at 16:39






              • 1




                awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
                – Stéphane Chazelas
                Jun 25 '14 at 18:50















              up vote
              1
              down vote













              Try this:



              find $logfolder -type f -exec stat --format='%y %n' "{}" + |
              grep "$year-$month-"|
              awk -F' ' '{print $4}'|
              grep 'log$|lst$' > $archivepath/filesToArchive





              share|improve this answer





















              • grep "^$year-$month-" so you don't the string in file names
                – Matt
                Jun 25 '14 at 16:29










              • @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
                – cuonglm
                Jun 25 '14 at 16:33










              • No worries.. will comment on OP
                – Matt
                Jun 25 '14 at 16:39






              • 1




                awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
                – Stéphane Chazelas
                Jun 25 '14 at 18:50













              up vote
              1
              down vote










              up vote
              1
              down vote









              Try this:



              find $logfolder -type f -exec stat --format='%y %n' "{}" + |
              grep "$year-$month-"|
              awk -F' ' '{print $4}'|
              grep 'log$|lst$' > $archivepath/filesToArchive





              share|improve this answer












              Try this:



              find $logfolder -type f -exec stat --format='%y %n' "{}" + |
              grep "$year-$month-"|
              awk -F' ' '{print $4}'|
              grep 'log$|lst$' > $archivepath/filesToArchive






              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Jun 25 '14 at 15:53









              cuonglm

              101k23196297




              101k23196297












              • grep "^$year-$month-" so you don't the string in file names
                – Matt
                Jun 25 '14 at 16:29










              • @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
                – cuonglm
                Jun 25 '14 at 16:33










              • No worries.. will comment on OP
                – Matt
                Jun 25 '14 at 16:39






              • 1




                awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
                – Stéphane Chazelas
                Jun 25 '14 at 18:50


















              • grep "^$year-$month-" so you don't the string in file names
                – Matt
                Jun 25 '14 at 16:29










              • @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
                – cuonglm
                Jun 25 '14 at 16:33










              • No worries.. will comment on OP
                – Matt
                Jun 25 '14 at 16:39






              • 1




                awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
                – Stéphane Chazelas
                Jun 25 '14 at 18:50
















              grep "^$year-$month-" so you don't the string in file names
              – Matt
              Jun 25 '14 at 16:29




              grep "^$year-$month-" so you don't the string in file names
              – Matt
              Jun 25 '14 at 16:29












              @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
              – cuonglm
              Jun 25 '14 at 16:33




              @mtm: I only use find to fix the error Argument list too long. It's very hard to know exactly what the OP want.
              – cuonglm
              Jun 25 '14 at 16:33












              No worries.. will comment on OP
              – Matt
              Jun 25 '14 at 16:39




              No worries.. will comment on OP
              – Matt
              Jun 25 '14 at 16:39




              1




              1




              awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
              – Stéphane Chazelas
              Jun 25 '14 at 18:50




              awk can do grep's job and more, you almost never need to pipe awk and grep together. Note that your last grep command is not portable/standard.
              – Stéphane Chazelas
              Jun 25 '14 at 18:50










              up vote
              0
              down vote













              ls -lh *.log *.lst logfolder | grep year | grep month






              share|improve this answer

























                up vote
                0
                down vote













                ls -lh *.log *.lst logfolder | grep year | grep month






                share|improve this answer























                  up vote
                  0
                  down vote










                  up vote
                  0
                  down vote









                  ls -lh *.log *.lst logfolder | grep year | grep month






                  share|improve this answer












                  ls -lh *.log *.lst logfolder | grep year | grep month







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Jun 25 '14 at 15:48









                  schaiba

                  5,39912028




                  5,39912028






























                       

                      draft saved


                      draft discarded



















































                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f139168%2fusr-bin-stat-argument-list-too-long-error%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Accessing regular linux commands in Huawei's Dopra Linux

                      Can't connect RFCOMM socket: Host is down

                      Kernel panic - not syncing: Fatal Exception in Interrupt