Pass shell variable as a /pattern/ to awk












55















Having the following in one of my shell functions:



function _process () {
awk -v l="$line" '
BEGIN {p=0}
/'"$1"'/ {p=1}
END{ if(p) print l >> "outfile.txt" }
'
}


, so when called as _process $arg, $arg gets passed as $1, and used as a search pattern. It works this way, because shell expands $1 in place of awk pattern! Also l can be used inside awk program, being declared with -v l="$line". All fine.



Is it possible in same manner give pattern to search as a variable?



Following will not work,



awk -v l="$line" -v search="$pattern" '
BEGIN {p=0}
/search/ {p=1}
END{ if(p) print l >> "outfile.txt" }
'


,as awk will not interpret /search/ as a variable, but instead literally.










share|improve this question





























    55















    Having the following in one of my shell functions:



    function _process () {
    awk -v l="$line" '
    BEGIN {p=0}
    /'"$1"'/ {p=1}
    END{ if(p) print l >> "outfile.txt" }
    '
    }


    , so when called as _process $arg, $arg gets passed as $1, and used as a search pattern. It works this way, because shell expands $1 in place of awk pattern! Also l can be used inside awk program, being declared with -v l="$line". All fine.



    Is it possible in same manner give pattern to search as a variable?



    Following will not work,



    awk -v l="$line" -v search="$pattern" '
    BEGIN {p=0}
    /search/ {p=1}
    END{ if(p) print l >> "outfile.txt" }
    '


    ,as awk will not interpret /search/ as a variable, but instead literally.










    share|improve this question



























      55












      55








      55


      7






      Having the following in one of my shell functions:



      function _process () {
      awk -v l="$line" '
      BEGIN {p=0}
      /'"$1"'/ {p=1}
      END{ if(p) print l >> "outfile.txt" }
      '
      }


      , so when called as _process $arg, $arg gets passed as $1, and used as a search pattern. It works this way, because shell expands $1 in place of awk pattern! Also l can be used inside awk program, being declared with -v l="$line". All fine.



      Is it possible in same manner give pattern to search as a variable?



      Following will not work,



      awk -v l="$line" -v search="$pattern" '
      BEGIN {p=0}
      /search/ {p=1}
      END{ if(p) print l >> "outfile.txt" }
      '


      ,as awk will not interpret /search/ as a variable, but instead literally.










      share|improve this question
















      Having the following in one of my shell functions:



      function _process () {
      awk -v l="$line" '
      BEGIN {p=0}
      /'"$1"'/ {p=1}
      END{ if(p) print l >> "outfile.txt" }
      '
      }


      , so when called as _process $arg, $arg gets passed as $1, and used as a search pattern. It works this way, because shell expands $1 in place of awk pattern! Also l can be used inside awk program, being declared with -v l="$line". All fine.



      Is it possible in same manner give pattern to search as a variable?



      Following will not work,



      awk -v l="$line" -v search="$pattern" '
      BEGIN {p=0}
      /search/ {p=1}
      END{ if(p) print l >> "outfile.txt" }
      '


      ,as awk will not interpret /search/ as a variable, but instead literally.







      shell awk quoting variable






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 21 '14 at 23:04









      Gilles

      543k12811001618




      543k12811001618










      asked Mar 21 '14 at 15:03









      branquitobranquito

      4451616




      4451616






















          5 Answers
          5






          active

          oldest

          votes


















          43














          Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:



          function _process () {
          awk -v l="$line" -v pattern="$1" '
          $0 ~ pattern {p=1}
          END {if(p) print l >> "outfile.txt"}
          '
          }


          Although this would be more efficient (don't have to read the whole file)



          function _process () {
          grep -q "$1" && echo "$line"
          }


          Depending on the pattern, may want grep -Eq "$1"






          share|improve this answer
























          • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

            – branquito
            Mar 21 '14 at 15:30






          • 1





            I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

            – glenn jackman
            Mar 21 '14 at 15:35











          • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

            – branquito
            Mar 21 '14 at 15:42











          • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

            – branquito
            Mar 21 '14 at 15:53











          • Please show the contents of $line -- do it in your question for proper formatting.

            – glenn jackman
            Mar 21 '14 at 15:57



















          16














          awk  -v pattern="$1" '$0 ~ pattern'


          Has an issue in that awk expands the ANSI C escape sequences (like n for newline, f for form feed, \ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:



          PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'


          How bad it's going to be will depend on the awk implementation.



          $ nawk -v 'a=.' 'BEGIN {print a}'
          .
          $ mawk -v 'a=.' 'BEGIN {print a}'
          .
          $ gawk -v 'a=.' 'BEGIN {print a}'
          gawk: warning: escape sequence `.' treated as plain `.'
          .


          All awks work the same for valid escape sequences though:



          $ a='\-b' awk 'BEGIN {print ENVIRON["a"]}' | od -tc
          0000000 - b n
          0000006


          (content of $a passed as-is)



          $ awk -v a='\-b' 'BEGIN {print a}' | od -tc
          0000000 - b n
          0000004


          (\ changed to and b changed to a backspace character).






          share|improve this answer


























          • So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

            – branquito
            Mar 21 '14 at 16:24






          • 2





            for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

            – Stéphane Chazelas
            Mar 21 '14 at 16:30













          • it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

            – branquito
            Mar 21 '14 at 16:34








          • 1





            Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

            – Stéphane Chazelas
            Mar 21 '14 at 17:03






          • 1





            Because you need to export a shell variable for it to be passed in the environment to a command.

            – Stéphane Chazelas
            Mar 21 '14 at 17:20



















          5














          Try something like:



          awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'





          share|improve this answer
























          • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

            – branquito
            Mar 21 '14 at 15:22






          • 1





            The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

            – Hunter Eidson
            Mar 21 '14 at 15:24



















          0














          No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:



          awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> "outfile.txt" }"


          Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.






          share|improve this answer
























          • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

            – branquito
            Mar 21 '14 at 15:14






          • 2





            Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

            – Kilian Foth
            Mar 21 '14 at 15:26






          • 4





            But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

            – Stéphane Chazelas
            Mar 21 '14 at 16:17











          • is that downside of this approach only, having all wrapped in "" ?

            – branquito
            Mar 21 '14 at 16:27



















          -3














          You could use the eval function which resolves in this example the nets variable before the awk is run.



          nets="searchtext"
          eval "awk '/"${nets}"/'" file.txt





          share|improve this answer

























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "106"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f120788%2fpass-shell-variable-as-a-pattern-to-awk%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            5 Answers
            5






            active

            oldest

            votes








            5 Answers
            5






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            43














            Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:



            function _process () {
            awk -v l="$line" -v pattern="$1" '
            $0 ~ pattern {p=1}
            END {if(p) print l >> "outfile.txt"}
            '
            }


            Although this would be more efficient (don't have to read the whole file)



            function _process () {
            grep -q "$1" && echo "$line"
            }


            Depending on the pattern, may want grep -Eq "$1"






            share|improve this answer
























            • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

              – branquito
              Mar 21 '14 at 15:30






            • 1





              I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

              – glenn jackman
              Mar 21 '14 at 15:35











            • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

              – branquito
              Mar 21 '14 at 15:42











            • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

              – branquito
              Mar 21 '14 at 15:53











            • Please show the contents of $line -- do it in your question for proper formatting.

              – glenn jackman
              Mar 21 '14 at 15:57
















            43














            Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:



            function _process () {
            awk -v l="$line" -v pattern="$1" '
            $0 ~ pattern {p=1}
            END {if(p) print l >> "outfile.txt"}
            '
            }


            Although this would be more efficient (don't have to read the whole file)



            function _process () {
            grep -q "$1" && echo "$line"
            }


            Depending on the pattern, may want grep -Eq "$1"






            share|improve this answer
























            • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

              – branquito
              Mar 21 '14 at 15:30






            • 1





              I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

              – glenn jackman
              Mar 21 '14 at 15:35











            • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

              – branquito
              Mar 21 '14 at 15:42











            • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

              – branquito
              Mar 21 '14 at 15:53











            • Please show the contents of $line -- do it in your question for proper formatting.

              – glenn jackman
              Mar 21 '14 at 15:57














            43












            43








            43







            Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:



            function _process () {
            awk -v l="$line" -v pattern="$1" '
            $0 ~ pattern {p=1}
            END {if(p) print l >> "outfile.txt"}
            '
            }


            Although this would be more efficient (don't have to read the whole file)



            function _process () {
            grep -q "$1" && echo "$line"
            }


            Depending on the pattern, may want grep -Eq "$1"






            share|improve this answer













            Use awk's ~ operator, and you don't need to provide a literal regex on the right-hand side:



            function _process () {
            awk -v l="$line" -v pattern="$1" '
            $0 ~ pattern {p=1}
            END {if(p) print l >> "outfile.txt"}
            '
            }


            Although this would be more efficient (don't have to read the whole file)



            function _process () {
            grep -q "$1" && echo "$line"
            }


            Depending on the pattern, may want grep -Eq "$1"







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 21 '14 at 15:16









            glenn jackmanglenn jackman

            52.5k573113




            52.5k573113













            • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

              – branquito
              Mar 21 '14 at 15:30






            • 1





              I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

              – glenn jackman
              Mar 21 '14 at 15:35











            • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

              – branquito
              Mar 21 '14 at 15:42











            • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

              – branquito
              Mar 21 '14 at 15:53











            • Please show the contents of $line -- do it in your question for proper formatting.

              – glenn jackman
              Mar 21 '14 at 15:57



















            • This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

              – branquito
              Mar 21 '14 at 15:30






            • 1





              I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

              – glenn jackman
              Mar 21 '14 at 15:35











            • yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

              – branquito
              Mar 21 '14 at 15:42











            • maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

              – branquito
              Mar 21 '14 at 15:53











            • Please show the contents of $line -- do it in your question for proper formatting.

              – glenn jackman
              Mar 21 '14 at 15:57

















            This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

            – branquito
            Mar 21 '14 at 15:30





            This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.

            – branquito
            Mar 21 '14 at 15:30




            1




            1





            I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

            – glenn jackman
            Mar 21 '14 at 15:35





            I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in if (p) ...

            – glenn jackman
            Mar 21 '14 at 15:35













            yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

            – branquito
            Mar 21 '14 at 15:42





            yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using $0 ~ pattern, and it does not work, however with /'"$1"'/ it does work!? :O

            – branquito
            Mar 21 '14 at 15:42













            maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

            – branquito
            Mar 21 '14 at 15:53





            maybe it has something to do with the way $line is retrieved, pattern search is done on the output of whois $line, $line coming from file in a WHILE DO block.

            – branquito
            Mar 21 '14 at 15:53













            Please show the contents of $line -- do it in your question for proper formatting.

            – glenn jackman
            Mar 21 '14 at 15:57





            Please show the contents of $line -- do it in your question for proper formatting.

            – glenn jackman
            Mar 21 '14 at 15:57













            16














            awk  -v pattern="$1" '$0 ~ pattern'


            Has an issue in that awk expands the ANSI C escape sequences (like n for newline, f for form feed, \ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:



            PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'


            How bad it's going to be will depend on the awk implementation.



            $ nawk -v 'a=.' 'BEGIN {print a}'
            .
            $ mawk -v 'a=.' 'BEGIN {print a}'
            .
            $ gawk -v 'a=.' 'BEGIN {print a}'
            gawk: warning: escape sequence `.' treated as plain `.'
            .


            All awks work the same for valid escape sequences though:



            $ a='\-b' awk 'BEGIN {print ENVIRON["a"]}' | od -tc
            0000000 - b n
            0000006


            (content of $a passed as-is)



            $ awk -v a='\-b' 'BEGIN {print a}' | od -tc
            0000000 - b n
            0000004


            (\ changed to and b changed to a backspace character).






            share|improve this answer


























            • So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

              – branquito
              Mar 21 '14 at 16:24






            • 2





              for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

              – Stéphane Chazelas
              Mar 21 '14 at 16:30













            • it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

              – branquito
              Mar 21 '14 at 16:34








            • 1





              Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

              – Stéphane Chazelas
              Mar 21 '14 at 17:03






            • 1





              Because you need to export a shell variable for it to be passed in the environment to a command.

              – Stéphane Chazelas
              Mar 21 '14 at 17:20
















            16














            awk  -v pattern="$1" '$0 ~ pattern'


            Has an issue in that awk expands the ANSI C escape sequences (like n for newline, f for form feed, \ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:



            PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'


            How bad it's going to be will depend on the awk implementation.



            $ nawk -v 'a=.' 'BEGIN {print a}'
            .
            $ mawk -v 'a=.' 'BEGIN {print a}'
            .
            $ gawk -v 'a=.' 'BEGIN {print a}'
            gawk: warning: escape sequence `.' treated as plain `.'
            .


            All awks work the same for valid escape sequences though:



            $ a='\-b' awk 'BEGIN {print ENVIRON["a"]}' | od -tc
            0000000 - b n
            0000006


            (content of $a passed as-is)



            $ awk -v a='\-b' 'BEGIN {print a}' | od -tc
            0000000 - b n
            0000004


            (\ changed to and b changed to a backspace character).






            share|improve this answer


























            • So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

              – branquito
              Mar 21 '14 at 16:24






            • 2





              for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

              – Stéphane Chazelas
              Mar 21 '14 at 16:30













            • it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

              – branquito
              Mar 21 '14 at 16:34








            • 1





              Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

              – Stéphane Chazelas
              Mar 21 '14 at 17:03






            • 1





              Because you need to export a shell variable for it to be passed in the environment to a command.

              – Stéphane Chazelas
              Mar 21 '14 at 17:20














            16












            16








            16







            awk  -v pattern="$1" '$0 ~ pattern'


            Has an issue in that awk expands the ANSI C escape sequences (like n for newline, f for form feed, \ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:



            PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'


            How bad it's going to be will depend on the awk implementation.



            $ nawk -v 'a=.' 'BEGIN {print a}'
            .
            $ mawk -v 'a=.' 'BEGIN {print a}'
            .
            $ gawk -v 'a=.' 'BEGIN {print a}'
            gawk: warning: escape sequence `.' treated as plain `.'
            .


            All awks work the same for valid escape sequences though:



            $ a='\-b' awk 'BEGIN {print ENVIRON["a"]}' | od -tc
            0000000 - b n
            0000006


            (content of $a passed as-is)



            $ awk -v a='\-b' 'BEGIN {print a}' | od -tc
            0000000 - b n
            0000004


            (\ changed to and b changed to a backspace character).






            share|improve this answer















            awk  -v pattern="$1" '$0 ~ pattern'


            Has an issue in that awk expands the ANSI C escape sequences (like n for newline, f for form feed, \ for backslash and so on) in $1. So it becomes an issue if $1 contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:



            PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'


            How bad it's going to be will depend on the awk implementation.



            $ nawk -v 'a=.' 'BEGIN {print a}'
            .
            $ mawk -v 'a=.' 'BEGIN {print a}'
            .
            $ gawk -v 'a=.' 'BEGIN {print a}'
            gawk: warning: escape sequence `.' treated as plain `.'
            .


            All awks work the same for valid escape sequences though:



            $ a='\-b' awk 'BEGIN {print ENVIRON["a"]}' | od -tc
            0000000 - b n
            0000006


            (content of $a passed as-is)



            $ awk -v a='\-b' 'BEGIN {print a}' | od -tc
            0000000 - b n
            0000004


            (\ changed to and b changed to a backspace character).







            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited 7 hours ago

























            answered Mar 21 '14 at 16:16









            Stéphane ChazelasStéphane Chazelas

            311k57586945




            311k57586945













            • So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

              – branquito
              Mar 21 '14 at 16:24






            • 2





              for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

              – Stéphane Chazelas
              Mar 21 '14 at 16:30













            • it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

              – branquito
              Mar 21 '14 at 16:34








            • 1





              Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

              – Stéphane Chazelas
              Mar 21 '14 at 17:03






            • 1





              Because you need to export a shell variable for it to be passed in the environment to a command.

              – Stéphane Chazelas
              Mar 21 '14 at 17:20



















            • So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

              – branquito
              Mar 21 '14 at 16:24






            • 2





              for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

              – Stéphane Chazelas
              Mar 21 '14 at 16:30













            • it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

              – branquito
              Mar 21 '14 at 16:34








            • 1





              Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

              – Stéphane Chazelas
              Mar 21 '14 at 17:03






            • 1





              Because you need to export a shell variable for it to be passed in the environment to a command.

              – Stéphane Chazelas
              Mar 21 '14 at 17:20

















            So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

            – branquito
            Mar 21 '14 at 16:24





            So you are saying that if pattern was for example d{3} to find three digits, that wouldn't work as expected, if I understood you well?

            – branquito
            Mar 21 '14 at 16:24




            2




            2





            for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

            – Stéphane Chazelas
            Mar 21 '14 at 16:30







            for d which is not a valid C escape sequence, that depends on your awk implementation (run awk -v 'a=d{3}' 'BEGIN{print a}' to check). But for ` or b, yes definitely. (BTW, I don't know of any awk implementations that understands d` as meaning a digit).

            – Stéphane Chazelas
            Mar 21 '14 at 16:30















            it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

            – branquito
            Mar 21 '14 at 16:34







            it says: awk warning - escape sequence d' treated as plain d' d{3}, so I guess I would have a problem in this case?

            – branquito
            Mar 21 '14 at 16:34






            1




            1





            Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

            – Stéphane Chazelas
            Mar 21 '14 at 17:03





            Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match ENVIRON["PATTERN"] for the PATTERN environment variable. If you want to use a shell variable, you need to export it first (export variable) or use the ENV=VALUE awk '...ENVIRON["ENV"]' env-var passing syntax as in my answer.

            – Stéphane Chazelas
            Mar 21 '14 at 17:03




            1




            1





            Because you need to export a shell variable for it to be passed in the environment to a command.

            – Stéphane Chazelas
            Mar 21 '14 at 17:20





            Because you need to export a shell variable for it to be passed in the environment to a command.

            – Stéphane Chazelas
            Mar 21 '14 at 17:20











            5














            Try something like:



            awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'





            share|improve this answer
























            • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

              – branquito
              Mar 21 '14 at 15:22






            • 1





              The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

              – Hunter Eidson
              Mar 21 '14 at 15:24
















            5














            Try something like:



            awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'





            share|improve this answer
























            • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

              – branquito
              Mar 21 '14 at 15:22






            • 1





              The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

              – Hunter Eidson
              Mar 21 '14 at 15:24














            5












            5








            5







            Try something like:



            awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'





            share|improve this answer













            Try something like:



            awk -v l="$line" -v search="$pattern" 'BEGIN {p=0}; { if ( match( $0, search )) {p=1}}; END{ if(p) print l >> "outfile.txt" }'






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 21 '14 at 15:15









            Hunter EidsonHunter Eidson

            17116




            17116













            • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

              – branquito
              Mar 21 '14 at 15:22






            • 1





              The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

              – Hunter Eidson
              Mar 21 '14 at 15:24



















            • If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

              – branquito
              Mar 21 '14 at 15:22






            • 1





              The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

              – Hunter Eidson
              Mar 21 '14 at 15:24

















            If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

            – branquito
            Mar 21 '14 at 15:22





            If this behaves same as /regex/ in terms of finding pattern, this could be a nice solution. I will try.

            – branquito
            Mar 21 '14 at 15:22




            1




            1





            The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

            – Hunter Eidson
            Mar 21 '14 at 15:24





            The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)

            – Hunter Eidson
            Mar 21 '14 at 15:24











            0














            No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:



            awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> "outfile.txt" }"


            Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.






            share|improve this answer
























            • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

              – branquito
              Mar 21 '14 at 15:14






            • 2





              Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

              – Kilian Foth
              Mar 21 '14 at 15:26






            • 4





              But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

              – Stéphane Chazelas
              Mar 21 '14 at 16:17











            • is that downside of this approach only, having all wrapped in "" ?

              – branquito
              Mar 21 '14 at 16:27
















            0














            No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:



            awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> "outfile.txt" }"


            Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.






            share|improve this answer
























            • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

              – branquito
              Mar 21 '14 at 15:14






            • 2





              Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

              – Kilian Foth
              Mar 21 '14 at 15:26






            • 4





              But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

              – Stéphane Chazelas
              Mar 21 '14 at 16:17











            • is that downside of this approach only, having all wrapped in "" ?

              – branquito
              Mar 21 '14 at 16:27














            0












            0








            0







            No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:



            awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> "outfile.txt" }"


            Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.






            share|improve this answer













            No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:



            awk -v l="$line" "BEGIN {p=0}; /$pattern/ {p=1}; END{ if(p) print l >> "outfile.txt" }"


            Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 21 '14 at 15:11









            Kilian FothKilian Foth

            680315




            680315













            • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

              – branquito
              Mar 21 '14 at 15:14






            • 2





              Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

              – Kilian Foth
              Mar 21 '14 at 15:26






            • 4





              But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

              – Stéphane Chazelas
              Mar 21 '14 at 16:17











            • is that downside of this approach only, having all wrapped in "" ?

              – branquito
              Mar 21 '14 at 16:27



















            • Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

              – branquito
              Mar 21 '14 at 15:14






            • 2





              Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

              – Kilian Foth
              Mar 21 '14 at 15:26






            • 4





              But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

              – Stéphane Chazelas
              Mar 21 '14 at 16:17











            • is that downside of this approach only, having all wrapped in "" ?

              – branquito
              Mar 21 '14 at 16:27

















            Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

            – branquito
            Mar 21 '14 at 15:14





            Is this way safe if $pattern contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.

            – branquito
            Mar 21 '14 at 15:14




            2




            2





            Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

            – Kilian Foth
            Mar 21 '14 at 15:26





            Your original example ends the single-quoted string at the second ', then protects the $1 via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1 via the outer single quotes - awk never sees the double quotes that you put around it.

            – Kilian Foth
            Mar 21 '14 at 15:26




            4




            4





            But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

            – Stéphane Chazelas
            Mar 21 '14 at 16:17





            But if $pattern contains ^/ {system("rm -rf /")};, then you're in big trouble.

            – Stéphane Chazelas
            Mar 21 '14 at 16:17













            is that downside of this approach only, having all wrapped in "" ?

            – branquito
            Mar 21 '14 at 16:27





            is that downside of this approach only, having all wrapped in "" ?

            – branquito
            Mar 21 '14 at 16:27











            -3














            You could use the eval function which resolves in this example the nets variable before the awk is run.



            nets="searchtext"
            eval "awk '/"${nets}"/'" file.txt





            share|improve this answer






























              -3














              You could use the eval function which resolves in this example the nets variable before the awk is run.



              nets="searchtext"
              eval "awk '/"${nets}"/'" file.txt





              share|improve this answer




























                -3












                -3








                -3







                You could use the eval function which resolves in this example the nets variable before the awk is run.



                nets="searchtext"
                eval "awk '/"${nets}"/'" file.txt





                share|improve this answer















                You could use the eval function which resolves in this example the nets variable before the awk is run.



                nets="searchtext"
                eval "awk '/"${nets}"/'" file.txt






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Feb 21 '17 at 11:51









                Noxy

                32




                32










                answered Feb 21 '17 at 11:19









                NoxyNoxy

                1




                1






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Unix & Linux Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f120788%2fpass-shell-variable-as-a-pattern-to-awk%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Entries order in /etc/network/interfaces

                    新発田市

                    Grub takes very long (several minutes) to open Menu (in Multi-Boot-System)