looping through JSON array in shell script












4














Below is the curl command output (file information about branch), need script or command to print file name, filetype and size.



I have tried with jq but was able fetch single value ( jq '.values.size')



{
"path": {
"components": ,
"name": "",
"toString": ""
},
"revision": "master",
"children": {
"size": 5,
"limit": 500,
"isLastPage": true,
"values": [
{
"path": {
"components": [
".gitignore"
],
"parent": "",
"name": ".gitignore",
"extension": "gitignore",
"toString": ".gitignore"
},
"contentId": "c9e472ef4e603480cdd85012b01bd5f4eddc86c6",
"type": "FILE",
"size": 224
},
{
"path": {
"components": [
"Jenkinsfile"
],
"parent": "",
"name": "Jenkinsfile",
"toString": "Jenkinsfile"
},
"contentId": "e878a88eed6b19b2eb0852c39bfd290151b865a4",
"type": "FILE",
"size": 1396
},
{
"path": {
"components": [
"README.md"
],
"parent": "",
"name": "README.md",
"extension": "md",
"toString": "README.md"
},
"contentId": "05782ad495bfe11e00a77c30ea3ce17c7fa39606",
"type": "FILE",
"size": 237
},
{
"path": {
"components": [
"pom.xml"
],
"parent": "",
"name": "pom.xml",
"extension": "xml",
"toString": "pom.xml"
},
"contentId": "9cd4887f8fc8c2ecc69ca08508b0f5d7b019dafd",
"type": "FILE",
"size": 2548
},
{
"path": {
"components": [
"src"
],
"parent": "",
"name": "src",
"toString": "src"
},
"node": "395c71003030308d1e4148b7786e9f331c269bdf",
"type": "DIRECTORY"
}
],
"start": 0
}
}


expected output should be something like below



.gitignore    FILE     224

Jenkinsfile FILE 1396









share|improve this question





























    4














    Below is the curl command output (file information about branch), need script or command to print file name, filetype and size.



    I have tried with jq but was able fetch single value ( jq '.values.size')



    {
    "path": {
    "components": ,
    "name": "",
    "toString": ""
    },
    "revision": "master",
    "children": {
    "size": 5,
    "limit": 500,
    "isLastPage": true,
    "values": [
    {
    "path": {
    "components": [
    ".gitignore"
    ],
    "parent": "",
    "name": ".gitignore",
    "extension": "gitignore",
    "toString": ".gitignore"
    },
    "contentId": "c9e472ef4e603480cdd85012b01bd5f4eddc86c6",
    "type": "FILE",
    "size": 224
    },
    {
    "path": {
    "components": [
    "Jenkinsfile"
    ],
    "parent": "",
    "name": "Jenkinsfile",
    "toString": "Jenkinsfile"
    },
    "contentId": "e878a88eed6b19b2eb0852c39bfd290151b865a4",
    "type": "FILE",
    "size": 1396
    },
    {
    "path": {
    "components": [
    "README.md"
    ],
    "parent": "",
    "name": "README.md",
    "extension": "md",
    "toString": "README.md"
    },
    "contentId": "05782ad495bfe11e00a77c30ea3ce17c7fa39606",
    "type": "FILE",
    "size": 237
    },
    {
    "path": {
    "components": [
    "pom.xml"
    ],
    "parent": "",
    "name": "pom.xml",
    "extension": "xml",
    "toString": "pom.xml"
    },
    "contentId": "9cd4887f8fc8c2ecc69ca08508b0f5d7b019dafd",
    "type": "FILE",
    "size": 2548
    },
    {
    "path": {
    "components": [
    "src"
    ],
    "parent": "",
    "name": "src",
    "toString": "src"
    },
    "node": "395c71003030308d1e4148b7786e9f331c269bdf",
    "type": "DIRECTORY"
    }
    ],
    "start": 0
    }
    }


    expected output should be something like below



    .gitignore    FILE     224

    Jenkinsfile FILE 1396









    share|improve this question



























      4












      4








      4


      1





      Below is the curl command output (file information about branch), need script or command to print file name, filetype and size.



      I have tried with jq but was able fetch single value ( jq '.values.size')



      {
      "path": {
      "components": ,
      "name": "",
      "toString": ""
      },
      "revision": "master",
      "children": {
      "size": 5,
      "limit": 500,
      "isLastPage": true,
      "values": [
      {
      "path": {
      "components": [
      ".gitignore"
      ],
      "parent": "",
      "name": ".gitignore",
      "extension": "gitignore",
      "toString": ".gitignore"
      },
      "contentId": "c9e472ef4e603480cdd85012b01bd5f4eddc86c6",
      "type": "FILE",
      "size": 224
      },
      {
      "path": {
      "components": [
      "Jenkinsfile"
      ],
      "parent": "",
      "name": "Jenkinsfile",
      "toString": "Jenkinsfile"
      },
      "contentId": "e878a88eed6b19b2eb0852c39bfd290151b865a4",
      "type": "FILE",
      "size": 1396
      },
      {
      "path": {
      "components": [
      "README.md"
      ],
      "parent": "",
      "name": "README.md",
      "extension": "md",
      "toString": "README.md"
      },
      "contentId": "05782ad495bfe11e00a77c30ea3ce17c7fa39606",
      "type": "FILE",
      "size": 237
      },
      {
      "path": {
      "components": [
      "pom.xml"
      ],
      "parent": "",
      "name": "pom.xml",
      "extension": "xml",
      "toString": "pom.xml"
      },
      "contentId": "9cd4887f8fc8c2ecc69ca08508b0f5d7b019dafd",
      "type": "FILE",
      "size": 2548
      },
      {
      "path": {
      "components": [
      "src"
      ],
      "parent": "",
      "name": "src",
      "toString": "src"
      },
      "node": "395c71003030308d1e4148b7786e9f331c269bdf",
      "type": "DIRECTORY"
      }
      ],
      "start": 0
      }
      }


      expected output should be something like below



      .gitignore    FILE     224

      Jenkinsfile FILE 1396









      share|improve this question















      Below is the curl command output (file information about branch), need script or command to print file name, filetype and size.



      I have tried with jq but was able fetch single value ( jq '.values.size')



      {
      "path": {
      "components": ,
      "name": "",
      "toString": ""
      },
      "revision": "master",
      "children": {
      "size": 5,
      "limit": 500,
      "isLastPage": true,
      "values": [
      {
      "path": {
      "components": [
      ".gitignore"
      ],
      "parent": "",
      "name": ".gitignore",
      "extension": "gitignore",
      "toString": ".gitignore"
      },
      "contentId": "c9e472ef4e603480cdd85012b01bd5f4eddc86c6",
      "type": "FILE",
      "size": 224
      },
      {
      "path": {
      "components": [
      "Jenkinsfile"
      ],
      "parent": "",
      "name": "Jenkinsfile",
      "toString": "Jenkinsfile"
      },
      "contentId": "e878a88eed6b19b2eb0852c39bfd290151b865a4",
      "type": "FILE",
      "size": 1396
      },
      {
      "path": {
      "components": [
      "README.md"
      ],
      "parent": "",
      "name": "README.md",
      "extension": "md",
      "toString": "README.md"
      },
      "contentId": "05782ad495bfe11e00a77c30ea3ce17c7fa39606",
      "type": "FILE",
      "size": 237
      },
      {
      "path": {
      "components": [
      "pom.xml"
      ],
      "parent": "",
      "name": "pom.xml",
      "extension": "xml",
      "toString": "pom.xml"
      },
      "contentId": "9cd4887f8fc8c2ecc69ca08508b0f5d7b019dafd",
      "type": "FILE",
      "size": 2548
      },
      {
      "path": {
      "components": [
      "src"
      ],
      "parent": "",
      "name": "src",
      "toString": "src"
      },
      "node": "395c71003030308d1e4148b7786e9f331c269bdf",
      "type": "DIRECTORY"
      }
      ],
      "start": 0
      }
      }


      expected output should be something like below



      .gitignore    FILE     224

      Jenkinsfile FILE 1396






      shell scripting json jq






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited 1 hour ago









      SouravGhosh

      457311




      457311










      asked Oct 23 at 9:08









      Sugatur Deekshith S N

      245




      245






















          4 Answers
          4






          active

          oldest

          votes


















          3














          For the use case provided in the Question, @JigglyNaga's answer is probably better than this, but for some more complicated task, you could also loop through the list items using keys:



          from file:



          for k in $(jq '.children.values | keys | .' file); do
          ...
          done


          or from string:



          for k in $(jq '.children.values | keys | .' <<< "$MYJSONSTRING"); do
          ...
          done


          So e.g. you might use:



          for k in $(jq '.children.values | keys | .' file); do
          value=$(jq -r ".children.values[$k]" file);
          name=$(jq -r '.path.name' <<< "$value");
          type=$(jq -r '.type' <<< "$value");
          size=$(jq -r '.size' <<< "$value");
          printf '%st%st%sn' "$name" "$type" "$size";
          done | column -t -s$'t'


          if you have no newlines for the values, you can make it with a single jq call inside the loop which makes it much faster:



          for k in $(jq '.children.values | keys | .' file); do
          IFS=$'n' read -r -d '' name type size
          <<< "$(jq -r ".children.values[$k] | .path.name,.type,.size" file)"
          printf '%st%st%sn' "$name" "$type" "$size";
          done | column -t -s$'t'





          share|improve this answer























          • is the file is a keyword ? or the json array should be present in file?
            – Sugatur Deekshith S N
            Oct 23 at 9:59










          • updated the Answer to clarify this.
            – RoVo
            Oct 23 at 11:03



















          7














          Extracting the members



          jq -c '.children.values|[.path.components[0],.type,.size]'




          • .children.values outputs every member of the array .values.


          • | pipes the previous result through the next filter, rather like a shell pipe


          • [...,...,...] makes all the terms inside appear in a single array

          • The -c option produces "compact" format ie. one object per line


          Result:



          [".gitignore","FILE",224]
          ["Jenkinsfile","FILE",1396]
          ["README.md","FILE",237]
          ...


          Formatting the result



          If you want to output a neatly-aligned table, that's a task better handled by other tools, such as column or paste.



          jq -c '.children.values|[.path.components[0],.type,.size]' | column -t -s',"'




          • -t tells column to guess the number of columns based on the input


          • -s... specifies the delimiter character(s)


          Result:



          .gitignore   FILE       224
          Jenkinsfile FILE 1396
          README.md FILE 237


          This relies on the characters [, ], , and " not appearing in your filenames, which is not a safe assumption.



          paste can also arrange multiple inputs side-by-side. For this, we can remove the JSON structures altogether, and output raw lines (hat-tip to @muru):



          jq -r '.children.values|.path.components[0],.type,.size' | paste - - -


          paste - - - means 3 columns, all read from the the same source. This time, the only assumption is that the filenames don't contain newlines.






          share|improve this answer























          • add " to -s and you have the exact output that OP wants
            – RoVo
            Oct 23 at 9:36






          • 1




            Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
            – muru
            Oct 23 at 9:37












          • @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
            – Sugatur Deekshith S N
            Oct 23 at 9:43












          • then you can use my answer ;-)
            – RoVo
            Oct 23 at 9:44






          • 1




            @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
            – JigglyNaga
            Oct 23 at 9:58



















          2














          jq can render its output into a variety of formats: see https://stedolan.github.io/jq/manual/#Formatstringsandescaping



          For tab-separated output:



          $ jq -r '.children.values | [.path.name, .type, .size] | @tsv' file.json
          .gitignore FILE 224
          Jenkinsfile FILE 1396
          README.md FILE 237
          pom.xml FILE 2548
          src DIRECTORY





          share|improve this answer





























            1














            Solution with ramda-cli:



            % curl ... | ramda -o tsv '.children.values' 'map flat' 'map props ["path.name", "type", "size"]'
            .gitignore FILE 224
            Jenkinsfile FILE 1396
            README.md FILE 237
            pom.xml FILE 2548
            src DIRECTORY


            First we traverse into the list of values, then map over the list with flat to convert each entry which is a deep object structure into a shallow one, with keys separated by dots.



            Then, we can map over the list again, and pick the wanted properties based on their paths, represented with strings.



            Finally, -o tsv takes care of converting the resulting list of lists into tsv format.



            To debug or further understand what is happening, you may check what each argument does by removing them one by one from the end of the command and observing the difference in output at each step. They are simply operations (or functions) applied on the data one at a time from left to right.






            share|improve this answer























              Your Answer








              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "106"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: false,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: null,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f477210%2flooping-through-json-array-in-shell-script%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              4 Answers
              4






              active

              oldest

              votes








              4 Answers
              4






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              3














              For the use case provided in the Question, @JigglyNaga's answer is probably better than this, but for some more complicated task, you could also loop through the list items using keys:



              from file:



              for k in $(jq '.children.values | keys | .' file); do
              ...
              done


              or from string:



              for k in $(jq '.children.values | keys | .' <<< "$MYJSONSTRING"); do
              ...
              done


              So e.g. you might use:



              for k in $(jq '.children.values | keys | .' file); do
              value=$(jq -r ".children.values[$k]" file);
              name=$(jq -r '.path.name' <<< "$value");
              type=$(jq -r '.type' <<< "$value");
              size=$(jq -r '.size' <<< "$value");
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'


              if you have no newlines for the values, you can make it with a single jq call inside the loop which makes it much faster:



              for k in $(jq '.children.values | keys | .' file); do
              IFS=$'n' read -r -d '' name type size
              <<< "$(jq -r ".children.values[$k] | .path.name,.type,.size" file)"
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'





              share|improve this answer























              • is the file is a keyword ? or the json array should be present in file?
                – Sugatur Deekshith S N
                Oct 23 at 9:59










              • updated the Answer to clarify this.
                – RoVo
                Oct 23 at 11:03
















              3














              For the use case provided in the Question, @JigglyNaga's answer is probably better than this, but for some more complicated task, you could also loop through the list items using keys:



              from file:



              for k in $(jq '.children.values | keys | .' file); do
              ...
              done


              or from string:



              for k in $(jq '.children.values | keys | .' <<< "$MYJSONSTRING"); do
              ...
              done


              So e.g. you might use:



              for k in $(jq '.children.values | keys | .' file); do
              value=$(jq -r ".children.values[$k]" file);
              name=$(jq -r '.path.name' <<< "$value");
              type=$(jq -r '.type' <<< "$value");
              size=$(jq -r '.size' <<< "$value");
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'


              if you have no newlines for the values, you can make it with a single jq call inside the loop which makes it much faster:



              for k in $(jq '.children.values | keys | .' file); do
              IFS=$'n' read -r -d '' name type size
              <<< "$(jq -r ".children.values[$k] | .path.name,.type,.size" file)"
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'





              share|improve this answer























              • is the file is a keyword ? or the json array should be present in file?
                – Sugatur Deekshith S N
                Oct 23 at 9:59










              • updated the Answer to clarify this.
                – RoVo
                Oct 23 at 11:03














              3












              3








              3






              For the use case provided in the Question, @JigglyNaga's answer is probably better than this, but for some more complicated task, you could also loop through the list items using keys:



              from file:



              for k in $(jq '.children.values | keys | .' file); do
              ...
              done


              or from string:



              for k in $(jq '.children.values | keys | .' <<< "$MYJSONSTRING"); do
              ...
              done


              So e.g. you might use:



              for k in $(jq '.children.values | keys | .' file); do
              value=$(jq -r ".children.values[$k]" file);
              name=$(jq -r '.path.name' <<< "$value");
              type=$(jq -r '.type' <<< "$value");
              size=$(jq -r '.size' <<< "$value");
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'


              if you have no newlines for the values, you can make it with a single jq call inside the loop which makes it much faster:



              for k in $(jq '.children.values | keys | .' file); do
              IFS=$'n' read -r -d '' name type size
              <<< "$(jq -r ".children.values[$k] | .path.name,.type,.size" file)"
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'





              share|improve this answer














              For the use case provided in the Question, @JigglyNaga's answer is probably better than this, but for some more complicated task, you could also loop through the list items using keys:



              from file:



              for k in $(jq '.children.values | keys | .' file); do
              ...
              done


              or from string:



              for k in $(jq '.children.values | keys | .' <<< "$MYJSONSTRING"); do
              ...
              done


              So e.g. you might use:



              for k in $(jq '.children.values | keys | .' file); do
              value=$(jq -r ".children.values[$k]" file);
              name=$(jq -r '.path.name' <<< "$value");
              type=$(jq -r '.type' <<< "$value");
              size=$(jq -r '.size' <<< "$value");
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'


              if you have no newlines for the values, you can make it with a single jq call inside the loop which makes it much faster:



              for k in $(jq '.children.values | keys | .' file); do
              IFS=$'n' read -r -d '' name type size
              <<< "$(jq -r ".children.values[$k] | .path.name,.type,.size" file)"
              printf '%st%st%sn' "$name" "$type" "$size";
              done | column -t -s$'t'






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Oct 23 at 11:33

























              answered Oct 23 at 9:39









              RoVo

              2,558215




              2,558215












              • is the file is a keyword ? or the json array should be present in file?
                – Sugatur Deekshith S N
                Oct 23 at 9:59










              • updated the Answer to clarify this.
                – RoVo
                Oct 23 at 11:03


















              • is the file is a keyword ? or the json array should be present in file?
                – Sugatur Deekshith S N
                Oct 23 at 9:59










              • updated the Answer to clarify this.
                – RoVo
                Oct 23 at 11:03
















              is the file is a keyword ? or the json array should be present in file?
              – Sugatur Deekshith S N
              Oct 23 at 9:59




              is the file is a keyword ? or the json array should be present in file?
              – Sugatur Deekshith S N
              Oct 23 at 9:59












              updated the Answer to clarify this.
              – RoVo
              Oct 23 at 11:03




              updated the Answer to clarify this.
              – RoVo
              Oct 23 at 11:03













              7














              Extracting the members



              jq -c '.children.values|[.path.components[0],.type,.size]'




              • .children.values outputs every member of the array .values.


              • | pipes the previous result through the next filter, rather like a shell pipe


              • [...,...,...] makes all the terms inside appear in a single array

              • The -c option produces "compact" format ie. one object per line


              Result:



              [".gitignore","FILE",224]
              ["Jenkinsfile","FILE",1396]
              ["README.md","FILE",237]
              ...


              Formatting the result



              If you want to output a neatly-aligned table, that's a task better handled by other tools, such as column or paste.



              jq -c '.children.values|[.path.components[0],.type,.size]' | column -t -s',"'




              • -t tells column to guess the number of columns based on the input


              • -s... specifies the delimiter character(s)


              Result:



              .gitignore   FILE       224
              Jenkinsfile FILE 1396
              README.md FILE 237


              This relies on the characters [, ], , and " not appearing in your filenames, which is not a safe assumption.



              paste can also arrange multiple inputs side-by-side. For this, we can remove the JSON structures altogether, and output raw lines (hat-tip to @muru):



              jq -r '.children.values|.path.components[0],.type,.size' | paste - - -


              paste - - - means 3 columns, all read from the the same source. This time, the only assumption is that the filenames don't contain newlines.






              share|improve this answer























              • add " to -s and you have the exact output that OP wants
                – RoVo
                Oct 23 at 9:36






              • 1




                Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
                – muru
                Oct 23 at 9:37












              • @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
                – Sugatur Deekshith S N
                Oct 23 at 9:43












              • then you can use my answer ;-)
                – RoVo
                Oct 23 at 9:44






              • 1




                @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
                – JigglyNaga
                Oct 23 at 9:58
















              7














              Extracting the members



              jq -c '.children.values|[.path.components[0],.type,.size]'




              • .children.values outputs every member of the array .values.


              • | pipes the previous result through the next filter, rather like a shell pipe


              • [...,...,...] makes all the terms inside appear in a single array

              • The -c option produces "compact" format ie. one object per line


              Result:



              [".gitignore","FILE",224]
              ["Jenkinsfile","FILE",1396]
              ["README.md","FILE",237]
              ...


              Formatting the result



              If you want to output a neatly-aligned table, that's a task better handled by other tools, such as column or paste.



              jq -c '.children.values|[.path.components[0],.type,.size]' | column -t -s',"'




              • -t tells column to guess the number of columns based on the input


              • -s... specifies the delimiter character(s)


              Result:



              .gitignore   FILE       224
              Jenkinsfile FILE 1396
              README.md FILE 237


              This relies on the characters [, ], , and " not appearing in your filenames, which is not a safe assumption.



              paste can also arrange multiple inputs side-by-side. For this, we can remove the JSON structures altogether, and output raw lines (hat-tip to @muru):



              jq -r '.children.values|.path.components[0],.type,.size' | paste - - -


              paste - - - means 3 columns, all read from the the same source. This time, the only assumption is that the filenames don't contain newlines.






              share|improve this answer























              • add " to -s and you have the exact output that OP wants
                – RoVo
                Oct 23 at 9:36






              • 1




                Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
                – muru
                Oct 23 at 9:37












              • @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
                – Sugatur Deekshith S N
                Oct 23 at 9:43












              • then you can use my answer ;-)
                – RoVo
                Oct 23 at 9:44






              • 1




                @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
                – JigglyNaga
                Oct 23 at 9:58














              7












              7








              7






              Extracting the members



              jq -c '.children.values|[.path.components[0],.type,.size]'




              • .children.values outputs every member of the array .values.


              • | pipes the previous result through the next filter, rather like a shell pipe


              • [...,...,...] makes all the terms inside appear in a single array

              • The -c option produces "compact" format ie. one object per line


              Result:



              [".gitignore","FILE",224]
              ["Jenkinsfile","FILE",1396]
              ["README.md","FILE",237]
              ...


              Formatting the result



              If you want to output a neatly-aligned table, that's a task better handled by other tools, such as column or paste.



              jq -c '.children.values|[.path.components[0],.type,.size]' | column -t -s',"'




              • -t tells column to guess the number of columns based on the input


              • -s... specifies the delimiter character(s)


              Result:



              .gitignore   FILE       224
              Jenkinsfile FILE 1396
              README.md FILE 237


              This relies on the characters [, ], , and " not appearing in your filenames, which is not a safe assumption.



              paste can also arrange multiple inputs side-by-side. For this, we can remove the JSON structures altogether, and output raw lines (hat-tip to @muru):



              jq -r '.children.values|.path.components[0],.type,.size' | paste - - -


              paste - - - means 3 columns, all read from the the same source. This time, the only assumption is that the filenames don't contain newlines.






              share|improve this answer














              Extracting the members



              jq -c '.children.values|[.path.components[0],.type,.size]'




              • .children.values outputs every member of the array .values.


              • | pipes the previous result through the next filter, rather like a shell pipe


              • [...,...,...] makes all the terms inside appear in a single array

              • The -c option produces "compact" format ie. one object per line


              Result:



              [".gitignore","FILE",224]
              ["Jenkinsfile","FILE",1396]
              ["README.md","FILE",237]
              ...


              Formatting the result



              If you want to output a neatly-aligned table, that's a task better handled by other tools, such as column or paste.



              jq -c '.children.values|[.path.components[0],.type,.size]' | column -t -s',"'




              • -t tells column to guess the number of columns based on the input


              • -s... specifies the delimiter character(s)


              Result:



              .gitignore   FILE       224
              Jenkinsfile FILE 1396
              README.md FILE 237


              This relies on the characters [, ], , and " not appearing in your filenames, which is not a safe assumption.



              paste can also arrange multiple inputs side-by-side. For this, we can remove the JSON structures altogether, and output raw lines (hat-tip to @muru):



              jq -r '.children.values|.path.components[0],.type,.size' | paste - - -


              paste - - - means 3 columns, all read from the the same source. This time, the only assumption is that the filenames don't contain newlines.







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Oct 23 at 9:51

























              answered Oct 23 at 9:25









              JigglyNaga

              3,698930




              3,698930












              • add " to -s and you have the exact output that OP wants
                – RoVo
                Oct 23 at 9:36






              • 1




                Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
                – muru
                Oct 23 at 9:37












              • @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
                – Sugatur Deekshith S N
                Oct 23 at 9:43












              • then you can use my answer ;-)
                – RoVo
                Oct 23 at 9:44






              • 1




                @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
                – JigglyNaga
                Oct 23 at 9:58


















              • add " to -s and you have the exact output that OP wants
                – RoVo
                Oct 23 at 9:36






              • 1




                Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
                – muru
                Oct 23 at 9:37












              • @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
                – Sugatur Deekshith S N
                Oct 23 at 9:43












              • then you can use my answer ;-)
                – RoVo
                Oct 23 at 9:44






              • 1




                @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
                – JigglyNaga
                Oct 23 at 9:58
















              add " to -s and you have the exact output that OP wants
              – RoVo
              Oct 23 at 9:36




              add " to -s and you have the exact output that OP wants
              – RoVo
              Oct 23 at 9:36




              1




              1




              Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
              – muru
              Oct 23 at 9:37






              Or if entries won't have newlines in them: jq -r '.children.values | .path.components[0],.type,.size' | paste - - -
              – muru
              Oct 23 at 9:37














              @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
              – Sugatur Deekshith S N
              Oct 23 at 9:43






              @JigglyNaga thanks for the detailed answer but I have one more question assume if I want store size value in variable how would I do that , because I want to convert those size values to MB
              – Sugatur Deekshith S N
              Oct 23 at 9:43














              then you can use my answer ;-)
              – RoVo
              Oct 23 at 9:44




              then you can use my answer ;-)
              – RoVo
              Oct 23 at 9:44




              1




              1




              @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
              – JigglyNaga
              Oct 23 at 9:58




              @muru Thanks, "no newlines" is more likely to be safe; although paste's single tab doesn't line up some of the later, longer entries.
              – JigglyNaga
              Oct 23 at 9:58











              2














              jq can render its output into a variety of formats: see https://stedolan.github.io/jq/manual/#Formatstringsandescaping



              For tab-separated output:



              $ jq -r '.children.values | [.path.name, .type, .size] | @tsv' file.json
              .gitignore FILE 224
              Jenkinsfile FILE 1396
              README.md FILE 237
              pom.xml FILE 2548
              src DIRECTORY





              share|improve this answer


























                2














                jq can render its output into a variety of formats: see https://stedolan.github.io/jq/manual/#Formatstringsandescaping



                For tab-separated output:



                $ jq -r '.children.values | [.path.name, .type, .size] | @tsv' file.json
                .gitignore FILE 224
                Jenkinsfile FILE 1396
                README.md FILE 237
                pom.xml FILE 2548
                src DIRECTORY





                share|improve this answer
























                  2












                  2








                  2






                  jq can render its output into a variety of formats: see https://stedolan.github.io/jq/manual/#Formatstringsandescaping



                  For tab-separated output:



                  $ jq -r '.children.values | [.path.name, .type, .size] | @tsv' file.json
                  .gitignore FILE 224
                  Jenkinsfile FILE 1396
                  README.md FILE 237
                  pom.xml FILE 2548
                  src DIRECTORY





                  share|improve this answer












                  jq can render its output into a variety of formats: see https://stedolan.github.io/jq/manual/#Formatstringsandescaping



                  For tab-separated output:



                  $ jq -r '.children.values | [.path.name, .type, .size] | @tsv' file.json
                  .gitignore FILE 224
                  Jenkinsfile FILE 1396
                  README.md FILE 237
                  pom.xml FILE 2548
                  src DIRECTORY






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Oct 23 at 16:37









                  glenn jackman

                  50.2k570107




                  50.2k570107























                      1














                      Solution with ramda-cli:



                      % curl ... | ramda -o tsv '.children.values' 'map flat' 'map props ["path.name", "type", "size"]'
                      .gitignore FILE 224
                      Jenkinsfile FILE 1396
                      README.md FILE 237
                      pom.xml FILE 2548
                      src DIRECTORY


                      First we traverse into the list of values, then map over the list with flat to convert each entry which is a deep object structure into a shallow one, with keys separated by dots.



                      Then, we can map over the list again, and pick the wanted properties based on their paths, represented with strings.



                      Finally, -o tsv takes care of converting the resulting list of lists into tsv format.



                      To debug or further understand what is happening, you may check what each argument does by removing them one by one from the end of the command and observing the difference in output at each step. They are simply operations (or functions) applied on the data one at a time from left to right.






                      share|improve this answer




























                        1














                        Solution with ramda-cli:



                        % curl ... | ramda -o tsv '.children.values' 'map flat' 'map props ["path.name", "type", "size"]'
                        .gitignore FILE 224
                        Jenkinsfile FILE 1396
                        README.md FILE 237
                        pom.xml FILE 2548
                        src DIRECTORY


                        First we traverse into the list of values, then map over the list with flat to convert each entry which is a deep object structure into a shallow one, with keys separated by dots.



                        Then, we can map over the list again, and pick the wanted properties based on their paths, represented with strings.



                        Finally, -o tsv takes care of converting the resulting list of lists into tsv format.



                        To debug or further understand what is happening, you may check what each argument does by removing them one by one from the end of the command and observing the difference in output at each step. They are simply operations (or functions) applied on the data one at a time from left to right.






                        share|improve this answer


























                          1












                          1








                          1






                          Solution with ramda-cli:



                          % curl ... | ramda -o tsv '.children.values' 'map flat' 'map props ["path.name", "type", "size"]'
                          .gitignore FILE 224
                          Jenkinsfile FILE 1396
                          README.md FILE 237
                          pom.xml FILE 2548
                          src DIRECTORY


                          First we traverse into the list of values, then map over the list with flat to convert each entry which is a deep object structure into a shallow one, with keys separated by dots.



                          Then, we can map over the list again, and pick the wanted properties based on their paths, represented with strings.



                          Finally, -o tsv takes care of converting the resulting list of lists into tsv format.



                          To debug or further understand what is happening, you may check what each argument does by removing them one by one from the end of the command and observing the difference in output at each step. They are simply operations (or functions) applied on the data one at a time from left to right.






                          share|improve this answer














                          Solution with ramda-cli:



                          % curl ... | ramda -o tsv '.children.values' 'map flat' 'map props ["path.name", "type", "size"]'
                          .gitignore FILE 224
                          Jenkinsfile FILE 1396
                          README.md FILE 237
                          pom.xml FILE 2548
                          src DIRECTORY


                          First we traverse into the list of values, then map over the list with flat to convert each entry which is a deep object structure into a shallow one, with keys separated by dots.



                          Then, we can map over the list again, and pick the wanted properties based on their paths, represented with strings.



                          Finally, -o tsv takes care of converting the resulting list of lists into tsv format.



                          To debug or further understand what is happening, you may check what each argument does by removing them one by one from the end of the command and observing the difference in output at each step. They are simply operations (or functions) applied on the data one at a time from left to right.







                          share|improve this answer














                          share|improve this answer



                          share|improve this answer








                          edited Oct 23 at 15:41

























                          answered Oct 23 at 15:34









                          raine

                          1114




                          1114






























                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Unix & Linux Stack Exchange!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.





                              Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                              Please pay close attention to the following guidance:


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f477210%2flooping-through-json-array-in-shell-script%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              Popular posts from this blog

                              Accessing regular linux commands in Huawei's Dopra Linux

                              Can't connect RFCOMM socket: Host is down

                              Kernel panic - not syncing: Fatal Exception in Interrupt