How to create a recursive manifest directory











up vote
1
down vote

favorite












I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question









New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    2 days ago










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    2 days ago










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    13 hours ago















up vote
1
down vote

favorite












I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question









New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    2 days ago










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    2 days ago










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    13 hours ago













up vote
1
down vote

favorite









up vote
1
down vote

favorite











I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question









New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx






linux ls






share|improve this question









New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 13 hours ago





















New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 days ago









ioshifting

62




62




New contributor




ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






ioshifting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    2 days ago










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    2 days ago










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    13 hours ago














  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    2 days ago










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    2 days ago










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    13 hours ago








1




1




In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
2 days ago




In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
2 days ago












ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago




ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago












Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago




Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago










3 Answers
3






active

oldest

votes

















up vote
1
down vote













From man:




  -R, --recursive
list subdirectories recursively



Try,



ls -R /path/to/dir


more closely,



ll -R /path/to/dir | awk '$1!="total"{print $NF}'





share|improve this answer





















  • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    13 hours ago










  • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    12 hours ago


















up vote
0
down vote













find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





share|improve this answer




























    up vote
    0
    down vote













    Since you want to find directories and then list their contents, why not use find for it:



    find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
    for pathname do
    header=0
    for filename in "$pathname"/*; do
    if [ ! -d "$filename" ]; then
    if [ "$header" -eq 0 ]; then
    printf "%s/n" "${pathname%/}"
    header=1
    fi
    printf "%sn" "${filename##*/}"
    fi
    done
    [ "$header" -eq 1 ] && printf "n"
    done' bash {} +


    This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



    The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



    Directories that are empty or that only contains directories are not listed.



    For a directory structure such as



    /dir
    `-- a
    |-- b
    | `-- c
    | |-- .hidden_file
    | `-- file
    `-- file


    This would produce the following output:



    /dir/a/
    file

    /dir/a/b/c/
    .hidden_file
    file





    share|improve this answer





















    • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
      – ioshifting
      13 hours ago










    • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
      – Kusalananda
      5 hours ago











    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "106"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });






    ioshifting is a new contributor. Be nice, and check out our Code of Conduct.










    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote













    From man:




      -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"{print $NF}'





    share|improve this answer





















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      13 hours ago










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      12 hours ago















    up vote
    1
    down vote













    From man:




      -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"{print $NF}'





    share|improve this answer





















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      13 hours ago










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      12 hours ago













    up vote
    1
    down vote










    up vote
    1
    down vote









    From man:




      -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"{print $NF}'





    share|improve this answer












    From man:




      -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"{print $NF}'






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered 2 days ago









    msp9011

    3,64043863




    3,64043863












    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      13 hours ago










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      12 hours ago


















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      13 hours ago










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      12 hours ago
















    the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    13 hours ago




    the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    13 hours ago












    do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    12 hours ago




    do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    12 hours ago












    up vote
    0
    down vote













    find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





    share|improve this answer

























      up vote
      0
      down vote













      find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





      share|improve this answer























        up vote
        0
        down vote










        up vote
        0
        down vote









        find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





        share|improve this answer












        find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 2 days ago









        Ole Tange

        11.9k1451105




        11.9k1451105






















            up vote
            0
            down vote













            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "${pathname%/}"
            header=1
            fi
            printf "%sn" "${filename##*/}"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash {} +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer





















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              13 hours ago










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              5 hours ago















            up vote
            0
            down vote













            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "${pathname%/}"
            header=1
            fi
            printf "%sn" "${filename##*/}"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash {} +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer





















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              13 hours ago










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              5 hours ago













            up vote
            0
            down vote










            up vote
            0
            down vote









            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "${pathname%/}"
            header=1
            fi
            printf "%sn" "${filename##*/}"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash {} +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer












            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "${pathname%/}"
            header=1
            fi
            printf "%sn" "${filename##*/}"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash {} +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered yesterday









            Kusalananda

            119k16223364




            119k16223364












            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              13 hours ago










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              5 hours ago


















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              13 hours ago










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              5 hours ago
















            This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
            – ioshifting
            13 hours ago




            This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
            – ioshifting
            13 hours ago












            @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
            – Kusalananda
            5 hours ago




            @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
            – Kusalananda
            5 hours ago










            ioshifting is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            ioshifting is a new contributor. Be nice, and check out our Code of Conduct.













            ioshifting is a new contributor. Be nice, and check out our Code of Conduct.












            ioshifting is a new contributor. Be nice, and check out our Code of Conduct.
















            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            サソリ

            広島県道265号伴広島線

            Accessing regular linux commands in Huawei's Dopra Linux