How to create a recursive manifest directory
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
New contributor
add a comment |
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
New contributor
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
New contributor
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
linux ls
New contributor
New contributor
edited 13 hours ago
New contributor
asked 2 days ago
ioshifting
62
62
New contributor
New contributor
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago
add a comment |
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago
1
1
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
2 days ago
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago
add a comment |
3 Answers
3
active
oldest
votes
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"{print $NF}'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "${pathname%/}"
header=1
fi
printf "%sn" "${filename##*/}"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash {} +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"{print $NF}'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
add a comment |
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"{print $NF}'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
add a comment |
up vote
1
down vote
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"{print $NF}'
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"{print $NF}'
answered 2 days ago
msp9011
3,64043863
3,64043863
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
add a comment |
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
13 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
12 hours ago
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
answered 2 days ago
Ole Tange
11.9k1451105
11.9k1451105
add a comment |
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "${pathname%/}"
header=1
fi
printf "%sn" "${filename##*/}"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash {} +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "${pathname%/}"
header=1
fi
printf "%sn" "${filename##*/}"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash {} +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
add a comment |
up vote
0
down vote
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "${pathname%/}"
header=1
fi
printf "%sn" "${filename##*/}"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash {} +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "${pathname%/}"
header=1
fi
printf "%sn" "${filename##*/}"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash {} +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
answered yesterday
Kusalananda
119k16223364
119k16223364
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
add a comment |
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
13 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
5 hours ago
add a comment |
ioshifting is a new contributor. Be nice, and check out our Code of Conduct.
ioshifting is a new contributor. Be nice, and check out our Code of Conduct.
ioshifting is a new contributor. Be nice, and check out our Code of Conduct.
ioshifting is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
2 days ago
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
2 days ago
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
13 hours ago