rsync: skip files for which I don't have permissions
I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.
permissions rsync error-handling
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
add a comment |
I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.
permissions rsync error-handling
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24
add a comment |
I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.
permissions rsync error-handling
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.
permissions rsync error-handling
permissions rsync error-handling
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
edited Feb 2 '13 at 0:19
jasonwryan
49.1k14134184
49.1k14134184
asked Feb 1 '13 at 22:26
JeffCharter
17815
asked Feb 1 '13 at 22:26
JeffCharter
17815
asked Feb 1 '13 at 22:26
JeffCharter
17815
17815
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24
add a comment |
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24
add a comment |
2 Answers
2
active
oldest
votes
Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.
cd /source/directory
exclude_file=$(mktemp)
find . ! -readable -o -type d ! -executable |
sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
rm "$exclude_file"
If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.
This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:
find . ( ! -readable -o -type d ! -executable ) -print0 |
perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
This is probably his best bet. Hopefully, he only needs to run this once, as having tostat()every file twice (once via rsync and once via find) would be pretty bad.
– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
@JeffC!quotes the!operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands!when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
– Gilles
Feb 4 '13 at 9:31
add a comment |
I made a simple workaround for this specific situation:
rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.
It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f63410%2frsync-skip-files-for-which-i-dont-have-permissions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.
cd /source/directory
exclude_file=$(mktemp)
find . ! -readable -o -type d ! -executable |
sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
rm "$exclude_file"
If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.
This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:
find . ( ! -readable -o -type d ! -executable ) -print0 |
perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
This is probably his best bet. Hopefully, he only needs to run this once, as having tostat()every file twice (once via rsync and once via find) would be pretty bad.
– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
@JeffC!quotes the!operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands!when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
– Gilles
Feb 4 '13 at 9:31
add a comment |
Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.
cd /source/directory
exclude_file=$(mktemp)
find . ! -readable -o -type d ! -executable |
sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
rm "$exclude_file"
If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.
This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:
find . ( ! -readable -o -type d ! -executable ) -print0 |
perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
This is probably his best bet. Hopefully, he only needs to run this once, as having tostat()every file twice (once via rsync and once via find) would be pretty bad.
– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
@JeffC!quotes the!operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands!when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
– Gilles
Feb 4 '13 at 9:31
add a comment |
Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.
cd /source/directory
exclude_file=$(mktemp)
find . ! -readable -o -type d ! -executable |
sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
rm "$exclude_file"
If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.
This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:
find . ( ! -readable -o -type d ! -executable ) -print0 |
perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.
cd /source/directory
exclude_file=$(mktemp)
find . ! -readable -o -type d ! -executable |
sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
rm "$exclude_file"
If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.
This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:
find . ( ! -readable -o -type d ! -executable ) -print0 |
perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
edited Feb 4 '13 at 9:30
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
answered Feb 2 '13 at 0:18
Gilles
528k12810571583
528k12810571583
This is probably his best bet. Hopefully, he only needs to run this once, as having tostat()every file twice (once via rsync and once via find) would be pretty bad.
– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
@JeffC!quotes the!operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands!when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
– Gilles
Feb 4 '13 at 9:31
add a comment |
This is probably his best bet. Hopefully, he only needs to run this once, as having tostat()every file twice (once via rsync and once via find) would be pretty bad.
– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
@JeffC!quotes the!operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands!when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
– Gilles
Feb 4 '13 at 9:31
This is probably his best bet. Hopefully, he only needs to run this once, as having to
stat() every file twice (once via rsync and once via find) would be pretty bad.– jordanm
Feb 2 '13 at 3:37
This is probably his best bet. Hopefully, he only needs to run this once, as having to
stat() every file twice (once via rsync and once via find) would be pretty bad.– jordanm
Feb 2 '13 at 3:37
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
@Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
– JeffCharter
Feb 4 '13 at 5:23
1
1
@JeffC
! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?– Gilles
Feb 4 '13 at 9:31
@JeffC
! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?– Gilles
Feb 4 '13 at 9:31
add a comment |
I made a simple workaround for this specific situation:
rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.
It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I made a simple workaround for this specific situation:
rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.
It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I made a simple workaround for this specific situation:
rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.
It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I made a simple workaround for this specific situation:
rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.
It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 8 mins ago
Gus
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 8 mins ago
Gus
1012
answered 8 mins ago
Gus
1012
1012
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f63410%2frsync-skip-files-for-which-i-dont-have-permissions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15
@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24