rsync: skip files for which I don't have permissions












15














I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.






permissions rsync error-handling







share|improve this question
























  • Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
    – jordanm
    Feb 1 '13 at 23:15










  • @jordanm But there's no error code that's specific enough to pinpoint this error only.
    – Gilles
    Feb 2 '13 at 0:24
















15














I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.






permissions rsync error-handling







share|improve this question
























  • Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
    – jordanm
    Feb 1 '13 at 23:15










  • @jordanm But there's no error code that's specific enough to pinpoint this error only.
    – Gilles
    Feb 2 '13 at 0:24














15












15








15


2





I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.






permissions rsync error-handling







share|improve this question















I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore these. The trouble is that if rsync return non-zero my bash -x script will exit.






permissions rsync error-handling




permissions rsync error-handling






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Feb 2 '13 at 0:19









jasonwryan

49.1k14134184




49.1k14134184










asked Feb 1 '13 at 22:26









JeffCharter

17815




17815












  • Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
    – jordanm
    Feb 1 '13 at 23:15










  • @jordanm But there's no error code that's specific enough to pinpoint this error only.
    – Gilles
    Feb 2 '13 at 0:24


















  • Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
    – jordanm
    Feb 1 '13 at 23:15










  • @jordanm But there's no error code that's specific enough to pinpoint this error only.
    – Gilles
    Feb 2 '13 at 0:24
















Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15




Rsync has well documented exit values. You shouldn't treat all of them as a failure if your situation doesn't call for it to be a failure.
– jordanm
Feb 1 '13 at 23:15












@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24




@jordanm But there's no error code that's specific enough to pinpoint this error only.
– Gilles
Feb 2 '13 at 0:24










2 Answers
2






active

oldest

votes


















6














Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.



cd /source/directory

exclude_file=$(mktemp)

find . ! -readable -o -type d ! -executable |

  sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"

rsync -rlptD --exclude-from="$exclude_file" . /target/directory

rm "$exclude_file"


If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.



This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:



find . ( ! -readable -o -type d ! -executable ) -print0 |

  perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"





share|improve this answer























  • This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
    – jordanm
    Feb 2 '13 at 3:37










  • @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
    – JeffCharter
    Feb 4 '13 at 5:23






  • 1




    @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
    – Gilles
    Feb 4 '13 at 9:31



















0














I made a simple workaround for this specific situation:



rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)


This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.



It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.





share








New contributor




Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.


















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "106"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f63410%2frsync-skip-files-for-which-i-dont-have-permissions%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    6














    Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.



    cd /source/directory
    
exclude_file=$(mktemp)
    
find . ! -readable -o -type d ! -executable |
    
  sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
    
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
    
rm "$exclude_file"


    If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.



    This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:



    find . ( ! -readable -o -type d ! -executable ) -print0 |
    
  perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"





    share|improve this answer























    • This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
      – jordanm
      Feb 2 '13 at 3:37










    • @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
      – JeffCharter
      Feb 4 '13 at 5:23






    • 1




      @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
      – Gilles
      Feb 4 '13 at 9:31
















    6














    Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.



    cd /source/directory
    
exclude_file=$(mktemp)
    
find . ! -readable -o -type d ! -executable |
    
  sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
    
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
    
rm "$exclude_file"


    If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.



    This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:



    find . ( ! -readable -o -type d ! -executable ) -print0 |
    
  perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"





    share|improve this answer























    • This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
      – jordanm
      Feb 2 '13 at 3:37










    • @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
      – JeffCharter
      Feb 4 '13 at 5:23






    • 1




      @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
      – Gilles
      Feb 4 '13 at 9:31














    6












    6








    6






    Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.



    cd /source/directory
    
exclude_file=$(mktemp)
    
find . ! -readable -o -type d ! -executable |
    
  sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
    
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
    
rm "$exclude_file"


    If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.



    This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:



    find . ( ! -readable -o -type d ! -executable ) -print0 |
    
  perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"





    share|improve this answer














    Rsync doesn't have an option for this. I see two solutions. One is to parse rsync error messages; this isn't very robust. The other is to generate a list of unreadable files to filter.



    cd /source/directory
    
exclude_file=$(mktemp)
    
find . ! -readable -o -type d ! -executable |
    
  sed -e 's:^./:/:' -e 's:[?*\:\1:g' >>"$exclude_file"
    
rsync -rlptD --exclude-from="$exclude_file" . /target/directory
    
rm "$exclude_file"


    If your find doesn't have -readable and -executable, replace them by the appropriate -perm directive.



    This assumes that there are no unreadable files whose name contains a newline. If you need to cope with those, you'll need to produce a null-delimited file list like this, and pass the -0 option to rsync:



    find . ( ! -readable -o -type d ! -executable ) -print0 |
    
  perl -0000 -pe 's:A./:/:' -e 's:[?*\:$1:g' >>"$exclude_file"






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Feb 4 '13 at 9:30

























    answered Feb 2 '13 at 0:18









    Gilles

    528k12810571583




    528k12810571583












    • This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
      – jordanm
      Feb 2 '13 at 3:37










    • @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
      – JeffCharter
      Feb 4 '13 at 5:23






    • 1




      @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
      – Gilles
      Feb 4 '13 at 9:31


















    • This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
      – jordanm
      Feb 2 '13 at 3:37










    • @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
      – JeffCharter
      Feb 4 '13 at 5:23






    • 1




      @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
      – Gilles
      Feb 4 '13 at 9:31
















    This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
    – jordanm
    Feb 2 '13 at 3:37




    This is probably his best bet. Hopefully, he only needs to run this once, as having to stat() every file twice (once via rsync and once via find) would be pretty bad.
    – jordanm
    Feb 2 '13 at 3:37












    @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
    – JeffCharter
    Feb 4 '13 at 5:23




    @Gilles this seems to work, except for hidden files. I'm assuming the same strategy will work with some minor tweaking. I'm not familiar with the ! (slash exclamation) could you explain that?
    – JeffCharter
    Feb 4 '13 at 5:23




    1




    1




    @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
    – Gilles
    Feb 4 '13 at 9:31




    @JeffC ! quotes the ! operator to protect it against shell expansion. The backslash isn't actually necessary here since no shell expands ! when it's followed by a space, but it doesn't hurt. What's wrong with hidden files?
    – Gilles
    Feb 4 '13 at 9:31













    0














    I made a simple workaround for this specific situation:



    rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)


    This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.



    It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.





    share








    New contributor




    Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.























      0














      I made a simple workaround for this specific situation:



      rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)


      This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.



      It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.





      share








      New contributor




      Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





















        0












        0








        0






        I made a simple workaround for this specific situation:



        rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)


        This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.



        It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.





        share








        New contributor




        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        I made a simple workaround for this specific situation:



        rsync --args || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)


        This returns 0 if the returned code was 0 or 23, and returns the exit code in all other cases.



        It is important to note, however, that this would ignore all Partial transfer due to error errors, not just permission ones, since it will catch everything that exits code 23. For more information about rsync status codes please refer to this link.






        share








        New contributor




        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.








        share


        share






        New contributor




        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        answered 8 mins ago









        Gus

        1012




        1012




        New contributor




        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.





        New contributor





        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






        Gus is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f63410%2frsync-skip-files-for-which-i-dont-have-permissions%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Accessing regular linux commands in Huawei's Dopra Linux

            Image
            4 Backstory: I have a Huawei HG8245 router in my house and I want to change the default username/password. I've tried following this guide but the config file always gets reset. My ISP isn't helping either so I'm looking for other ways to accomplish this. The router has telnet access so this seems like a way to change the credentials. The first shell I gain access to looks like this: WAP> Pressing "?" gives me a list of available commands which I've pasted here, the two interesting ones are "su" and "shell". I can't seem to get su access, even after logging in as root (challenge password prompt, and then nothing happens), but shell gives me a proper linux shell. WAP>shell BusyBox v1.18.4 (2017-12-26 17:06:34 CST) built-in shell (ash) Enter 'hel...
            Read more

            Can't connect RFCOMM socket: Host is down

            Image
            up vote 3 down vote favorite I have bluez 5.4.3 and I am attempting to communicate with a ble device through minicom. These two deceives (my computer and a bluno nano) are alreadt paired and this has been my method so far. #bluetoothctl # pair xx:xx:xx:xx:xx:xx # exit Everything is working well so far then... #sudo rfcomm connect /dev/rfcomm0 xx:xx:xx:xx:xx:xx 1 & [1] 3665 Can't connect RFCOMM socket: Host is down Does anyone know how to resolve this issue? Any help would be very much appreciated. bluetooth bluez share | improve this question asked Feb 9 '17 at 18:46 Brandon Knape 46 2 ...
            Read more

            Kernel panic - not syncing: Fatal Exception in Interrupt

            Image
            0 Every day this error occured on my pc, kernel panic error on interrupt, what will be the problem? linux-mint share asked 6 mins ago Vinod Selvin Vinod Selvin 1 1 New contributor Vinod Selvin is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. add a comment ...
            Read more