Find files with the same content in one folder (and all its sub-folders)












1














Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).



Is there any way I can do this on Mac?






macos file







share|improve this question
























  • I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
    – Nimesh Neema
    1 hour ago
















1














Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).



Is there any way I can do this on Mac?






macos file







share|improve this question
























  • I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
    – Nimesh Neema
    1 hour ago














1












1








1







Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).



Is there any way I can do this on Mac?






macos file







share|improve this question















Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).



Is there any way I can do this on Mac?






macos file




macos file






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago









Nimesh Neema

14.7k43871




14.7k43871










asked 2 hours ago









AGamePlayer

3681516




3681516












  • I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
    – Nimesh Neema
    1 hour ago


















  • I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
    – Nimesh Neema
    1 hour ago
















I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
1 hour ago




I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
1 hour ago










1 Answer
1






active

oldest

votes


















2














If you're confortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.



You'll need to install it with Homebrew first: brew install rdfind



Then, let's say you want to run a dry test on your ~/Download folder:



rdfind -n true -outputname result.log ~/Downloads/




  • -n true is for dry mode: no files will be affected


  • -outputname result.log will output the scanning into that file


  • ~/Downloads/ is the argument of the root folder you want to scan
    (you can pass more than one folder)


Running that command will look something like that:



$ rdfind -n true -outputname result.log ~/Downloads/

(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.

(DRYRUN MODE) Now have 20132 files in total.

(DRYRUN MODE) Removed 0 files due to nonunique device and inode.

(DRYRUN MODE) Now removing files with zero size from list...removed 75 files

(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB

(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.

(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.

(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.

(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.

(DRYRUN MODE) It seems like you have 15455 files that are not unique

(DRYRUN MODE) Totally, 324 MiB can be reduced.

(DRYRUN MODE) Now making results file result.log



Or, you can use an application with a graphical interface like dupeguru:



choose folder to scan



scan result






share|improve this answer










New contributor




Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.


















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "118"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f347217%2ffind-files-with-the-same-content-in-one-folder-and-all-its-sub-folders%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2














    If you're confortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.



    You'll need to install it with Homebrew first: brew install rdfind



    Then, let's say you want to run a dry test on your ~/Download folder:



    rdfind -n true -outputname result.log ~/Downloads/




    • -n true is for dry mode: no files will be affected


    • -outputname result.log will output the scanning into that file


    • ~/Downloads/ is the argument of the root folder you want to scan
      (you can pass more than one folder)


    Running that command will look something like that:



    $ rdfind -n true -outputname result.log ~/Downloads/
    
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
    
(DRYRUN MODE) Now have 20132 files in total.
    
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
    
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
    
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
    
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
    
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
    
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
    
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
    
(DRYRUN MODE) It seems like you have 15455 files that are not unique
    
(DRYRUN MODE) Totally, 324 MiB can be reduced.
    
(DRYRUN MODE) Now making results file result.log



    Or, you can use an application with a graphical interface like dupeguru:



    choose folder to scan



    scan result






    share|improve this answer










    New contributor




    Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.























      2














      If you're confortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.



      You'll need to install it with Homebrew first: brew install rdfind



      Then, let's say you want to run a dry test on your ~/Download folder:



      rdfind -n true -outputname result.log ~/Downloads/




      • -n true is for dry mode: no files will be affected


      • -outputname result.log will output the scanning into that file


      • ~/Downloads/ is the argument of the root folder you want to scan
        (you can pass more than one folder)


      Running that command will look something like that:



      $ rdfind -n true -outputname result.log ~/Downloads/
      
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
      
(DRYRUN MODE) Now have 20132 files in total.
      
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
      
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
      
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
      
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
      
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
      
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
      
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
      
(DRYRUN MODE) It seems like you have 15455 files that are not unique
      
(DRYRUN MODE) Totally, 324 MiB can be reduced.
      
(DRYRUN MODE) Now making results file result.log



      Or, you can use an application with a graphical interface like dupeguru:



      choose folder to scan



      scan result






      share|improve this answer










      New contributor




      Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





















        2












        2








        2






        If you're confortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.



        You'll need to install it with Homebrew first: brew install rdfind



        Then, let's say you want to run a dry test on your ~/Download folder:



        rdfind -n true -outputname result.log ~/Downloads/




        • -n true is for dry mode: no files will be affected


        • -outputname result.log will output the scanning into that file


        • ~/Downloads/ is the argument of the root folder you want to scan
          (you can pass more than one folder)


        Running that command will look something like that:



        $ rdfind -n true -outputname result.log ~/Downloads/
        
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
        
(DRYRUN MODE) Now have 20132 files in total.
        
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
        
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
        
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
        
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
        
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
        
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
        
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
        
(DRYRUN MODE) It seems like you have 15455 files that are not unique
        
(DRYRUN MODE) Totally, 324 MiB can be reduced.
        
(DRYRUN MODE) Now making results file result.log



        Or, you can use an application with a graphical interface like dupeguru:



        choose folder to scan



        scan result






        share|improve this answer










        New contributor




        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        If you're confortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.



        You'll need to install it with Homebrew first: brew install rdfind



        Then, let's say you want to run a dry test on your ~/Download folder:



        rdfind -n true -outputname result.log ~/Downloads/




        • -n true is for dry mode: no files will be affected


        • -outputname result.log will output the scanning into that file


        • ~/Downloads/ is the argument of the root folder you want to scan
          (you can pass more than one folder)


        Running that command will look something like that:



        $ rdfind -n true -outputname result.log ~/Downloads/
        
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
        
(DRYRUN MODE) Now have 20132 files in total.
        
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
        
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
        
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
        
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
        
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
        
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
        
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
        
(DRYRUN MODE) It seems like you have 15455 files that are not unique
        
(DRYRUN MODE) Totally, 324 MiB can be reduced.
        
(DRYRUN MODE) Now making results file result.log



        Or, you can use an application with a graphical interface like dupeguru:



        choose folder to scan



        scan result







        share|improve this answer










        New contributor




        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        share|improve this answer



        share|improve this answer








        edited 52 mins ago





















        New contributor




        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        answered 1 hour ago









        Yoric

        2335




        2335




        New contributor




        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.





        New contributor





        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






        Yoric is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Ask Different!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f347217%2ffind-files-with-the-same-content-in-one-folder-and-all-its-sub-folders%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Accessing regular linux commands in Huawei's Dopra Linux

            Image
            4 Backstory: I have a Huawei HG8245 router in my house and I want to change the default username/password. I've tried following this guide but the config file always gets reset. My ISP isn't helping either so I'm looking for other ways to accomplish this. The router has telnet access so this seems like a way to change the credentials. The first shell I gain access to looks like this: WAP> Pressing "?" gives me a list of available commands which I've pasted here, the two interesting ones are "su" and "shell". I can't seem to get su access, even after logging in as root (challenge password prompt, and then nothing happens), but shell gives me a proper linux shell. WAP>shell BusyBox v1.18.4 (2017-12-26 17:06:34 CST) built-in shell (ash) Enter 'hel...
            Read more

            Can't connect RFCOMM socket: Host is down

            Image
            up vote 3 down vote favorite I have bluez 5.4.3 and I am attempting to communicate with a ble device through minicom. These two deceives (my computer and a bluno nano) are alreadt paired and this has been my method so far. #bluetoothctl # pair xx:xx:xx:xx:xx:xx # exit Everything is working well so far then... #sudo rfcomm connect /dev/rfcomm0 xx:xx:xx:xx:xx:xx 1 & [1] 3665 Can't connect RFCOMM socket: Host is down Does anyone know how to resolve this issue? Any help would be very much appreciated. bluetooth bluez share | improve this question asked Feb 9 '17 at 18:46 Brandon Knape 46 2 ...
            Read more

            Kernel panic - not syncing: Fatal Exception in Interrupt

            Image
            0 Every day this error occured on my pc, kernel panic error on interrupt, what will be the problem? linux-mint share asked 6 mins ago Vinod Selvin Vinod Selvin 1 1 New contributor Vinod Selvin is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. add a comment ...
            Read more