CLI: Numeric way to compare two video files












0















I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



What I have



For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




  • it only works for images


  • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


  • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)







shell zsh






share



























    0















    I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



    What I have



    For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




    • it only works for images


    • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


    • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)







    shell zsh






    share

























      0












      0








      0








      I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



      What I have



      For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




      • it only works for images


      • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


      • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)







      shell zsh






      share














      I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



      What I have



      For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




      • it only works for images


      • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


      • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)







      shell zsh




      shell zsh





      share












      share










      share



      share










      asked 7 mins ago









      WhatWhat

      938




      938






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "106"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f503060%2fcli-numeric-way-to-compare-two-video-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Unix & Linux Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f503060%2fcli-numeric-way-to-compare-two-video-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Accessing regular linux commands in Huawei's Dopra Linux

          Image
          4 Backstory: I have a Huawei HG8245 router in my house and I want to change the default username/password. I've tried following this guide but the config file always gets reset. My ISP isn't helping either so I'm looking for other ways to accomplish this. The router has telnet access so this seems like a way to change the credentials. The first shell I gain access to looks like this: WAP> Pressing "?" gives me a list of available commands which I've pasted here, the two interesting ones are "su" and "shell". I can't seem to get su access, even after logging in as root (challenge password prompt, and then nothing happens), but shell gives me a proper linux shell. WAP>shell BusyBox v1.18.4 (2017-12-26 17:06:34 CST) built-in shell (ash) Enter 'hel...
          Read more

          Can't connect RFCOMM socket: Host is down

          Image
          up vote 3 down vote favorite I have bluez 5.4.3 and I am attempting to communicate with a ble device through minicom. These two deceives (my computer and a bluno nano) are alreadt paired and this has been my method so far. #bluetoothctl # pair xx:xx:xx:xx:xx:xx # exit Everything is working well so far then... #sudo rfcomm connect /dev/rfcomm0 xx:xx:xx:xx:xx:xx 1 & [1] 3665 Can't connect RFCOMM socket: Host is down Does anyone know how to resolve this issue? Any help would be very much appreciated. bluetooth bluez share | improve this question asked Feb 9 '17 at 18:46 Brandon Knape 46 2 ...
          Read more

          Kernel panic - not syncing: Fatal Exception in Interrupt

          Image
          0 Every day this error occured on my pc, kernel panic error on interrupt, what will be the problem? linux-mint share asked 6 mins ago Vinod Selvin Vinod Selvin 1 1 New contributor Vinod Selvin is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. add a comment ...
          Read more