CLI: Numeric way to compare two video files

Multi tool use
Multi tool use












0















I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



What I have



For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




  • it only works for images


  • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


  • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)










share



























    0















    I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



    What I have



    For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




    • it only works for images


    • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


    • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)










    share

























      0












      0








      0








      I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



      What I have



      For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




      • it only works for images


      • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


      • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)










      share














      I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another.



      What I have



      For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One idea so far is to use ffmpeg to get pictures for every frame each x seconds, use compare from ImageMagic to get a numerical distance and use awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:




      • it only works for images


      • I have to manually tweak the threshold of the compare metric until I'm technically below that of artifacts to be confident enough to allow automatic deletions


      • I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)








      shell zsh





      share












      share










      share



      share










      asked 7 mins ago









      WhatWhat

      938




      938






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "106"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f503060%2fcli-numeric-way-to-compare-two-video-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Unix & Linux Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f503060%2fcli-numeric-way-to-compare-two-video-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          nu6GxvwD i5ifv7IJe8wwwGB,cI6n 6yr8G8YOEucOARUw9MbCs7M,Mt1AXt9c0e f2tYDV S9
          4sHQJ,5XhQugtQVh41b6,4o75nPdRxW6pJcM3zuE

          Popular posts from this blog

          Accessing regular linux commands in Huawei's Dopra Linux

          Can't connect RFCOMM socket: Host is down

          Kernel panic - not syncing: Fatal Exception in Interrupt