Sstrhsrtj

Given a large list of files, containing the following:

FILE1.doc

FILE1.pdf

FILE2.doc

FILE3.doc

FILE3.pdf

FILE4.doc

Is there a terminal command that would allow me to remove all files that do not have a duplicate name in the list? In this case...FILE2.doc and FILE4.doc?

Using bash, this will remove all files that don't have another file with the same name but different extension:

for f in *; do same=("${f%.*}".*); [ "${#same[@]}" -eq 1 ] && rm "$f"; done

This approach is safe for all file names, even those with white space in their names.

How it works

• 
  

  for f in *; do

  
  
  

  This starts a loop over all files in the current directory.

  
  


• 
  

  same=("${f%.*}".*)

  
  
  

  This creates a bash array with the names of all files with the same basename.

  
  
  

  $f is the name of our file. ${f%.*} is the name of the file without its extension. If, for example, the file is FILE1.doc, then ${f%.*} is FILE1. "${f%.*}".* is all the files with the same basename but any extension. ("${f%.*}".*) is a bash array of those names. same=("${f%.*}".*) assigns the array to the variable same.

  
  


• 
  

  [ "${#same[@]}" -eq 1 ] && rm "$f"

  
  
  

  If there is only one file with this basename, we delete it.

  
  
  

  "${#same[@]}" is the number of files in the array same. [ "${#same[@]}" -eq 1 ] is true if there is only one such file.

  
  
  

  && is logical-and. It causes the statement which follows, rm "$f" to be executed only if the statement which precedes it returns logical true.

  
  


• 
  

  done

  
  
  

  This marks the end of the for loop.

  
  

Suppose your list of files is in some file /tmp/files.list, e.g. after ls * > /tmp/files.list ;

Then sort -u /tmp/files.list gives you a sorted file lists without duplicate (not needed if you did the ls * > /tmp/files.list above). You could process that with some awk script inspired from this, e.g.

sort -u /tmp/files.list | awk '{

  function basename(file) {

    sub(".*/", "", file)

    return file

  }

curfil=$0;

if (basename(curfil)==basename(prevfil)) system("rm " + curfil);

prevfil=curfil;

}'

Beware, I have not tested this.

Another simple looking complex way can be:

for x in `for i in *; do echo $i ; done | cut -d'.' -f1 | uniq -u `; do rm $x.*; done