Software Developer

Linux Terminal Commands For Managing Files in Bulk – Cheat Sheet

This is a quick reference cheat sheet for Linux commands to manage files on the command line.

Counting Files

Count the files (including directories) in a directory. This gives you the total number of files:

ls | wc -l

Count the files in a directory, per each type of extension. This will tell you how many of each type of file there is in a directory. It’s a quick summary and doesn’t list each single file:

find -type f | sed -e 's/.*\.//' | sort | uniq -c

Count all the files in a directory of one particular extension. This example counts all .jpg files. It will give you the count, but this doesn’t list each file. (The following command below this next one lists each file.)

ls | grep '\.jpg$' | wc -l

Listing Filenames

List Files by Extension
List all files of a particular extension in a directory. This one is not recursive. (See below for the recursive one.) This example lists all .jpg files:

ls | grep '\.jpg$'

Recursively list all files of a specific extension in a directory and all sub-directories. This example lists all .jpg files, and the sub-directory it is found in, if any:

find . -type f -name "*.jpg"

 

List Files By Size
Find all files in a directory with a specific file size. This example lists specific files that are 50000k or larger:

find -type f -size +50000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }' 

List Directories Sorted by Size

List all folders in a directory sorted by MegaBytes. This will give the size, in MB, of all folders:

du --max-depth=1 | sort -n | awk 'BEGIN {OFMT = "%.0f"} {print $1/1024,"MB", $2}' 

Compare List of Files in 2 Text Files – Get the Difference of 2 Lists

If you have two text files that each contain a list of filenames, this lets you find the filenames that are in one list and not in the other list. This will find and list all files that are in “file_1.txt” but not in “file_2.txt”:

grep -v -f file_2.txt file_1.txt

To see the opposite, that is, the files that are in “file_2.txt” but not in “file_1.txt”, flip the order the files in the command, like this:

grep -v -f file_1.txt file_2.txt

Compare Files in a Directory

Compare the contents of two folders with the diff command. This lets you see the difference between two directories, meaning the difference of files between two folders. This compares just filenames, and not the file contents:

diff <(cd folder1 && find . | sort) <(cd folder2 && find . | sort)

Search Contents of All Files For Strings Listed in a Text File

Search the content of all files in a directory for each string listed in a text file. In other words, search to find all strings that are listed in a text file, searching the contents of a specified directory, recursively. In the following example command, the search strings are listed in a file called “searchlist.txt” with each search string on a new line.

find /home/path/to/search/dir -type f -exec grep -if searchlist.txt /dev/null {} +

Deleting Files

To delete all files listed in a text file, assuming your list is saved in a file called “delete_list.txt” with each file on a new line. It should list the full path to the file, not just the filename. This command will delete all files listed in the text file:

xargs rm -r <delete_list.txt

Renaming Files

Bulk rename all files in a folder. This example changes the extension of all files in the directory from .htm to .html:

rename 's/\.htm$/\.html/' *.htm

To do a test run for the line above, add -n after rename. This will not actually change anything. It will show the proposed changes:

rename -n 's/\.htm$/\.html/' *.htm

More examples of bulk renaming here.

By

Questions and Comments are Welcome

Your email address will not be published. All comments will be moderated.

Please wrap code in "code" bracket tags like this:

[code]

YOUR CODE HERE 

[/code]