Automate Boring Tasks: Organise Files and Folders with Python

Many people first feel the power of programming when Python does a dull job for them: sorting a downloads folder, renaming hundreds of photos, collecting files from many folders into one place. The tools are pathlib for paths and shutil for moving and copying. Because these programs change things on your computer, we will go carefully, test on practice files first and build in safety checks.

Safety first

• Practise on a copy or on practice files, never on your only copy of something important.
• Print what you would do before you do it. A *dry run* shows the plan and changes nothing. When it looks right, switch to the real run.
• Never overwrite silently. Check whether the destination exists, and skip or rename.
• Avoid deleting. Moving files into a folder is safer than removing them. If you do use rmtree or unlink, be very sure of the path.
• Keep a record of what you changed, even a simple printed list.

The program

The program below creates its own practice folder with seven small files, sorts them by type into sub-folders, shows the final layout and removes the practice folder at the end. Read it before you run it.

import shutil
from pathlib import Path

# Make a practice folder with some untidy files
folder = Path("downloads_demo")
folder.mkdir(exist_ok=True)
for name in ["cv.pdf", "photo1.jpg", "photo2.JPG", "notes.txt", "budget.xlsx", "report.pdf", "README"]:
    (folder / name).write_text("demo", encoding="utf-8")

groups = {
    ".pdf": "documents", ".txt": "documents",
    ".jpg": "images", ".png": "images",
    ".xlsx": "spreadsheets",
}

for item in sorted(folder.iterdir()):
    if not item.is_file():
        continue
    target_name = groups.get(item.suffix.lower())
    if target_name is None:
        print("Left alone:", item.name)
        continue
    target = folder / target_name
    target.mkdir(exist_ok=True)
    destination = target / item.name.lower()
    if destination.exists():
        print("Skipped (exists):", item.name)
        continue
    shutil.move(str(item), destination)
    print(f"Moved {item.name} -> {target_name}/")

for path in sorted(folder.rglob("*")):
    print(path.relative_to(folder))

shutil.rmtree(folder)

Left alone: README
Moved budget.xlsx -> spreadsheets/
Moved cv.pdf -> documents/
Moved notes.txt -> documents/
Moved photo1.jpg -> images/
Moved photo2.JPG -> images/
Moved report.pdf -> documents/
README
documents
documents/cv.pdf
documents/notes.txt
documents/report.pdf
images
images/photo1.jpg
images/photo2.jpg
spreadsheets
spreadsheets/budget.xlsx

What happens

• folder.iterdir() lists everything in the folder. We loop over sorted(...) so the order is the same on every computer. item.is_file() skips folders.
• item.suffix is the file ending, such as .pdf. .lower() makes .JPG and .jpg the same. A dictionary maps each ending to a group name. groups.get(...) returns None for unknown endings, like the file README, which is left alone.
• target.mkdir(exist_ok=True) creates the group folder if it does not exist, and does nothing if it does.
• Before moving, we check destination.exists(). If a file with that name is already there, the program skips it and says so, so nothing is overwritten.
• shutil.move moves the file. The name is made lower case in the new place, which is why photo2.JPG became photo2.jpg.
• folder.rglob("*") walks through all files and folders below, and relative_to shortens the printed paths.

Turning it into a real tool

To use the idea on a real folder, change the folder line to the path you want, for example Path.home() / "Downloads", remove the lines that create the practice files and the final rmtree, and add the dry-run switch below. Run it in dry-run mode first and read the plan.

DRY_RUN = True   # change to False only after you have read the plan

if DRY_RUN:
    print(f"Would move {item.name} -> {target_name}/")
else:
    target.mkdir(exist_ok=True)
    shutil.move(str(item), destination)
    print(f"Moved {item.name} -> {target_name}/")

More ideas for automation

• Rename in bulk: add a date to file names, replace spaces with underscores, or number photos in order. Use item.rename(new_path) or item.with_name(new_name).
• Find files: folder.rglob("*.pdf") finds every PDF in all sub-folders.
• Merge CSV files: loop over files with rglob, read each with csv, and write one combined file.
• Clean old files: list files older than 90 days using item.stat().st_mtime. Print them first, and move them to an archive folder before deciding to delete.
• Make backups: shutil.copytree copies a whole folder, and shutil.make_archive makes a zip.

Running a script on a schedule

Once a script works, you can have the computer run it automatically. On Windows, Task Scheduler does this. On macOS and Linux, cron does it. First make sure that the script works when it is started from any folder, which means using full paths, and that it writes a log file, so you can see what happened while you were away.

Working with spreadsheets and documents

Python can also read and write Excel files with the openpyxl package, and Word files with python-docx. They need installing with pip inside a virtual environment, as shown in lesson 10. The idea is the same as with CSV: read the data, process it with loops and dictionaries, and write a result. Check each package's own documentation, because details change between versions.

Your task

1. Run the practice program and compare the printed layout with the output above.
2. Add the endings .docx and .pptx to a group called documents, and .zip to a group called archives.
3. Add the DRY_RUN switch. Run once with True and once with False.
4. Write a second script that lists all .txt files below a folder and prints their size in kilobytes.

How to check it worked

• Your layout shows documents, images and spreadsheets folders with the files inside.
• In dry-run mode, no file moves and the messages start with Would move.
• A file whose name already exists in the target is skipped with a message.