Python for Automation

Python is the better choice once logic grows: real data structures, error handling, libraries for everything and code that stays readable.

A script skeleton

#!/usr/bin/env python3
"""Archive log files older than N days."""
import argparse
import logging
import sys
from datetime import datetime, timedelta
from pathlib import Path

log = logging.getLogger("archive")


def parse_args():
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument("folder", type=Path)
    p.add_argument("--days", type=int, default=7)
    p.add_argument("--dry-run", action="store_true")
    return p.parse_args()


def main():
    args = parse_args()
    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
    cutoff = datetime.now() - timedelta(days=args.days)
    archive = args.folder / "archive"
    archive.mkdir(exist_ok=True)
    for f in args.folder.glob("*.log"):
        if datetime.fromtimestamp(f.stat().st_mtime) < cutoff:
            log.info("archiving %s", f.name)
            if not args.dry_run:
                f.rename(archive / f.name)
    return 0


if __name__ == "__main__":
    sys.exit(main())

argparse gives you --help for free; logging adds timestamps and levels; pathlib handles paths on every OS.

Running other programs safely

import subprocess

result = subprocess.run(
    ["pg_dump", "-Fc", "pandadev", "-f", "/tmp/db.dump"],
    capture_output=True, text=True, check=True, timeout=600,
)

• Pass a list of arguments. Never build a command string from user input with shell=True; that is a command-injection vulnerability.
• check=True raises an error if the command fails.
• timeout stops a hung process.

Reading configuration

import os

DATABASE_URL = os.environ["DATABASE_URL"]          # required: crash early if missing
BACKUP_DAYS = int(os.environ.get("BACKUP_DAYS", "14"))

Secrets come from environment variables or a secrets manager, never from the script file.

Working with data

import csv, json

with open("users.csv", newline="", encoding="utf-8") as fh:
    rows = list(csv.DictReader(fh))

active = [r for r in rows if r["status"] == "active"]
Path("active.json").write_text(json.dumps(active, indent=2), encoding="utf-8")

Error handling that helps

try:
    upload(path)
except ConnectionError as exc:
    log.error("upload failed for %s: %s", path, exc)
    return 1

Catch specific exceptions and log useful context. A bare except: hides bugs.

Exercise

Turn the disk-alert.sh exercise from the previous module into Python using shutil.disk_usage, with --threshold and --dry-run options and logging.