Creating Command Line Tools in Python

A command-line parser should accept the arguments its documentation promises, reject the ones it cannot interpret, and avoid silently assigning a different meaning to either. In Python, argparse is a good default, but its defaults are still design choices. For example, it accepts abbreviated long options unless you disable that behavior. An option such as --out may work today because --output is the only matching name, then become ambiguous when a later release adds --outdir.

import argparse

parser = argparse.ArgumentParser(
    prog="report",
    allow_abbrev=False,
)
parser.add_argument("--output", required=True)
parser.add_argument("--verbose", action="store_true")

args = parser.parse_args()

Whether abbreviations are desirable depends on the program, but accidental abbreviations are a poor interface contract. If users rely on them, adding a new option can break an existing script. Disabling them makes every accepted spelling explicit and keeps the command’s behavior stable as its option set grows.

Be equally deliberate about what happens to arguments the parser does not recognize. parse_args() reports them as errors; parse_known_args() returns them to the caller. The latter is useful when a wrapper intentionally forwards arguments to another program, but it is dangerous as a general-purpose way to “be flexible”: a misspelled option can otherwise pass unnoticed.

parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument("--format", choices=("text", "json"), default="text")

args, remaining = parser.parse_known_args()

if remaining:
    print(f"Forwarding arguments: {remaining}")

Use that pattern only when forwarding is part of the command’s documented behavior. If the program owns the whole command line, parse all of it and let misspellings fail clearly. A user who types --formt json should not get a successful run that quietly produces text output.

Defaults should also be unsurprising. Choose a default that is safe and consistent with ordinary usage, and make it visible when its effect matters. Avoid defaults that depend on the current working directory, the machine’s locale, or an implicit environment setting unless that dependency is part of the interface. When a value can come from several places—such as a command-line option, an environment variable, or a configuration file—define and document the precedence rather than letting implementation order decide it.

import argparse
import os

parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument("--timeout", type=float)

args = parser.parse_args()

timeout = (
    args.timeout
    if args.timeout is not None
    else float(os.environ.get("REPORT_TIMEOUT", "10"))
)

Keep “not supplied” distinct from a supplied value that happens to be false, zero, or empty. A default of 0 cannot tell you whether the user explicitly requested zero; None can. That distinction matters when command-line input is supposed to override configuration, including when the requested value is meaningful precisely because it disables a behavior.

Validate values at the boundary where they become meaningful. A parser can convert a string to a number, but conversion alone does not establish that the number is valid for the program. Report the option and the invalid value together, rather than allowing a later operation to fail with an unrelated exception.

def positive_int(value):
    try:
        number = int(value)
    except ValueError:
        raise argparse.ArgumentTypeError(
            f"expected an integer, got {value!r}"
        )

    if number <= 0:
        raise argparse.ArgumentTypeError(
            f"expected a positive integer, got {number}"
        )

    return number


parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument("--workers", type=positive_int, default=4)

Do not make parsing perform the requested work. Parsing should turn tokens into a structured description of the user’s request; opening files, deleting data, or starting network activity before the full command has been validated makes failures harder to understand and can leave partial side effects. Parse first, check combinations of options next, and only then execute the operation.

Choosing between positional arguments and options

Positional arguments work best for the command’s primary operands: the files to process, the name of a resource to inspect, or the value that identifies the operation. Their position supplies meaning, so they are easy to type in the common case, but each additional positional argument makes the command’s grammar more dependent on order.

import argparse

parser = argparse.ArgumentParser(prog="convert", allow_abbrev=False)
parser.add_argument("source")
parser.add_argument("destination")
parser.add_argument("--format", choices=("json", "yaml"), default="json")

args = parser.parse_args()

This interface is clear when every invocation has exactly one source and one destination. It becomes less clear when a third positional value is added, especially if that value is optional or has the same shape as a filename. Users must remember which slot it occupies, and changing the order later can break scripts that have no option names to signal their intent.

Use options for settings that modify the operation, select among independent behaviors, or may be omitted without changing the identity of the main operands. Names such as --format and --timeout make those values self-describing; their order can vary, and new options can usually be added without changing existing positional syntax.

parser = argparse.ArgumentParser(prog="convert", allow_abbrev=False)
parser.add_argument("source")
parser.add_argument("--output", required=True)
parser.add_argument("--format", choices=("json", "yaml"), default="json")
parser.add_argument("--overwrite", action="store_true")

args = parser.parse_args()

A required option is sometimes preferable to a second positional argument when the value is easy to confuse with another operand, or when the command has several modes with different required inputs. The extra spelling is a small cost for a command line that remains understandable when read later in a shell history or build script.

For a variable number of homogeneous operands, a positional with nargs can be concise, but it needs a clear boundary from options and other values. A common arrangement is one or more input paths followed by named options; avoid placing another positional after a greedy positional, because users and parsers can have difficulty determining which token belongs to which argument.

parser = argparse.ArgumentParser(prog="archive", allow_abbrev=False)
parser.add_argument("inputs", nargs="+")
parser.add_argument("--output", required=True)
parser.add_argument("--compression", choices=("fast", "best"), default="fast")

args = parser.parse_args()

If a value is repeatable and has a distinct role, a repeated option often communicates that role better than an undifferentiated list of positionals. It also allows the interface to grow without assigning a new meaning to a token based solely on its location.

parser = argparse.ArgumentParser(prog="search", allow_abbrev=False)
parser.add_argument("root")
parser.add_argument("--include", action="append", default=[])
parser.add_argument("--exclude", action="append", default=[])

args = parser.parse_args()

Some filenames begin with a hyphen and can be mistaken for options. Users can mark the end of option parsing with --, after which tokens are treated as positional arguments, including names such as -draft.txt.

# Example invocation:
# search . --include "*.py" -- -draft.txt

Do not overload a positional token with different meanings based on whether it happens to look like a path, a mode, or a special keyword. If a command has distinct operations, subcommands make the grammar explicit: each operation gets its own arguments, and help output can describe those arguments without requiring users to infer which combination applies.

Reporting errors with useful context

An error message should identify what failed, include the value that caused the failure, and say what the program expected instead. A message such as invalid value forces users to reconstruct the context from the command they typed; argument --workers: expected a positive integer, got 0 does not.

argparse already adds useful context when a converter raises ArgumentTypeError. Use that exception for invalid individual values, and keep the converter’s message focused on the constraint rather than repeating the option name.

def positive_int(value):
    try:
        number = int(value)
    except ValueError:
        raise argparse.ArgumentTypeError(
            f"expected an integer, got {value!r}"
        )

    if number <= 0:
        raise argparse.ArgumentTypeError(
            f"expected a positive integer, got {number}"
        )

    return number

Some invalid requests involve more than one argument. Validate those combinations after parsing, when all values are available, and report the conflicting options explicitly. Calling parser.error() produces a concise diagnostic on standard error and exits with the conventional usage-error status of 2.

parser = argparse.ArgumentParser(prog="backup", allow_abbrev=False)
parser.add_argument("--source", required=True)
parser.add_argument("--destination", required=True)
parser.add_argument("--delete-source", action="store_true")

args = parser.parse_args()

if args.delete_source and args.source == args.destination:
    parser.error(
        "--delete-source cannot be used when --source and "
        "--destination refer to the same path"
    )

Do not report an error and then continue with a guessed value. If an invalid value makes the requested operation ambiguous or unsafe, stop before performing it. Conversely, do not reject harmless combinations merely because they are unusual; an error should describe a real constraint of the operation, not an assumption about what users probably intended.

Failures that occur while doing the work need the same care as parse-time failures. Catch expected operational exceptions at the layer that has enough context to explain them. For example, an OSError alone may say that a file could not be opened, but the user needs to know which input path was involved.

def read_input(path, parser):
    try:
        return path.read_bytes()
    except OSError as exc:
        parser.error(f"cannot read input {str(path)!r}: {exc}")

For a reusable program, it's often cleaner to keep argument parsing and work separate, then handle expected failures in main(). This lets library code raise exceptions normally while the command-line boundary turns them into concise diagnostics. Catch only errors the program can explain; a broad catch for Exception can hide programming defects behind a misleading user error.

import argparse
from pathlib import Path
import sys


def run(source, destination):
    data = source.read_bytes()
    destination.write_bytes(data)


def main(argv=None):
    parser = argparse.ArgumentParser(prog="copy-data", allow_abbrev=False)
    parser.add_argument("source", type=Path)
    parser.add_argument("destination", type=Path)
    args = parser.parse_args(argv)

    try:
        run(args.source, args.destination)
    except OSError as exc:
        print(
            f"{parser.prog}: I/O error while processing "
            f"{args.source!s} or {args.destination!s}: {exc}",
            file=sys.stderr,
        )
        return 1

    return 0

Include the original exception text when it adds actionable detail, such as Permission denied or No such file or directory. Avoid dumping a traceback for an ordinary failure caused by a missing file or an invalid option. A traceback is useful for diagnosing a defect; for a normal command-line error, it obscures the message the user needs. Preserve the exception as a cause if the same code is also used in contexts where a traceback is appropriate.

Do not discard the path, option, or operation while translating an exception. A message such as could not complete request is technically accurate but weak when the program knows it failed to write a particular destination. Add context without inventing certainty: if a multi-step operation could have failed on either of two paths, say so rather than claiming one was responsible.

Designing help output that answers real questions

Help output is part of the command-line interface, not a dump of parser internals. A user looking at --help needs to know what the command does, what each operand means, which options change its behavior, and what a typical invocation looks like. Give the parser a concise description, and give each argument a help string that explains its effect rather than merely restating its name.

import argparse

parser = argparse.ArgumentParser(
    prog="report",
    description="Generate a report from one or more input files.",
    allow_abbrev=False,
)
parser.add_argument(
    "inputs",
    metavar="FILE",
    nargs="+",
    help="input files to include in the report",
)
parser.add_argument(
    "-o", "--output",
    metavar="PATH",
    default="-",
    help="write the report to PATH; use - for standard output",
)
parser.add_argument(
    "--format",
    choices=("text", "json"),
    default="text",
    help="report format",
)

Names in the usage line should describe the kind of value expected. FILE and PATH are more useful than INPUTS and OUTPUT when they tell users what to supply; choices are often clearer when shown literally. Help text should also explain non-obvious conventions, such as whether a path may be -, whether an option can be repeated, or whether an omitted value triggers an automatic choice.

When defaults affect behavior, show them in the option descriptions. ArgumentDefaultsHelpFormatter can append defaults automatically, but explicit text is often preferable when the default needs explanation. Avoid displaying secrets or environment-dependent values as defaults; describe their source and precedence instead.

parser = argparse.ArgumentParser(
    prog="fetch",
    description="Download a resource and save it locally.",
    formatter_class=argparse.ArgumentDefaultsHelpFormatter,
    allow_abbrev=False,
)
parser.add_argument(
    "--timeout",
    type=float,
    default=15.0,
    metavar="SECONDS",
    help="maximum time to wait for a response",
)

For commands with distinct operations, put operation-specific guidance on the subcommand parser. That keeps the top-level help focused on choosing an operation, while each subcommand’s help can describe its own operands and options.

parser = argparse.ArgumentParser(prog="store", allow_abbrev=False)
commands = parser.add_subparsers(dest="command", required=True)

put = commands.add_parser(
    "put",
    help="save files in the store",
    description="Add files to the store under the specified name.",
    allow_abbrev=False,
)
put.add_argument("name", help="name to assign to the stored entry")
put.add_argument("files", nargs="+", metavar="FILE")

get = commands.add_parser(
    "get",
    help="retrieve an entry from the store",
    description="Write a stored entry to a file or standard output.",
    allow_abbrev=False,
)
get.add_argument("name", help="name of the entry to retrieve")
get.add_argument("--output", metavar="PATH", help="destination path")

Examples answer questions that option descriptions often cannot: how options combine, where operands go, and what a realistic invocation looks like. Include a small number of representative commands in the epilog. Keep them synchronized with the actual parser; an example that uses an obsolete option is worse than no example.

parser = argparse.ArgumentParser(
    prog="report",
    description="Generate reports from input files.",
    epilog=(
        "Examples:n"
        "  report sales.csv --format json --output sales.jsonn"
        "  report north.csv south.csv --output -n"
    ),
    formatter_class=argparse.RawDescriptionHelpFormatter,
    allow_abbrev=False,
)

Do not treat help output as a substitute for complete documentation. It should make common invocations and immediate choices understandable without overwhelming the terminal with implementation details. For options whose behavior depends on interactions, document the important relationship directly in the relevant help text—for example, that --overwrite permits replacing an existing destination.

Testing command-line behavior from end to end

Test a command-line program as a user runs it: start a fresh process with the intended arguments, then inspect its exit status, standard output, standard error, and effects on files or other external state. Calling the parsing function directly is useful for focused unit tests, but it does not verify the executable entry point, process exit behavior, or stream routing.

import subprocess
import sys
from pathlib import Path


SCRIPT = Path(__file__).resolve().parents[1] / "copy_data.py"


def run_cli(*args, cwd=None):
    return subprocess.run(
        [sys.executable, str(SCRIPT), *map(str, args)],
        cwd=cwd,
        text=True,
        capture_output=True,
        check=False,
    )

Using sys.executable runs the script with the same Python interpreter that runs the tests. Capturing both streams makes their intended roles testable: normal results belong on standard output, diagnostics belong on standard error. Keep check=False when testing failures, since check=True raises an exception before the test can inspect the program’s diagnostic.

def test_copy_data_writes_destination(tmp_path):
    source = tmp_path / "input.bin"
    destination = tmp_path / "output.bin"
    source.write_bytes(b"x00payload")

    result = run_cli(source, destination)

    assert result.returncode == 0
    assert result.stdout == ""
    assert result.stderr == ""
    assert destination.read_bytes() == b"x00payload"

Assert observable behavior, not the parser’s internal representation. A test of args.destination would not catch a broken entry point or a command that parses correctly and writes to the wrong path. Temporary directories keep filesystem tests isolated, and byte-oriented content catches accidental text decoding or newline conversion.

Exercise invalid invocations as well as successful ones. Check the conventional exit status and the part of the diagnostic that is part of the interface, but avoid matching an entire error string when harmless wording or formatting may vary between Python versions.

def test_missing_source_reports_path_without_traceback(tmp_path):
    source = tmp_path / "missing.bin"
    destination = tmp_path / "output.bin"

    result = run_cli(source, destination)

    assert result.returncode == 1
    assert result.stdout == ""
    assert "I/O error" in result.stderr
    assert str(source) in result.stderr
    assert "Traceback" not in result.stderr
    assert not destination.exists()


def test_unknown_option_is_a_usage_error(tmp_path):
    result = run_cli("--not-an-option", cwd=tmp_path)

    assert result.returncode == 2
    assert result.stdout == ""
    assert "unrecognized arguments" in result.stderr

Include help in the end-to-end suite. It should succeed without requiring operands, write to standard output, and expose the important options and usage information. This catches regressions where a required argument prevents users from asking for help, or where documentation strings disappear during a parser refactor.

def test_help_is_available_without_operands():
    result = run_cli("--help")

    assert result.returncode == 0
    assert "usage:" in result.stdout.lower()
    assert "source" in result.stdout
    assert "destination" in result.stdout
    assert result.stderr == ""

Control the process environment when it can affect behavior. Set variables explicitly for tests of environment-based defaults, and remove them when verifying that the command behaves independently of a developer’s shell. Supplying a temporary working directory also reveals accidental dependence on the directory from which the tests happen to run.

def test_timeout_uses_environment_default(tmp_path):
    environment = {
        **__import__("os").environ,
        "REPORT_TIMEOUT": "3",
    }

    result = subprocess.run(
        [sys.executable, str(SCRIPT), "--help"],
        cwd=tmp_path,
        env=environment,
        text=True,
        capture_output=True,
        check=False,
    )

    assert result.returncode == 0

A subprocess test should invoke the same public entry point users install or run. For a package, that may be a console-script executable or python -m package; test that route rather than a module function alone. Ensure the script or package has an ordinary executable guard so importing it in another context does not unexpectedly parse arguments or perform work.

if __name__ == "__main__":
    raise SystemExit(main())

Use direct calls to main(argv) for fast tests of argument combinations and internal decision logic, then keep a smaller set of subprocess tests for the contract visible outside the process. That division exercises the details cheaply while still checking that exit codes, streams, help, and side effects survive the complete command-line path.