Errors, Testing, and Evidence-Driven Debugging

Good backend code distinguishes expected domain failures from programming defects and unavailable dependencies. Tests preserve intended behavior; debugging compares a prediction with observable evidence until the smallest false assumption is found. Catch errors only where you can add context, translate them, retry safely, compensate, or complete cleanup.

What will you be able to do?

define meaningful application exceptions;
preserve causes with exception chaining;
write focused positive and negative pytest tests;
reduce a bug to a reproducible failing example.

How should failures be modeled?

python

class TicketError(Exception):
    """Base class for expected ticket failures."""


class TicketNotFound(TicketError):
    pass


class TicketAlreadyResolved(TicketError):
    pass


def resolve_ticket(ticket: dict[str, str]) -> dict[str, str]:
    if ticket["status"] == "resolved":
        raise TicketAlreadyResolved(ticket["id"])
    return {**ticket, "status": "resolved"}

Test both paths in tests/test_tickets.py:

python

import pytest

from supportdesk_ai.tickets import TicketAlreadyResolved, resolve_ticket


def test_resolve_open_ticket() -> None:
    result = resolve_ticket({"id": "t_1", "status": "open"})
    assert result["status"] == "resolved"


def test_reject_resolving_twice() -> None:
    with pytest.raises(TicketAlreadyResolved):
        resolve_ticket({"id": "t_1", "status": "resolved"})

Run a narrow test while debugging, then the suite:

bash

uv run pytest tests/test_tickets.py -q
uv run pytest -q

How do you debug systematically?

Record the exact input, environment, observed output, and trace ID.
Reproduce without customer secrets.
Identify the first layer where evidence differs from expectation.
Reduce to the smallest failing test.
Fix the cause, not the final symptom.
Add a regression test and run broader checks.

If a low-level exception needs domain context, preserve its cause:

python

try:
    row = repository.get(ticket_id)
except DatabaseUnavailable as error:
    raise TicketReadUnavailable(ticket_id) from error

Why does this matter in AI-native systems?

AI failures include provider timeout, refusal, invalid structure, safety block, exhausted budget, missing evidence, and low evaluation confidence. Do not collapse them into “AI error.” Tests should use deterministic fake gateways for business paths and a small, separately managed set of provider contract tests.

Common mistakes

except Exception: pass destroys evidence.
Tests assert implementation calls instead of observable behavior.
Live model calls make the main test suite slow, expensive, and flaky.
Production data is copied into bug reports without redaction.
A retry test ignores duplicate side effects.

AI Pair-Programmer Prompt

text

Help me debug from evidence. Ask for expected behavior, minimal input, actual output,
stack trace, environment, and last known good version. Build a hypothesis table and
one discriminating experiment at a time. Redact secrets and customer data. End with
a regression test and verification commands, not confidence language.

Exercise

Add a repository fake and test: missing ticket, database unavailable, valid resolution, and duplicate resolution. Ensure each maps to a distinct application outcome.

Acceptance criteria: all branches are deterministic, no network is used, and removing any production branch makes a corresponding test fail.

Knowledge check

When should an exception be caught?
Why preserve an exception cause?
Why avoid live model calls in most tests?

Answers

Where code can meaningfully handle, translate, enrich, retry, compensate, or clean up.
It retains diagnostic evidence while adding higher-level meaning.
They introduce cost, latency, nondeterminism, quotas, and external failure.

Completion checklist

Expected failure categories are named.
Positive and negative tests pass.
My debugging notes contain evidence, not guesses.

Errors, Testing, and Evidence-Driven Debugging

What will you be able to do?

How should failures be modeled?

How do you debug systematically?

Why does this matter in AI-native systems?

Common mistakes

AI Pair-Programmer Prompt

Exercise

Knowledge check

Answers

Completion checklist

Primary references

Errors, Testing, and Evidence-Driven Debugging

What will you be able to do?

How should failures be modeled?

How do you debug systematically?

Why does this matter in AI-native systems?

Common mistakes

AI Pair-Programmer Prompt

Exercise

Knowledge check

Answers

Completion checklist

Primary references