LEANDRO CALADO FERREIRA

Python · Tool contracts · Failure recovery

Forthcoming book · Free lab available now

AI Agent
Reliability Engineering

A Hands-On Python Guide to Testing, Evaluating, Debugging, and Recovering Production Agents

By Leandro Calado Ferreira

The tool commits an action. Its response disappears. Your agent retries.
Does the action happen twice?

Run the free incident labRead the opening chapters

One failure. A correction you can inspect.

Reproduce a duplicate synthetic credit after a lost tool response. Then use a stable operation identity and a local transaction to preserve one authorized effect.

Controlled scenarioBroken retryCorrected runner
Effects after a lost response21
Total synthetic credit2,000 cents1,000 cents

Results from a constructed deterministic incident, not a benchmark of model quality or production traffic.

Free · No account · No API key

Run the lab in Python.

Python 3.11+ with SQLite support. No pip packages or model calls. The archive includes source, MIT license, instructions, tests and the four articles.

Download the Python labView source on GitHub
cd agent-reliability-lab
python -m unittest -v
python lab.py demo
python lab.py evaluate

20 regression tests

Checked on Python 3.11, 3.12 and 3.13. Covers response loss, restart, concurrent delivery, changed payloads, authorization and attempt budgets.

Inspect the recorded test run

Explicit guarantee boundaries

The synthetic effect and receipt share one SQLite transaction. Remote APIs still need their own idempotency and reconciliation strategy.

From demonstration to verifiable behavior.

The book is being developed for readers who already know Python and APIs and need concrete ways to test what an agent is allowed to do, what actually happened, and how interrupted work resumes.

Test the execution boundary

Tool schemas, trusted task facts, authorization, side effects and evaluation cases.

Diagnose and recover

Traces, uncertain writes, operation identities, duplicate delivery and restart behavior.

The complete edition is in preparation. The opening chapters are a working sample. Real-model, multi-step, retrieval, MCP and cost/latency laboratories are still being completed.

Read the sample onlineDownload the sample

The book is not yet available for purchase on Amazon. There is no preorder or payment required for these resources.

Learn by reproducing failures

Four practical starting points.