How small errors compound across an agent pipeline

Under the equal, independent-stage assumptions, take the eighth root of the overall target. That gives approximately 99.36% per stage for 95% overall, and 99.874% for 99% overall. Unequal stage quality calls for a different allocation of effort.

Prompt changes may improve a stage, but they are one option alongside data preparation, rules, model selection, and workflow changes. Compare each candidate against a baseline using the same evaluation criteria.

Remove unnecessary steps. Fewer handoffs can reduce failure opportunities, but combining stages also changes their difficulty. Re-evaluate the combined workflow rather than assuming that fewer calls means higher accuracy.

Use deterministic rules where the specification is clear. A regex, lookup, or schema check is repeatable, but it can still encode a wrong rule, use stale data, or reject a valid case. Test the rules and retain them in the end-to-end error budget.

Verify against a separate source or checking path. A second model pass can share the first model's errors, so agreement alone is not proof. Measure both missed errors and unnecessary escalations.

The pattern this produces is consistent enough to be a rule. Deterministic passes run first and narrow the space — tiered matching, closed vocabularies, regex extraction, schema constraints. Whatever survives that is handed to the model, which now faces a genuinely ambiguous residual rather than the whole input.

This division can reduce model calls and focus review on unresolved cases. It only helps if the deterministic path is accurate for the input population, so compare its accepted matches and exceptions against reviewed examples.

Choose rules and models by task. Extraction, classification, and exception handling can have different accuracy and cost requirements. Test less expensive options on the same inputs before changing a production route.

Self-review can catch mistakes, but a model's stated confidence is not an independently measured probability of correctness. Evaluate that signal against reviewed examples before relying on it.

Where possible, re-check values against source text, a trusted record, or explicit constraints. A model-based verifier can add coverage, but its errors may be correlated with the generator. Return a check result, its evidence, and any unresolved ambiguity.

Calibrated scores can help prioritize review. Pair them with source references and a clear explanation of how the value was established so a reviewer can decide what to do next.

Replacing the scalar with a deterministic ladder makes review tractable. Each tier states how the value was established, and each carries a prescribed action.

The costly bug is not the one that crashes. It is output that is quietly incomplete and looks finished — a field that came back empty because a parse failed three stages up, sitting in a report that renders perfectly.

Expose failures and missing inputs in the output people review. Explain why a value is absent and whether the team should retry, supply information, or resolve an exception. Do not let an extraction failure look like a valid blank.

Reproducibility needs more than a temperature setting. Low temperature can reduce variation, but it does not guarantee identical model outputs. Record inputs, prompts, model versions, parameters, tool responses, and available backend identifiers. Repeat evaluations and investigate material variation.

None of the above is verifiable without an eval corpus, and the corpus is where the discipline actually lives.

Use development examples for iteration and reserve a separate holdout for evaluation. Repeated tuning against a holdout can bias the result; inspecting one failure does not make every remaining result worthless. Track exposure, refresh the set, and define regression limits before a release decision.

Include representative formats, difficult exceptions, and reviewed expected outcomes. Report sample size, uncertainty, and error severity alongside the aggregate score. A small clean set cannot establish how the workflow will behave across the business.