Start typing to search this publication.
techcirkle
Open menu

Subscribe to techcirkle

Get new posts delivered straight to your inbox.

Exception Handling Is the Product — Designing Automation Fallout Paths

Automating the happy path can make the remaining work harder. Here is the architecture that makes fallout cases cheaper, not more expensive.

techcirkle
6 min read
Cover image for Exception Handling Is the Product — Designing Automation Fallout Paths

There is a way to automate 60% of a process and end up with more human effort than you started with. It is not rare and it is not the result of bad engineering. It happens by default whenever the fallout path is not designed.

Engineer reviewing an automation workflow diagram
Engineer reviewing an automation workflow diagram

The mechanism is simple. Before automation, a person handled a case from beginning to end. They knew what had been done because they did it. After automation, a person picks up a case that has been partially processed by a system, and their first task is archaeology: what did it do, where did it stop, what state is this in, and can I trust what it already decided?

That archaeology is pure added cost. It did not exist before. And on a hard case — which is exactly the kind that falls out — it can exceed the cost of the original manual handling.

What a fallout artefact should be

The design principle: automation should hand a human a resumable case, never an error.

An error says something went wrong. A resumable case says here is what has been established, here is what remains, here is what I was uncertain about, and here is the control to continue. Those are entirely different artefacts, and only one of them makes the human faster than they were before.

Concretely, when a case exits the automated path it should carry:

{
  "case_id": "case_44821",
  "status": "needs_human",
  "reason": "confidence_below_threshold",
  "reason_detail": "supplier_match confidence 0.61 (threshold 0.85)",
  "completed_steps": [
    { "step": "extract",  "outcome": "ok", "confidence": 0.97,
      "output_ref": "extract_9912" },
    { "step": "validate", "outcome": "ok", "confidence": 1.0 }
  ],
  "blocked_step": {
    "step": "match_supplier",
    "candidates": [
      { "id": "sup_221", "score": 0.61, "why": "name fuzzy match, VAT differs" },
      { "id": "sup_884", "score": 0.44, "why": "VAT match, name differs" }
    ]
  },
  "remaining_steps": ["post_to_erp", "notify_requester"],
  "resume_token": "rt_7f2a1c"
}

The candidates array is the part that turns a cost into a saving. The human is not asked to redo the matching — they are shown the work, the alternatives, and the reason for uncertainty, and asked to make one decision. That is a ten-second task instead of a five-minute one.

The resume_token matters equally: after the human decides, the workflow continues from that step. It does not restart, and it does not hand the remaining steps back to the person.

Confidence thresholds belong in configuration

A recurring pattern: a step starts producing worse output because an upstream data source changed. Operations notices before engineering does. And the only way to make the system more cautious is a code change and a deployment.

Confidence thresholds should be operational controls:

steps:
  match_supplier:
    auto_threshold: 0.85      # act automatically above this
    reject_threshold: 0.30    # below this, do not even suggest
    escalate_to: ap_review_queue

Operations can raise auto_threshold to 0.95 on Monday morning when something looks wrong, without waiting for a release. Every threshold change is logged with who and why, which also produces a useful record of where the system is struggling.

Robotic automation system in a warehouse
Robotic automation system in a warehouse

Categorise exceptions, then automate the categories

Every exception should carry a machine-readable reason code, and those codes should be counted.

Then read the counts weekly. What you will find is that exceptions are not uniformly distributed — a handful of categories account for most of the volume. Each of those categories is a specification for the next increment of automation, written by production rather than by a requirements workshop.

This is what makes the difference between a program that improves and one that plateaus. Automation number one covers the main path. The exception distribution then tells you precisely what automation 1.1 should handle, and you are working on the highest-frequency real problem rather than the next item on a roadmap written before you had data.

Capture the corrections

The single highest-leverage design decision in the whole architecture, and the one most often missed.

When a human overrides an automated decision, that is a labelled example produced by a domain expert at the exact point where your system was wrong. It is the most valuable training data you will ever have access to, and it arrives free.

Most systems throw it away. The override is applied to the case, logged as an operational event, and that is the end of it.

Capture it properly instead:

{
  "type": "human_override",
  "step": "match_supplier",
  "system_output": { "supplier_id": "sup_221", "confidence": 0.61 },
  "human_output":  { "supplier_id": "sup_884" },
  "human_reason":  "VAT is authoritative here; name changed after acquisition",
  "actor": "u_1182",
  "case_ref": "case_44821"
}

The human_reason field is worth the friction of asking for it. It converts a correction into an explanation, and explanations are what let you fix the rule rather than just the case. In the example above, "VAT is authoritative" is a business rule that should be promoted into Tier 1 logic — which removes the entire exception category rather than improving a model's odds on it.

A system with this feedback loop improves every month. One without it is exactly as good on its last day as its first.

The measurement that keeps you honest

Report exception rate alongside handling time, always. And measure end-to-end cost per case including fallout, not cost per automated case.

If you only measure the automated path, a system that automates 60% of cases and doubles the cost of the other 40% looks like a success. It is not. The single metric that cannot be gamed is total human minutes per hundred cases, before and after.

Full guide — automation tiers, agent boundaries, process discovery, ROI framing and a 12-week rollout blueprint: Business Process Automation in 2026.

TechCirkle: agentic workflow development | AI development services

More from TechCirkle

Frequently Asked Questions

What is the difference between an error and a resumable case?

An error reports that something went wrong. A resumable case presents what has been established, what remains, what the system was uncertain about, and a control to continue from that point. Only the second makes the human faster than they were before automation existed.

Why should confidence thresholds be configurable?

Because output quality degrades for reasons operations notices before engineering does — an upstream data change, a new document format, a supplier behaving differently. If tightening a threshold requires a deployment, the system stays wrong for as long as the release cycle takes.

How should exceptions be categorised?

With machine-readable reason codes attached at the point of exit, counted and reviewed weekly. The distribution is never uniform; a few categories dominate. Each dominant category is a specification for the next increment of automation, derived from production rather than from a requirements workshop.

What should be captured when a human overrides a decision?

The system's output and confidence, the human's output, and a short reason. The reason field is the valuable part — it converts a correction into an explanation, which often reveals a business rule that should be promoted into deterministic logic, removing the whole exception category rather than improving model odds.

How do I measure whether automation actually helped?

Total human minutes per hundred cases, before and after, including fallout handling. Measuring only the automated path lets a system that automates 60% of cases and doubles the cost of the remaining 40% look like a success.

Should every exception go to the same queue?

No. Route by reason code to the people equipped to resolve that category. A supplier matching ambiguity and a document extraction failure need different expertise, and a single undifferentiated queue means every reviewer context-switches on every case.

What if the automation is uncertain about a step it already completed?

Record the confidence per step, not just for the case. A downstream reviewer needs to know whether to trust the extraction as well as whether to resolve the block. Per-step confidence in the fallout artefact is what makes that possible.

Subscribe to techcirkle