Missing Data in Document Automation: When to Escalate

At a glance

A confident guess is the most dangerous output a document-automation system can produce: it looks like a finding but isn't one.
A decision table below covers seven situations — from an absent field to a missing attachment — with the system's behavior, when a rule-based inference is allowed, and what a reviewer sees.
Inference is allowed only for two cases: a pre-approved format or unit rule for an out-of-range value, and a policy-named default or derived value for an absent optional field, both logged and labeled. Every other case — an absent required field, illegible text, conflicting values, an unrecognized type, low confidence, or a missing attachment — is abstained, requested from the sender, escalated, or rejected, never inferred.
Abstention rate is a metric to track by field and document type, not a number to publish once and forget.

The most dangerous output is a confident guess

How a system handles missing data in document automation decides whether reviewers can trust anything it outputs. A field that's genuinely absent from a document is a known, visible gap; a hallucinated value in that same field is invisible, because it looks exactly like a real finding. NIST's AI RMF makes the general point that deploying a system that is inaccurate, unreliable, or poorly generalized beyond its training data increases risk and reduces trustworthiness ^[1].

Three terms are worth separating early. A null value is an honest, explicit "not present": the system checked the whole document and the field isn't there. A hallucinated value is a plausible-looking figure the model produced without support in the document. Abstention is the system declining to output a value or a decision and flagging the field, with a reason, for a person to resolve — it's what turns a null from a blank the workflow quietly passes through into something a reviewer actually acts on.

A blank field prompts a person to check the document. A confident-looking guess doesn't — it can pass through review unread.
An extraction system that never abstains hasn't solved missing data; it has hidden it inside its output.
"We don't know" is a legitimate, useful answer from a document-automation system, not a failure state.

The abstain/escalate logic

The table below sets out seven situations a document workflow runs into, the system's default behavior for each, when a rule-based inference is ever allowed, and what a reviewer sees when a case reaches them. Treat it as a starting policy to adapt to your own fields and documents, not a fixed standard.

Abstain/escalate decision logic for missing or uncertain data
Situation	System behavior	Is inference ever allowed?	What the reviewer sees
Field absent from the document	Abstain and mark	Only a policy-named default for an optional field, or a value computed from two other high-confidence fields — logged and labeled as such. Never for a required field; it stays blank	The field marked "not present," with the page(s) checked for it — or the default/derived value, labeled as such
Text present but illegible (poor scan, handwriting the model can't parse)	Abstain and mark; request a legible copy from the sender	Never on the illegible content itself	The image region flagged "illegible," with a link to the original page and the outbound request logged
Two extracted values conflict for the same field	Escalate to reviewer	Never — a conflict is never resolved by picking one value automatically	Both values shown side by side, each with its own source location
Extracted value falls outside the expected range	Apply a documented rule if one fits (for example, a unit or date-format conversion), then flag; otherwise escalate to reviewer	Only the specific, pre-approved rule that fits this exact situation, applied the same way every time and logged. Never adjust a value just to fit the range	The raw extracted value, the expected range, and the rule applied, if any
Document type isn't recognized	Reject and route to manual triage	Never — an unrecognized type is never treated as a known one	The document flagged "type unrecognized," unprocessed
Extraction confidence falls below the field's threshold	Escalate to reviewer	Never — a low-confidence extraction is escalated, not replaced by a default or let stand as if it were high-confidence	The value with its confidence score and the threshold it fell under
Required attachment is missing from the packet	Request from sender	Never — the workflow doesn't proceed as if the attachment exists	The checklist item marked missing, with the outbound request logged

Selective-prediction research frames the System behavior column as a deliberate trade-off. A classifier can be set to reject instances as needed to hold to a risk level the operator chooses, trading some coverage for reliability ^[2]. A document workflow can set that same trade-off per field: a party name might tolerate less coverage than a due date, if a wrong party name costs more.

When inference is allowed, and when it never is

"Apply a documented rule" and "policy-named default" in the table above are deliberately narrow: each means a pre-approved, deterministic action, logged as a rule application, not a best guess dressed up as one — and each applies only when it genuinely fits. Where none fits, the system escalates instead of improvising.

Allowed: a fixed date-format correction or unit conversion applied to an out-of-range value, when that specific rule fits — applied the same way every time, logged as a rule application, not a finding.
Allowed: a named default for a field your policy already treats as optional, or a value computed from two other high-confidence fields — used only to fill a genuinely absent, non-required field, labeled as such.
Never allowed: treating illegible content as legible, picking between two conflicting extracted values, guessing at an unrecognized document's type, or adjusting a value just to force it inside an expected range.
Never allowed: letting a low-confidence extraction stand as if it were high-confidence, or replacing it with a default, because the alternative is an empty field.

Designing what the reviewer sees

An escalated case is only useful if the reviewer can resolve it without opening a second system. NIST's AI RMF treats human oversight as something to define in policy, with roles and responsibilities for human-AI configurations set out in advance ^[3]. We recommend going one step further: decide, for each row in the table above, exactly what a reviewer sees.

Flag raised
The system attaches a reason code — absent, illegible, conflicting, out-of-range, unrecognized, low-confidence, or missing attachment — to the case, not a generic "needs review."
Routed to the right queue
Cases route by reason and document type, so a conflicting-value case reaches someone who can read both source documents, not a general queue.
Reviewer resolves
The reviewer sees the original document location, every candidate value with its source, and the confidence score or rule that triggered the flag — enough to decide without opening a second system.
Resolution logged
The reviewer's decision, and a reason if they overrode a system suggestion, is recorded against the case before it can close.
Case closes or escalates further
A case the reviewer resolves closes here. A case they can't resolve — it needs the sender's input, or a second sign-off — moves up, with the same record attached.

Measuring abstention

Abstention rate is the share of fields, or documents, the system marks unresolved or routes to a reviewer rather than resolving on its own. The table above assigns a reason code to every abstention, so the more useful view breaks the rate down by code and document type: a spike in one code points to a specific cause; an overall number just looks like noise.

A rising abstention rate on a previously reliable field is a signal worth investigating before it's a problem a reviewer notices on their own. A falling rate is worth checking too — it can mean the system got better, or that a threshold was loosened and low-confidence values are now passing through unflagged.

Abstention rate by field: the share of instances of that field marked unresolved or escalated, tracked over time.
Resolution outcome on escalated cases: how often a reviewer confirms the flagged concern versus finds nothing wrong — a high false-flag rate wastes reviewer time; a low one risks a threshold set too loose.
Reason-code mix: which of the seven situations above is driving abstentions, so a spike has an obvious first place to look.
Abstention rate by document source or sender: a rising rate from one sender can point to a document-quality problem upstream, not a model problem.
Sampled review of auto-resolved fields: how often a value the system didn't flag turns out wrong when a reviewer checks anyway — the silent-error side of the trade-off, and the way to see whether the system abstains too little.

Reducing missing data at the source

Every abstention the table above produces is also a data point about where documents arrive incomplete, illegible, or in a format the system doesn't expect. Treat that as a checklist for the intake process itself, not only a downstream queue to staff.

Publish a required-document checklist to senders before they submit, naming exactly which attachments and fields are expected.
Request structured formats where you control the request — a fillable form beats a photographed printout for the fields it covers.
Feed the reason-code mix back to whoever owns the intake relationship, so a recurring "illegible" flag becomes a conversation, not a permanent queue item.
Review a system like document review with human approval for how abstention, playbook comparison, and reviewer sign-off fit together end to end.

In lending specifically, where a loan file depends on many supporting documents from multiple senders, a loan underwriting document workflow needs the same abstain/escalate logic built in from the start, not bolted on after a bad file gets through.

When this doesn't apply

Your documents already arrive in a structured, validated format (an EDI feed, a form with required-field validation) where missing data is caught before extraction even runs.
Your volume is low enough that a person already reads every document; adding an abstention layer adds process without reducing review work.
The gap isn't a data problem at all — it's a business decision about whether to proceed without a document, which stays with a person regardless of what the system flags.

Frequently asked questions

Should the model return null or guess?

It should return "not present" only once the system has confirmed the field is genuinely absent, and abstain — flagging the field for a person — otherwise, rather than guess. A blank or flagged field prompts a person to check it; a confident guess can pass through unreviewed, looking like a real finding.

How do you stop it hallucinating values?

Treat every extracted field as a lookup, not a prediction: the system scores confidence per field, and anything below threshold triggers an abstain/escalate reason code instead of a best guess. A value only reaches a reviewer carrying that reason code, never silently as if it were certain.

"Not present" vs "not found"?

"Not present" means the system checked the document and the field genuinely isn't there. "Not found" is ambiguous and can mean the extraction step failed to locate a value that may exist. Distinguishing them matters: one tells a reviewer the document is incomplete; the other tells them to check the extraction itself.

When should missing data trigger a request vs a review?

Missing data triggers a request to the sender when the document itself needs replacing or completing — a missing attachment, an unreadable scan. It triggers escalation to a reviewer when the documents on hand are fine but the system can't resolve a conflict or exception on its own, such as two values that disagree.

How do you measure abstention rate?

Abstention rate is the share of fields, or documents, marked unresolved or routed to a reviewer rather than resolved automatically, tracked by field and by document type so a change in one area doesn't hide inside an overall average. A rising rate on a previously reliable field is worth investigating before it appears anywhere else.

Document review with human approval covers the full workflow this abstain/escalate logic sits inside: extraction, playbook comparison, and reviewer sign-off.
Loan underwriting document workflow applies the same logic in a lending-specific document workflow.
Underwriting document workflow audit trails covers what to record once a case is resolved, so the resolution itself can be reconstructed later.
AI agent implementation describes how an abstain/escalate policy like this one is scoped, tested, and handed over.

Sources

[1] — AI RMF 1.0 (NIST AI 100-1), Section 3.1, "Valid and Reliable" (accessed 2026-09-29)
[2] — Geifman & El-Yaniv, "Selective Classification for Deep Neural Networks" (2017), Abstract (accessed 2026-09-29)
[3] — AI RMF 1.0 (NIST AI 100-1), Section 5, GOVERN function, Table 1, Subcategory GOVERN 3.2 (accessed 2026-09-29)

340B Program Documentation: Where AI Agents Reduce Audit Risk (Not Replace Compliance Officers)

How AI agents can assemble 340B program documentation, from split-billing records to contract pharmacy files, while your compliance officer keeps each decision.

Pharmacy Chains

Oct 29, 2026·9 min read

Document intelligence

Chargeback and Returns Dispute Document Automation for E-Commerce Brands

Chargeback and returns dispute automation for e-commerce: a step-by-step dispute-document workflow and evidence table, with submission kept with your staff.

Ecommerce Operations

Oct 29, 2026·11 min read