At a glance
- Dollar-weighted accuracy, not field-level accuracy, is the measure that tracks payment error.
- Field-level accuracy and document-level accuracy are different measurements, and neither one directly tells you whether the payable amount is correct.
- A distinction table separates four accuracy metrics — field, document, dollar-weighted, and decision — by what each one counts and when to use it.
- A worked, synthetic example shows how one wrong field can change an invoice's payable amount even when field accuracy looks strong.
Two kinds of accuracy
Invoice extraction accuracy vs. financial accuracy comes down to two different questions. Field-level accuracy asks whether each field a system reads back — vendor, date, code, quantity, rate, amount — matches the correct value. Document-level accuracy, or exact match, asks whether every field on one invoice is correct at once. One invoice-extraction study reports field-level and document-level results as separate measurements[1]. The metric you choose changes the number you get.
Is 97% field accuracy good enough for invoices?
Not on its own: a single field-accuracy score doesn't say whether the fields that set the payable amount are the ones that are right. Because a payment decision typically depends on more than one field, a blended accuracy rate doesn't say, by itself, how often the payable amount will be correct. What matters is which fields sit inside that rate, and whether the ones that set the dollar amount are among them.
Illustrative example: field-level errors compound. If a system is independently correct on each field with probability p, and a workflow checks N fields on every invoice, the chance every one of those fields is correct is p raised to the Nth power. At p = 97% and N = 10 fields, that multiplies out to roughly 0.97 raised to the 10th power — under 74%. A single field-accuracy number doesn't say how many complete invoices, or complete dollar amounts, come out right.
The metric distinction table
| Metric | What it counts | Unit | Why it can mislead | When to use it |
|---|---|---|---|---|
| Field-level accuracy | Whether each individual field (vendor, date, code, quantity, rate, amount) matches the correct value, averaged across every field checked | % of fields correct | Gives every field equal weight, so a rare but expensive miss (a wrong unit count) counts the same as a common, low-stakes one (a misspelled address) | Comparing extraction engines on a like-for-like technical benchmark, or finding which field type needs more training data |
| Document-level exact match | Whether every field on one invoice is correct at once, so a single miss fails the whole document | % of documents fully correct | Falls faster than the field rate as invoices carry more fields, because the chance of at least one miss compounds across them | Judging whether a document can move through the workflow with no review at all |
| Dollar-weighted accuracy | Whether the payable amount computed from the extracted data matches the correct payable amount, weighted by the dollar value at stake on each invoice | % of dollars correctly payable | Requires someone to define the correct payable amount per invoice first, which means checking the arithmetic, not just the text | Sizing the real financial exposure of extraction errors, and deciding where automation is safe to run unreviewed |
| Decision accuracy | Whether the downstream pay, adjust, or deny decision made from the extracted data matches the decision a reviewer would have made | % of decisions correct | Mixes extraction error and rule error together, so a low score doesn't say which one to fix | Measuring the system the way reviewers actually experience it, once both extraction and rules are in place |
Field-level accuracy also isn't one number across a document's fields. On EPHOIE, not invoices, one model's per-field scores ranged from about 97% to 100% across ten field types checked on the same documents, with a mean over 99%[2]. These are EPHOIE scores, not invoice results, and they don't show what accuracy is achievable on invoices. On your own invoices, measure each field separately: the lowest-scoring field that feeds the payable amount sets a floor on how often that amount will be wrong.
Illustrative example: five invoices
| Invoice (synthetic) | Fields checked / correct | Field accuracy | Field the extraction missed | Payable: correct → extracted | Dollar difference |
|---|---|---|---|---|---|
| Invoice A | 20 / 19 | 95% | Units billed: read as 3, invoice states 1 | $180.00 → $540.00 | +$360.00 |
| Invoice B | 20 / 19 | 95% | Procedure code: read as Code X2, invoice states Code X1 (synthetic codes) | $110.00 → $170.00 | +$60.00 |
| Invoice C | 20 / 19 | 95% | Date of service: read as inside the coverage period; the actual date of service is one week outside it | $0.00 → $410.00 (outside coverage) | +$410.00 |
| Invoice D | 20 / 19 | 95% | Line rate: read as $85.00, invoice states $65.00 | $325.00 → $425.00 | +$100.00 |
| Invoice E | 20 / 19 | 95% | Line total: read as $1,200.00, invoice states $120.00 | $120.00 → $1,200.00 | +$1,080.00 |
| Batch of 5 (synthetic) | 100 / 95 | 95% | — | $735.00 → $2,745.00 | +$2,010.00 |
Every invoice in this batch scored 95% on field accuracy. None of the five had a correct payable amount, and the batch's dollar-weighted accuracy — the share of dollars computed correctly — was zero. Field accuracy and financial accuracy are not the same measurement, and a system can be strong on one while failing the other completely, on the same invoices, on the same day.
Weight errors by what they cost
Dollar-weighted accuracy asks a narrower, more useful question: does the payable amount computed from the extracted data match the payable amount a correct read would produce, weighted by the dollar value at stake on each invoice? Computing it means replaying a held-out sample of invoices you already know the right answer for, running extraction on them, and comparing the resulting payable amounts to the correct ones.
Weight the comparison by dollar amount, not by invoice count. A batch where many small, correct invoices sit next to a few invoices with large errors can still look nearly perfect by invoice count while carrying most of its financial risk in those few. Report the dollar variance the way you'd report a reconciliation break: as an amount, not just a rate.
- Quantity or units billed: multiplies directly into the line amount, so a miscount changes the payable dollar figure by that exact multiple.
- Rate or fee-schedule lookup: sets the price per unit, so a wrong lookup changes which price applies to the whole line.
- Procedure or line code (for example, a CPT code): often decides which fee-schedule entry applies in the first place, so an extraction error here can silently swap in the wrong price.
- Date of service: decides whether the line falls inside the coverage period at all. Miss this field and the payable amount isn't slightly off — it's the wrong decision.
- Vendor name and invoice number: matter for routing and duplicate detection, which is a different kind of financial risk than a wrong dollar amount on one line.
What to ask a vendor for
A vendor's aggregate accuracy figure, on its own, says very little about your financial exposure. Ask for the numbers that actually predict it.
- Ask for field-level accuracy broken out by field
Not one blended number. Ask which specific fields were measured, and how each one performed separately, especially the fields that set the payable amount.
- Ask for accuracy measured on your own invoices
A benchmark run on someone else's document set doesn't tell you how the system performs on your vendors, your layouts, and your fee schedules.
- Ask for dollar-weighted results, not just percentages
A vendor that reports field accuracy but not dollar-weighted accuracy hasn't measured the number that determines your financial exposure.
- Ask how confidence scores map to real accuracy
A confidence score is only useful once you know what accuracy it corresponds to at the threshold you'd actually use, on your own sample.
- Ask for the exception rate at that threshold
The share of invoices that would be sent to a person, not just the accuracy on the ones the system stays confident about.
Setting acceptance thresholds
Agree acceptance thresholds before go-live, and test them by replaying a held-out sample of your own historical invoices — never live production traffic — against invoices you already know the correct payable amount for.
No single metric covers every failure mode. Field accuracy catches extraction quality; document-level exact match tells you when a whole invoice can skip review; dollar-weighted accuracy sizes the financial risk; decision accuracy tells you what a reviewer will actually experience once your rules are layered on top.
- Field accuracy measured per field type, on a held-out sample of your own invoices, weighted by how often each field drives the payable amount.
- Document-level exact-match rate at whatever field set your workflow needs for straight-through processing, if any lines are meant to skip review entirely.
- Dollar-weighted accuracy, or dollar variance, computed by replaying the sample against payable amounts you already know are correct — not against the vendor's own benchmark set.
- Decision accuracy measured after your rules are applied, so extraction error can be told apart from rule error.
- A named threshold for each metric, agreed before go-live, along with the exception rate that threshold implies.
When this doesn't apply
- Invoice volume is low enough that full manual review already catches what matters.
- Every field that drives the payable amount already arrives as structured, validated data, so extraction error isn't the binding constraint.
- No one acts on the extracted data before a person reviews it, so a metric more sophisticated than a simple error count adds overhead without changing a decision.
- You can't yet define a correct payable amount for your own invoices to replay against, because the underlying fee schedules or coverage rules aren't documented in a checkable form. Fix that first.
Frequently asked questions
Why does per-field accuracy compound across a document?
Each field checked is a separate chance to get something wrong. If a system is right independently on each field, the odds that every field on one invoice is correct multiply together, so document-level correctness falls faster than the field rate alone suggests as more fields get checked on the same invoice.
Which invoice fields matter most financially?
Fields that feed directly into the payable calculation: quantity or units, the rate or fee-schedule lookup, the procedure or line code, and any date that determines coverage. A wrong vendor name or invoice number is a routing problem; a wrong quantity, code, or date changes the dollar amount or whether the line should be paid at all.
How do I measure dollar-weighted accuracy?
Replay a held-out sample of invoices you already know the correct payable amount for, compute the payable amount from what the system extracted, and compare the two. Weight the comparison by dollar amount rather than by invoice count, so a few large errors aren't hidden by many small, correct invoices.
How accurate are LLMs at invoice extraction?
There's no single trustworthy number: accuracy depends heavily on document quality, layout variety, and which fields are being checked, and a vendor's published benchmark wasn't run on your invoices. The figure worth acting on is dollar-weighted accuracy, measured on a held-out sample of your own historical invoices replayed against known-correct answers.
Are LLMs better than OCR for line items?
Neither approach is uniformly better; results depend on layout consistency, image quality, and how line items are labeled in training data. Rather than trust a general claim, test both approaches on the same held-out sample of your own line items and compare dollar-weighted accuracy, since that determines whether either is safe to rely on.
Related reading
- Our solution page on insurance invoice analysis covers the full line-item review workflow this measurement approach supports.
- For the document-extraction layer itself, see document intelligence for legal and compliance.
- If invoice review is one of several automation candidates, the 360° AI Blueprint ranks it against the rest, and AI agent implementation describes how a system like this is scoped, tested, and handed over.
Sources
- Yashwant et al., Invoice Information Extraction: Methods and Performance Evaluation (arXiv:2510.15727) (accessed 2026-09-28)
- Microsoft, LayoutLMv3 (GitHub README, microsoft/unilm) (accessed 2026-09-28)
More in Insurance operations
What AI Claims Automation Costs: A Cost and ROI Model for Carriers, TPAs and MGAs
AI claims automation cost depends on scope, not a price list. Get the cost drivers, an LAE baseline worksheet, and a payback method for carriers, TPAs and MGAs.
Insurance CompaniesBordereaux Automation for MGAs and Carriers: Ingestion, Validation and Reconciliation Checks
Bordereaux automation for MGAs and carriers: a validation-and-reconciliation check matrix with data source, rule, tolerance, owner and action for every check.
Insurance CompaniesInsurance Commission Reconciliation Automation: AMS Module, Point Tool or Custom AI Agent?
Choosing insurance commission reconciliation automation: a decision matrix for AMS module vs. point tool vs. custom agent, and a discrepancy routing table.
Insurance Companies