Structured outputs: getting AI to fill the form right

Most of the business value of language models has nothing to do with fluent paragraphs. It is an invoice turned into fourteen correct fields, a claim routed to the right queue, a contract mapped against your standard clauses. The question that decides whether any of this ships is LLM structured output reliability: can the system fill in the form right, every time, at volume. Writing nice prose was never the hard part. "Every time" is, and it is a property of the system around the model, not of the model itself.

That distinction is the whole article. Teams that treat reliability as a model-selection problem stall at a demo that is impressive and unshippable. Teams that treat it as a pipeline problem ship.

Why a good model still gets the form wrong

Models are probabilistic, documents are ambiguous, and the real world mails you edge cases daily: the scanned invoice with a handwritten correction, the date that reads as March 12 in one convention and December 3 in another, the total in a currency the header does not mention. Even a strong extraction setup will be wrong some percentage of the time, and the arithmetic of volume is unforgiving: a system that is right 95 percent of the time commits five hundred errors a month at ten thousand documents.

The dangerous errors are the silent ones. A missing field announces itself. A plausible wrong value, 47 instead of 47,000, the right amount in the wrong field, sails through and surfaces weeks later in accounting. Reliability engineering is mostly a campaign against silent plausibility.

The structured output reliability stack

Production systems layer five mechanisms, each catching what the previous one missed.

Schema constraints come first. Structured output modes and function calling force the model to emit the right shape: valid JSON, correct types, values from your enums. This eliminates the parsing failures that plagued early systems, and it solves format only. The model can now produce a perfectly valid wrong answer, which is progress, because valid answers can be checked.

Validation rules do the checking. Deterministic code, not another model: totals that must add up, dates that must fall in plausible ranges, supplier IDs that must exist in your master data, cross-field consistency. Cheap code catching expensive errors, and every rule you write converts a silent failure into a visible one.

Retries with feedback turn many failures into successes. A validation error is not an ending; feed it back to the model with the failing field named, or escalate the retry to a stronger model with more context. A structured second pass resolves a good share of first-pass failures, and whatever survives two passes gets flagged, never silently accepted.

Confidence scoring decides what happens next. The model’s own declared confidence is a weak signal. Better ones: agreement between two independent extraction passes, whether a value can be traced to a specific span in the source document, and the validation outcomes themselves. Composite score, three routes: auto-accept the confident, queue the doubtful for review, reject the hopeless back to the sender.

Human review is the final layer, and it deserves design rather than apology. A well-built review queue, showing the document and the extraction side by side with the doubtful fields highlighted, lets one person verify in seconds what took minutes to type. The queue is what lets you ship safely at less-than-perfect accuracy, and every correction made in it becomes test data.

Measure it per field, or you are guessing

Reliability claims need a denominator. Build an eval set from your real documents, the ugly ones included, and score accuracy per field, not per document: an aggregate number hides the fact that everything works except the one field finance actually depends on. Rerun the set on every prompt, model or rule change, the regression discipline described in evals: the unglamorous work, and keep monitoring in production, because your document mix will drift and the system will not announce it.

This loop, corrections feeding evals feeding changes, is what separates the document processing systems that hold up from the demos. The free-text cousin of this problem, models inventing facts in prose, yields to the same architectural mindset: hallucinations are a system design problem too.

The machinery in this article, schemas, validators, retries, queues, is unglamorous on purpose: it is the part of the build that decides whether the system is still trusted a year in. It is also most of the engineering in the systems we build, which is why our proposals talk about review queues when the deck you saw elsewhere talked about magic.