Summary: Getting a model to return clean JSON stopped being the hard problem. Every major API now enforces schemas at generation time, and the failure mode moved: the JSON parses, and a value inside it is wrong. This guide covers what structured outputs solve, what they cannot solve, and the validation architecture that regulated teams put around them. It extends our earlier deep-dive on prompt formats, which remains the reference for format benchmarks.
What structure solves
Three problems disappear when you enforce schemas at the API layer instead of asking for formats in the prompt:
- Malformed output. Constrained decoding makes invalid JSON effectively impossible, versus the few-percent failure rate of prompt-level requests.
- Downstream parsing code. Typed fields mean your pipeline consumes model output like any API response, with no regex archaeology.
- Scope creep in answers. A tight schema is itself an instruction: a field named
net_operating_income_usdwith a number type pins the model to a task in a way prose instructions do not.
Every serious provider supports this now, whichever model family you run: the GPT-5.6 tiers, the Claude 5 family, Gemini, or open-weight models like GLM 5.2 served through inference layers that implement constrained decoding.
What structure cannot solve
The JSON parses. The number is wrong. That is the failure mode that matters in 2026, and no schema prevents it: structure guarantees shape, not truth.
For enterprise workflows the gap shows up three ways:
- Fabricated values. A syntactically perfect field containing a number that appears nowhere in the source document.
- Right value, wrong provenance. The model found a revenue figure, from the wrong year’s statement in a 200-page package.
- Silent unit and sign errors. Thousands versus millions, positive versus negative, monthly versus annual.
If a human reviews every output, these are annoyances. If the output feeds a decision, they are incidents.
The architecture around the schema
Production systems in regulated industries wrap structured outputs in three layers:
Evidence pointers. Every extracted value carries a link to its source: the document, the page, the character offset. A value without provenance is unverifiable by construction; a value with provenance can be checked by machine or human in seconds. This is core to how MightyBot’s document intelligence works, and it converts extraction from trust-me to show-me.
Deterministic validation. Cross-field checks (do the line items sum to the total), range checks, format checks, and reconciliation across sources (does the invoice match the budget line) run as code, not as another model call. Zero tokens, zero ambiguity.
Confidence routing. Extractions the system cannot verify against evidence route to human review with the source alongside; verified values proceed. The schema defines the data; the routing defines the trust.
One extraction pattern is worth adopting regardless of platform: constrain each call’s scope. One field group per call, with evidence required, outperforms one giant document-wide schema both in accuracy and in debuggability.
From structured outputs to structured execution
Structured outputs discipline a single model call. The same idea scaled to a whole workflow is compiled execution: the workflow’s steps, dependencies, and validation compile into a fixed plan; deterministic checks run as code; model calls are constrained, schema-bound, and evidence-linked. Structure at the call level makes outputs parseable. Structure at the execution level is what makes a workflow auditable, which is the bar regulated work has to clear.
For the token-cost side of constrained versus free-running calls, the measured cost study quantifies what unconstrained iteration costs; for policy-bound workflows on top of these techniques, start with the policy engine.
