Structured outputs: getting reliable JSON out of a text model

The moment an LLM's output feeds another system instead of a human, free-form text becomes a liability. You need JSON that parses every time, with the right fields and types. Asking politely for JSON in the prompt gets you 95% there — and that last 5% is exactly the failures that page you. Structured outputs close the gap.

From 'please return JSON' to a guarantee

The naive approach is to describe the format in the prompt and hope. It mostly works, until the model adds a friendly preamble, trails a comma, or invents a field. Modern APIs offer a stronger contract: pass a JSON Schema and the provider constrains generation so the output is valid against it — not by asking, but by enforcing.

• JSON mode — guarantees syntactically valid JSON, but not any particular shape.
• Schema-enforced output — you supply a schema; the response is guaranteed to match its fields and types.
• Tool / function calling — the model returns arguments that fit your function's schema, which is structured output by another name.

const res = await client.chat.completions.create({
  model: 'gpt-4o',
  messages,
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'extraction',
      strict: true,                       // actually enforce the schema
      schema: {
        type: 'object',
        properties: {
          name:    { type: 'string' },
          email:   { type: 'string' },
          urgency: { type: 'string', enum: ['low', 'medium', 'high'] },
        },
        required: ['name', 'email', 'urgency'],
        additionalProperties: false,      // required by strict mode
      },
    },
  },
});
// res parses cleanly, every time

Still validate

Enforced schemas fix the shape, not the meaning — the model can still put a wrong-but-valid value in a field. Keep validating the parsed object against your business rules, and decide what happens on a bad-but-valid answer. Structure removes the parsing failures so you can focus on the ones that actually require judgement.

A model that returns prose is a chatbot. A model that returns a schema you can trust is an API — and APIs are what you build on.