JSON Output

When you build software around an LLM, free-flowing prose is a problem — code can't reliably parse a paragraph. JSON output solves this: you ask the model to return its answer as structured JSON so your program can read specific fields directly. It's the bridge between an LLM's language ability and a real application.

💡 In one line: JSON output makes the model return machine-readable structured data instead of prose, so your code can parse and use it reliably.

Why JSON Output?

  • Machine-readable — parse fields directly instead of scraping text.
  • Integrates with apps, databases, and APIs.
  • Predictable structure — the same keys every time.
  • Enables automation — feed the output straight into other systems.

How to Get JSON

  1. Instruct clearly — "Respond only with valid JSON. No extra text."
  2. Give a schema or example — show the exact keys and types you want.
  3. Prefill the assistant with { — nudges it straight into JSON (see Assistant Prompt).
  4. Use JSON mode / structured outputs — API features that guarantee valid JSON.

Defining a Schema

Tell the model the exact shape you expect — keys, types, and nesting:

json
{
  "name": "string",
  "age": "number",
  "skills": ["string"],
  "address": { "city": "string", "country": "string" }
}

A clear schema dramatically improves consistency.

JSON Mode & Structured Outputs

Most modern LLM APIs offer a JSON mode or structured output feature. With schema-constrained decoding, the model is forced to produce output that is valid JSON (and often to match your schema exactly). This is the most reliable approach — use it when available, instead of hoping the instructions hold.

Common Problems & Fixes

ProblemFix
Extra text around the JSON"Respond with JSON only"; prefill {
Wrapped in ```json fencesStrip fences, or use JSON mode
Invalid / truncated JSONRaise max tokens; validate + retry
Missing or wrong fieldsProvide a schema; validate against it

The Reliable-JSON Loop

In production, you validate the output and retry if needed.

Whiteboard
Whiteboard diagram

Code Example

Best Practices
 
     
  • Specify a schema and give an example.
    •  
  • Prefer a JSON mode / structured output feature when available.
    •  
  • Always validate the parsed result before trusting it.
    •  
  • Handle failures with a repair or retry step.
    •  
  • Keep max_tokens high enough to avoid truncated JSON.
    •  
 
Summary
 
     
  • JSON output returns structured, machine-readable data instead of prose.
    •  
  • Get it via clear instructions, a schema/example, prefilling {, or a JSON mode.
    •  
  • Schema-constrained decoding is the most reliable — it guarantees valid JSON.
    •  
  • Watch for extra text, fences, invalid JSON, and missing fields.
    •  
  • Always validate and be ready to retry — the JSON is the bridge to your application. EOF echo "created"