Output Parsing

Output parsing is the process of converting an AI model's free-form text response into structured data that your application can reliably use. It is the essential step between "the AI said something useful" and "my software can act on what the AI said."

Why parsing matters

AI models generate text. Software systems need structured data — JSON objects, database records, typed variables. Without parsing, you cannot reliably connect AI to the rest of your technology stack.

Consider an example: you ask an AI to extract contact information from an email. The AI responds: "The sender is John Smith, his email is john@example.com, and he is the CTO of Acme Corp." That is useful to a human but useless to a database INSERT statement. Output parsing converts this into a structured object your code can process.

Parsing approaches

• JSON mode: Ask the model to respond in JSON format. Most reliable but requires provider support.
• Structured output / function calling: Define a schema and let the model fill it. The most robust approach.
• Regex extraction: Use regular expressions to pull specific patterns from the text. Brittle but simple.
• Prompt-based formatting: Ask the model to use a specific format (XML tags, markdown tables, delimiters) and parse that format.
• Secondary LLM call: Send the model's output to a second, cheaper model specifically for extraction. Adds latency but improves reliability.

Common challenges

• Inconsistent formatting: The model may not always follow your requested format exactly.
• Missing fields: The model might omit a field you expected, requiring defaults or error handling.
• Extra content: The model may include explanatory text around the structured data.
• Type mismatches: Numbers might come as strings, dates in unexpected formats.

Best practices

• Use structured output or JSON mode when available — they eliminate most parsing issues.
• Define clear schemas that show the model exactly what fields you expect.
• Always include error handling for malformed responses.
• Validate parsed output against expected types and ranges before using it.
• Keep extraction prompts simple and focused on one task.

Framework support

Libraries like LangChain, LlamaIndex, and Instructor provide built-in output parsers that handle common formats and validation, reducing boilerplate code.