Fix: langchain.schema.OutputParserException — Failed to Parse LLM Output

Updated 2026-03-06

The Error

OutputParserException is raised when the LLM’s response cannot be parsed into the expected format by the parser attached to your chain. Here are the most common variants you’ll see:

langchain_core.exceptions.OutputParserException: Got invalid return object. Expected the LLM to return a string with one of the following formats:

`Foo`
`Bar`
`Baz`

Got: Sure, here's the answer: Baz but let me explain...
langchain_core.exceptions.OutputParserException: Failed to parse:
{
  "name": "Alice",
  "age": "twenty-five"
}

Error: value_error.integer - ensure this value is an integer (type=value_error.integer)
langchain_core.exceptions.OutputParserException: Could not parse JSON:
~~~json
{"items": ["apple", "banana"]}
~~~

Error: json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 24)

What This Means

The LLM returned text that didn’t match the format your parser expects. This isn’t a bug in your code—LLMs are probabilistic systems, and even with explicit format instructions, they occasionally drift: wrapping JSON in markdown fences, adding explanation prose, returning invalid JSON, or truncating long responses. At scale (thousands of requests), expect a percentage of these failures. The fix depends on which parser failed and why the output was malformed.

The Fix

Cause 1: LLM wraps JSON in markdown fences

The Problem: The LLM returns valid JSON but surrounded by markdown code blocks:

~~~json
{"name": "Alice", "age": 30}
~~~

Most parsers fail on the backticks and json language tag.

The Solution: Use JsonOutputParser which handles this automatically, or strip fences before parsing:

from langchain_core.output_parsers import JsonOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o", temperature=0)

prompt = PromptTemplate(
    template="Extract the person info and return ONLY valid JSON:\n{query}",
    input_variables=["query"]
)

parser = JsonOutputParser()

chain = prompt | model | parser

result = chain.invoke({"query": "John is 25 years old and works as a engineer"})
print(result)  # Works even if LLM returns ```json {...}```

Tested on: langchain 0.3.18, Python 3.12, March 2026

Alternatively, if you’re using a custom parser, strip fences pre-parsing:

def strip_markdown_fences(text: str) -> str:
    """Remove ```json ... ``` wrapper if present."""
    text = text.strip()
    if text.startswith("```"):
        text = text.split("```")[1]
        if text.startswith("json"):
            text = text[4:]  # Remove "json" language tag
    return text.strip()

from langchain_core.output_parsers import BaseOutputParser
import json

class RobustJsonParser(BaseOutputParser):
    def parse(self, text: str):
        cleaned = strip_markdown_fences(text)
        return json.loads(cleaned)

Cause 2: LLM adds explanation text before/after the JSON

The Problem: The LLM returns explanatory prose surrounding the actual data:

Sure! Here's the extracted info:
{"name": "Alice", "age": 30}

I found this by looking at the document carefully.

No parser handles raw text outside the JSON.

The Solution: Use .with_structured_output() on modern models (OpenAI, Anthropic), which bypasses natural language entirely and uses tool use / function calling to guarantee format:

from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

class PersonInfo(BaseModel):
    name: str = Field(description="Person's full name")
    age: int = Field(description="Person's age in years")
    occupation: str = Field(description="Job title or occupation")

model = ChatOpenAI(model="gpt-4o", temperature=0)

#Bind structured output directly — LLM must return valid schema or fail safely
structured_model = model.with_structured_output(PersonInfo)

result = structured_model.invoke("John is 25 and works as an engineer")
print(result)  # PersonInfo(name='John', age=25, occupation='engineer')
print(type(result))  # <class '__main__.PersonInfo'>

With Anthropic (same API):

from langchain_anthropic import ChatAnthropic
from pydantic import BaseModel, Field

class PersonInfo(BaseModel):
    name: str = Field(description="Person's full name")
    age: int = Field(description="Person's age in years")
    occupation: str = Field(description="Job title or occupation")

model = ChatAnthropic(model="claude-3-5-sonnet-20241022", temperature=0)

structured_model = model.with_structured_output(PersonInfo)

result = structured_model.invoke("John is 25 and works as an engineer")
print(result)  # Same output structure

Why this works: Structured output uses the model’s native tool-use capability (e.g., OpenAI’s function calling, Anthropic’s tool_use blocks). The model generates structured data directly, not prose—format is guaranteed at the API level. This is the single most reliable fix.

Tested on: langchain 0.3.18, OpenAI API, Anthropic API, Python 3.12, March 2026

Cause 3: Pydantic schema mismatch

The Problem: The LLM returns valid JSON but it doesn’t match your schema:

  • Wrong field type: "age": "thirty" when expecting an integer
  • Missing required field: {"name": "Alice"} without the required age field
  • Extra unexpected fields: {"name": "Alice", "age": 30, "nickname": "Ali"} when nickname isn’t in the schema
langchain_core.exceptions.OutputParserException: Failed to parse:
{"name": "Alice", "age": "thirty"}

Error: value_error.integer - ensure this value is an integer (type=value_error.integer)

The Solution: Use OutputFixingParser which wraps your original parser and auto-corrects via LLM:

from langchain_core.output_parsers import PydanticOutputParser, OutputFixingParser
from langchain_openai import ChatOpenAI
from pydantic import BaseModel

class PersonInfo(BaseModel):
    name: str
    age: int
    occupation: str

model = ChatOpenAI(model="gpt-4o", temperature=0)

#Original parser
original_parser = PydanticOutputParser(pydantic_object=PersonInfo)

#Wrap with auto-correcting layer
fixing_parser = OutputFixingParser.from_llm(
    llm=model,
    parser=original_parser
)

#When LLM returns invalid output, OutputFixingParser sends it back to LLM
#with error message: "Failed to parse: {...} Error: age must be integer"
#LLM self-corrects and returns valid JSON

bad_output = '{"name": "Alice", "age": "thirty", "occupation": "engineer"}'
result = fixing_parser.parse(bad_output)
print(result)  # PersonInfo(name='Alice', age=30, occupation='engineer')

This adds one extra LLM call on failure but saves your pipeline. Use it when:

  • You have strict schemas and can’t relax them
  • Structured output isn’t available (older models)
  • You need Pydantic v1 validation logic

Tested on: langchain 0.3.18, Python 3.12, March 2026

Cause 4: Truncated output

The Problem: Long structured responses hit the token limit and get cut off:

{"items": [
  {"id": 1, "name": "apple", "price": 0.5},
  {"id": 2, "name": "banana", "price": 0.3},
  {"id": 3, "name": "

The JSON is incomplete and unparseable.

The Solution: Increase max_tokens or request data in smaller chunks:

from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import JsonOutputParser

model = ChatOpenAI(
    model="gpt-4o",
    temperature=0,
    max_tokens=4000  # Increase from default ~2000
)

prompt = "List all items with full details in JSON format"
parser = JsonOutputParser()

chain = prompt | model | parser
result = chain.invoke({})

Or request paginated results:

from langchain_core.prompts import PromptTemplate

prompt = PromptTemplate(
    template="Return items {start} to {end} only, as JSON array:\n{query}",
    input_variables=["start", "end", "query"]
)

all_results = []
batch_size = 10

for i in range(0, 100, batch_size):
    batch = chain.invoke({
        "start": i,
        "end": i + batch_size,
        "query": "List all items with details"
    })
    all_results.extend(batch)

print(all_results)

Tested on: langchain 0.3.18, Python 3.12, March 2026

Why This Happens

LLMs are probabilistic—every token is a probability distribution, and the model samples from it. Even at temperature=0, there’s no absolute guarantee of format compliance:

  1. Training conflict: LLMs are trained to generate natural language. Format instructions (“return ONLY JSON”) compete with this training when composed into a prompt.
  2. Context window limits: Longer schemas or explanations consume tokens, leaving less context for the model to “understand” the format instruction.
  3. Token-by-token sampling: Each token is independent. The model might decide } is 70% likely but generates , at 75% probability instead.
  4. At scale, failure is guaranteed: If any request has a 1% failure rate, one million requests will fail 10,000 times. The correct architecture assumes failures and handles them gracefully.

This is why structured output APIs (tool use, function calling) are superior—they bypass token sampling for the format layer. Format is enforced at the API level, not prompt level.

Edge Cases

Streaming + JSON parsing

Most parsers (including PydanticOutputParser) require complete text. Use JsonOutputParser which supports partial streaming:

from langchain_core.output_parsers import JsonOutputParser
from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o")
parser = JsonOutputParser()

for chunk in (prompt | model).stream({}):
    print(chunk, end="", flush=True)

JsonOutputParser works with .stream() but PydanticOutputParser does not—you must consume the full response first.

Pydantic v1 vs v2 differences

LangChain 0.3 officially uses Pydantic v2, but some projects still run v1. Error messages differ:

Pydantic v1:

value_error.integer - ensure this value is an integer

Pydantic v2:

Input should be a valid integer, unable to parse string as an integer

If you see old error formats in your logs, check your environment’s pip show pydantic.

Multilingual prompts

Format instructions in non-English can confuse models:

#Bad: competing instructions
prompt = """
Extrae informacion en formato JSON. Responde en español.
[formato JSON aquí]
"""

#Good: separate the language instruction from format
prompt = """
Extract information and format as JSON below. Respond in Spanish for explanations only.
Format: {...}
"""

See Also