No Output Format Specified: Why You Get a Wall of Prose

You asked the model to “extract the key facts from this customer email and summarize.” You got 600 words of essay prose. The order number is buried in paragraph 2, the SLA breach time is hedged in paragraph 4, and the issue category is implied rather than stated. None of it is consumable by your ticket system. You re-prompt “give me a JSON object” and get clean JSON in two seconds. The model was always capable of structured output. You never asked for it, so it defaulted to essay, because that is what its training data labels as a “thoughtful response.”

Fastest fix: add an explicit ## Output format block with a literal schema and the rule “Return only this, no prose.” If the output feeds code, skip prompt-only requests and use the API’s structured-output mode (OpenAI response_format with strict: true, Anthropic Structured Outputs, or Gemini responseSchema) so the model is decoded against your schema and cannot return malformed JSON.

This page covers why explicit format specification is one of the highest-leverage prompt changes, and how to write schema blocks that hold up across runs.

Which bucket are you in?

Symptom	Most likely cause	Jump to
Prompt has no format section at all	No output spec	Step 1
You said “give me a summary” but got prose	Format mentioned in passing, not as a schema	Step 1 + 2
Output is “structured-ish” but keys vary run to run	Example shown, format not stated	Step 4
You get valid JSON wrapped in “Here is your JSON:“	No rule forbidding preamble	Step 2
Turn 1 was JSON, turn 4 drifted back to prose	Format leaked out of context	Step 5
Tone and format fight (“be warm” + “return JSON”)	Conflicting cues	Step 2 + tone note
Code parses your JSON and it still breaks sometimes	Prompt-level format is not enforced	Step 6

Common causes

1. No “Output format” section at all

If you never mention a format, you get the default: a five-paragraph essay. The model has no signal to do otherwise. Spot it: your prompt has no ## Output section, no schema block, no “return as.”

2. Format mentioned in passing, not as a schema

“Give me a quick summary” is not a format. “Return a 3-bullet list with bold field labels” is. Spot it: the format instruction is one phrase, not a structured spec.

3. Example shown but format not stated

You showed one structured example and expected the model to copy the shape. It might. It might also infer “structured-ish” and produce something close but not identical. Spot it: an example is present but there is no explicit “match this shape exactly” rule.

4. Conflicting cues push toward prose

“Be warm and conversational” plus “return as JSON” pulls in two directions; the model resolves toward prose because warmth lives in prose. Spot it: tone and format pull opposite ways.

5. Format forgotten on follow-up turns

You specified JSON on turn 1. By turn 4 the model has drifted back to prose because you stopped repeating it. Format leaks out of context as the conversation grows. Spot it: turn 1 is correct, later turns regress.

Before you change anything

• Decide the exact downstream consumer: human reader, JSON parser, database row, ticket field.
• Sketch the ideal shape: fields, order, types, allowed values.
• For machine consumption, define a strict schema.
• For human reading, define a structural template (headings, bullet counts, max length).
• Plan to enforce the format on every turn, not just turn 1.

Collect: the current prompt with any format hints, the prose output you got, a sample of the format you actually want, the downstream constraints (required JSON keys, max length, allowed enum values), and the model plus any system prompt.

Shortest path to fix

Step 1: Add an explicit “Output format” block

End the prompt with an ## Output format heading, the line “Return only this JSON (no prose, no explanation),” then a fenced schema:

{
  "order_number": "string, format ORD-XXXXX",
  "issue_category": "billing | shipping | refund | other",
  "sla_breach_minutes": "integer or null",
  "customer_sentiment": "positive | neutral | negative"
}

The explicit block dominates the output shape. Spell out types and allowed values inside each field. "billing | shipping | refund | other" constrains the model far better than a bare "category".

Step 2: Forbid prose around the format

Output rules:
- Return only the JSON block. No prefix, no suffix.
- No "Here is your output:" preamble.
- No commentary after the closing brace.
- If the input is unparseable, return the schema with every field set to null.

This kills the “Here is your JSON: …” wrapper that breaks parsers. The “set every field to null” rule prevents the model from narrating an apology instead of returning the shape.

Step 3: Use code fences and a clear language tag

For JSON, YAML, SQL, or code, fence the block with a language tag (```json). Many parsers, and the model itself, treat a fenced block as a protected zone, which reduces stray prose leaking inside it.

Step 4: For human-facing output, give a structural template plus one example

Output format:
- 3 bullets, each starting with **<field>:** in bold.
- Field names: Cause, Fix, Verify.
- Each bullet under 25 words.
- No introduction, no conclusion.

Example:
- **Cause:** Stripe webhook secret expired on Friday.
- **Fix:** Rotate in Stripe dashboard, paste into Vercel env var.
- **Verify:** Send a test webhook and confirm 200 in logs.

Structure plus one example anchors the shape better than either alone. The example pins exact field names and bullet style; the rules pin the count and length.

Step 5: Pin the format on every turn

For chat workflows, repeat the format block at the end of every prompt, or move it into the system prompt / project instructions (custom instructions in ChatGPT, a Project in Claude, a saved system prompt). Recency matters: the latest turn dominates, so a format stated only on turn 1 fades as the thread grows.

Step 6: For API workflows, use structured-output mode (not just a prompt)

If output feeds code, a prompt-level request is the weakest option. Every major API now has a mode that decodes the model against your schema, so it cannot emit invalid JSON. As of June 2026:

• OpenAI Structured Outputs. Pass response_format: { type: "json_schema", json_schema: { name, schema, strict: true } } (Responses API uses text.format with the same fields). With strict: true, the decoder physically cannot emit tokens that violate your schema. This is the production default. The older JSON mode (response_format: { type: "json_object" }) only guarantees valid JSON syntax, not your keys or types, and OpenAI now treats it as legacy. Prefer json_schema strict mode.
• Anthropic Structured Outputs. The Claude Developer Platform added Structured Outputs so you can require a response to conform to a JSON schema, plus strict tool use so tool inputs match their schema exactly. Before this feature, the standard trick was a single forced tool with the schema as its input; that still works, but native Structured Outputs is cleaner. Claude Opus 4.7 and Sonnet 4.6 support it.
• Gemini. Set responseMimeType: "application/json" and pass your schema in responseSchema in the generation config. For a closed set of choices, use responseMimeType: "text/x.enum" with an enum schema. Gemini 3.1 Pro supports this.

Mechanical enforcement is far stronger than any prompt phrasing, because it operates at the sampling layer, not as a post-hoc instruction.

How to confirm it’s fixed

• The output parses without modification by your downstream system.
• There is no prose outside the schema block.
• Running the same prompt 3 times produces 3 outputs of identical shape.
• Programmatic validation (a JSON schema validator, a Pydantic model, JSON.parse) passes on the first try.
• A teammate can describe the format from looking at one output.

If it still fails

1. Move to structured-output mode (Step 6) so the schema is mechanically enforced rather than requested.
2. Validate the output programmatically and re-prompt with both the schema and the exact validation error; models fix a named error far more reliably than a vague “that was wrong.”
3. Lower the temperature. Format stability improves at lower temperatures because the model commits to the most probable token shape.
4. Try a different model. Instruction-following varies, and a smaller, cheaper model sometimes holds a fixed format better than a larger one tuned for richer prose.

Prevention

• Default rule: every prompt ends with an explicit output-format block.
• For machine consumption: use structured-output mode (strict json_schema, Anthropic Structured Outputs, Gemini responseSchema) at the API level, not a prompt request.
• Validate output programmatically; fail fast and re-prompt rather than parse-and-fix later.
• Reserve prose for genuinely conversational tasks.
• For chat workflows, pin the format in the system prompt / project instructions.
• Audit production prompts: any prompt without an explicit output-format block is a risk.

FAQ

Why does the model ignore “respond in JSON” but obey a full schema block? “Respond in JSON” is a hint; a literal schema with field names, types, and allowed values is a target. The model has something concrete to match, and you have something concrete to validate against. Add the rule “return only the JSON, no prose” so it does not wrap the block in commentary.

JSON mode vs Structured Outputs: which should I use? On OpenAI, prefer Structured Outputs (json_schema with strict: true). Legacy JSON mode (json_object) only guarantees valid JSON syntax, not that your keys, types, or enums are honored, so you still need to validate and retry. Strict Structured Outputs guarantees schema adherence at decode time, as of June 2026.

Can Claude or Gemini guarantee a schema like OpenAI? Yes. Anthropic shipped Structured Outputs on the Claude Developer Platform for JSON-schema-conforming responses and strict tool use, supported on Opus 4.7 and Sonnet 4.6. Gemini enforces a schema via responseMimeType: "application/json" plus responseSchema (and text/x.enum for closed choices) on Gemini 3.1 Pro.

The format is perfect on turn 1 but drifts back to prose later. Why? Format leaks out of context as the conversation grows and the original instruction loses weight. Restate the format block at the end of each prompt, or move it into the system prompt / project instructions so it applies to every turn instead of only the first.

My JSON is valid but the model still adds “Here is your output:” before it. That preamble breaks naive parsers. Add explicit rules: “Return only the JSON block. No prefix, no suffix. No commentary after the closing brace.” For an API workflow, structured-output mode removes the preamble entirely because only schema-conforming tokens are emitted.

Should I lower temperature for more stable formatting? Yes, when format stability matters more than variety. Lower temperature makes the model commit to the most probable token shape, which keeps the structure consistent across runs. Combine it with structured-output mode for the strongest guarantee.

Related reading

• AI output style drift
• Style vs format conflict
• Long prompt degrades output
• No success criteria
• Output polished not actionable

Tags: #Troubleshooting #Prompt #Prompt quality #Prompt engineering

Related Articles

Troubleshooting

Few-Shot Examples Have Uneven Quality and Drag Output Down

You gave the model 5 examples; 2 are great, 3 are mediocre, and it averages toward the mediocre ones. Why example variance hurts and how to curate down to 3-5 consistent ones.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Returns Invalid JSON Because Schema Was Described, Not Enforced

You asked for JSON matching a schema. Most calls return valid JSON, some return prose with JSON inside, some omit fields. Description vs enforcement, and how to fix at the API layer.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Invented Fake Citations and URLs

The model produced a citation like Smith et al. 2019 and the paper does not exist, or a URL that 404s. Why citation hallucination happens and how to stop it for good.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Model Replies in the Wrong Language (How to Lock It)

You prompted in English and the model answered in Chinese, or it switched mid-output. The exact cause of language drift and the system-prompt + retry pattern that locks the output language, verified June 2026.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

LLM Response Cut Off Mid-Sentence: max_tokens Too Low (2026 Fix)

The model's reply ends mid-sentence, mid-JSON, or with an unclosed code block. It is almost always the token cap. How to size it, detect truncation per SDK, and recover.

May 24, 2026 #Prompt engineering #Troubleshooting

Troubleshooting

Prompt Asks for 10 Items, Model Returns 3 and Stops

You asked for 10 ideas and got 3, or 10 slots padded with filler. Why list-length prompts under-deliver, and the prompt + schema fixes that actually get you N distinct items.

May 24, 2026 #Prompt engineering #Troubleshooting