AI AI Tools Guidebook
Troubleshooting

No Output Format Specified — Why You Get a Wall of Prose

You wanted a structured answer; you got 600 words of paragraphs you cannot copy-paste anywhere.

Published: Author: AI Productivity Guide Team 🌐 查看中文版本

You asked the model to “extract the key facts from this customer email and summarize”. You got 600 words of essay prose. The customer’s order number is buried in paragraph 2, the SLA breach time is in paragraph 4 with a hedge, and the issue category is implied rather than stated. None of this is consumable by your ticket system. You re-prompted “give me a JSON object” and got perfect JSON in two seconds. The model was always capable of structured output — you never asked for it, so it defaulted to essay. Models default to prose because that is what training data labels as “thoughtful response”.

This page walks through why explicit format specification is one of the single highest-leverage prompt changes, and how to write schema blocks that hold up across runs.

Common causes

1. No “Output format” section at all

If you never mention a format, you get the default — five-paragraph essay. The model has no signal to do otherwise.

How to spot it: your prompt has no ## Output section, no schema block, no “return as”.

2. Format mentioned in passing, not as a schema

“Give me a quick summary” is not a format. “Return a 3-bullet list with bold field labels” is.

How to spot it: format instruction is one phrase, not a structured spec.

3. Example shown but format not stated

You showed one structured example expecting the model to copy the shape. It might. It might also infer “structured-ish” and produce something close but not identical.

How to spot it: example present but no explicit “match this shape” rule.

4. Conflicting cues push toward prose

“Be warm and conversational” + “return as JSON” — the model resolves toward prose because warmth lives in prose.

How to spot it: tone and format pull opposite directions.

5. Format forgotten on follow-up turns

You specified JSON on turn 1. By turn 4 the model has drifted back to prose because you stopped specifying. Format leaks out of context.

How to spot it: turn 1 format correct, later turns regress.

Before you change anything

  • Decide the exact downstream consumer of the output: human reader, JSON parser, database row, ticket field.
  • Sketch the ideal shape: fields, order, types, allowed values.
  • For machine consumption, define a strict schema.
  • For human reading, define a structural template (headings, bullet counts, max length).
  • Plan to enforce the format on every turn, not just turn 1.

Information to collect

  • Current prompt with any format hints.
  • The prose output you got.
  • Sample of the format you actually want.
  • Downstream system constraints (JSON keys required, max length, allowed enum values).
  • Model and any system prompt.

Shortest path to fix

Step 1: Add an explicit “Output format” block

## Output format

Return only this JSON (no prose, no explanation):
\`\`\`json
{
  "order_number": "<string, format ORD-XXXXX>",
  "issue_category": "billing | shipping | refund | other",
  "sla_breach_minutes": "<integer or null>",
  "customer_sentiment": "positive | neutral | negative"
}
\`\`\`

The explicit block dominates the output shape.

Step 2: Forbid prose around the format

Output rules:
- Return only the JSON block. No prefix, no suffix.
- No "Here is your output:" preamble.
- No commentary after the closing brace.
- If the input is unparseable, return the schema with all fields set to null.

This catches the “Here is your JSON: …” wrapper that breaks parsers.

Step 3: Use code fences and a clear language tag

For JSON / YAML / SQL / code, fence the block:

\`\`\`json
{ ... }
\`\`\`

Many parsers and the model itself treat fenced blocks as protected zones.

Step 4: For human-facing output, give a structural template

Output format:
- 3 bullets, each starting with **<field>:** in bold.
- Field names: Cause, Fix, Verify.
- Each bullet under 25 words.
- No introduction, no conclusion.

Example:
- **Cause:** Stripe webhook secret expired on Friday.
- **Fix:** Rotate in Stripe dashboard, paste into Vercel env var.
- **Verify:** Send a test webhook and confirm 200 in logs.

Structure + example anchors the shape better than either alone.

Step 5: Pin format on every turn

For chat workflows, repeat the format block at the end of every prompt, or move it to the system prompt / project instruction. Recency bias means the latest turn dictates the format.

Step 6: For API workflows, use structured output mode

OpenAI’s JSON mode, Anthropic’s tool use with schemas, Gemini’s structured output — these mechanically enforce the schema. The model literally cannot produce invalid JSON. Far stronger than prompt-level requests.

How to confirm the fix

  • Output parses without modification by your downstream system.
  • No prose outside the schema block.
  • Running the same prompt 3 times produces 3 outputs of identical shape.
  • Programmatic validation passes on first try.
  • A teammate can describe the format from looking at one output.

If it still fails

  1. Move to structured output mode (JSON mode, tool use) — schema becomes mechanically enforced.
  2. Validate output programmatically and re-prompt with the schema and the validation error.
  3. Lower temperature; format stability improves at lower temperatures.
  4. Try a model with stronger instruction-following (sometimes smaller models follow format better than larger ones).

Prevention

  • Default: every prompt ends with an explicit output format block.
  • For machine consumption: use structured output / JSON mode at the API level.
  • Validate output programmatically; fail fast and re-prompt rather than parse-and-fix later.
  • Reserve prose for genuinely conversational tasks.
  • For chat workflows, pin format in system prompt / project instructions.
  • Audit production prompts: any prompt without an explicit output format block is a risk.

Tags: #Troubleshooting #Prompt #Prompt quality #Prompt engineering

Related Articles