You spent 45 minutes writing the perfect prompt. It is 1,400 words. It covers tone, audience, constraints, edge cases, what to avoid, three reference snippets, and a checklist. The output is 200 words of sludge. Earlier the same day, a 60-word version of the same request produced a sharp answer. Adding more does not help: somewhere past roughly 500 words, every extra sentence dilutes priority instead of clarifying it.
This is not your imagination, and it is not a model that “cannot read.” Frontier models read 1,400 words instantly. The problem is position and hierarchy. Long inputs hide which sentences matter most, and the architecture itself attends unevenly to a long block of text. Without a clear shape, the model averages, and averages are mediocre.
TL;DR
- Long prompts fail mostly from structure, not length limits. The first imperative frames the answer, the last block gets the most recency weight, and the middle gets the least attention (the well-documented “lost in the middle” effect).
- Put the task in line 1 and the output schema last. Move bulky reference text into tagged blocks the model treats as data, not instructions.
- As of June 2026, even 1M-token models lose accuracy long before the window is full. Effective multi-fact context for GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro sits well below the advertised maximum, so a tighter prompt usually beats a fuller one.
- Target the smallest prompt that still produces the right answer, not the biggest prompt with every detail.
Why length quietly degrades the answer
Two effects compound as a prompt grows.
Lost in the middle. Models attend most strongly to the start and end of a long input and least to the middle, producing a U-shaped accuracy curve. A widely cited Stanford/Princeton/Berkeley study (“Lost in the Middle: How Language Models Use Long Contexts”) showed accuracy dropping by tens of points when a key fact moved from the edges to the center. This comes partly from how rotary position embeddings (RoPE) decay attention with distance, so it is baked into most current architectures, not a quirk of one model. A 2025 follow-up (Chroma’s “context rot” study) tested 18 frontier models and found all of them degraded as input length grew — often not gradually but dropping off a cliff past a model-specific threshold; Claude models decayed the slowest, but none were immune.
Effective context is shorter than advertised context. Every flagship now claims a 1M-token window, but retrieval and multi-fact reasoning quality falls off well before the window fills. As of June 2026, only Gemini’s deep-reasoning mode holds quality near the full window in independent testing; for GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro, effective context for multi-needle production work sits roughly in the 200K–400K band, above which accuracy degrades noticeably. A 1,400-word prompt is nowhere near those limits, so length is not your hard ceiling — but the same attention bias that breaks 300K-token inputs is already nibbling at your 1,400-word one. Structure is the lever; length is the symptom.
What this means for a 1,400-word prompt
| Position in your prompt | Attention the model gives it | What to put there |
|---|---|---|
| First 1–3 lines | Highest (primacy) | The single task / deliverable |
| Last block | High (recency) | The output schema or format |
| Middle bulk | Lowest | Reference data, fenced and labeled — not buried instructions |
The takeaway: instructions that must be obeyed belong at the top or bottom. Anything in the middle should be data the model looks up, not a rule it has to remember.
Common causes
1. Goal buried in the middle
The first imperative sentence frames the response. If yours is in paragraph 4, the model has already locked into the wrong frame by the time it reads it.
How to spot it: search your prompt for the deliverable verb (write, produce, return, decide). If it first appears after line 5, it is buried.
2. Hidden constraint conflicts
“Be comprehensive AND concise.” “Cover all cases AND stay under 200 words.” Long prompts accumulate these without you noticing. The model averages, which satisfies neither.
How to spot it: list every constraint on one sheet and look for adjective pairs pulling against each other.
3. Background is 80% of the prompt
If 1,100 of your 1,400 words are background and only 300 are task / constraints / output spec, the model interprets the prompt as “engage with this background” rather than “produce X.”
How to spot it: count words per section. If background outweighs task by 3:1 or worse, you have buried the ask.
4. No output format
Long prompt, no schema. The model defaults to a five-paragraph essay because that is what a training-distribution “thoughtful long answer” looks like. Even if you specified the format in passing, without a schema block it does not land.
How to spot it: your output keeps coming back as essay prose when you wanted JSON, a table, or bullets.
5. Repetitive emphasis collapses
If you wrote “really important” five times, none of them feel important. The model parses repetition as “this is the genre,” not “pay extra attention here.”
How to spot it: count how many times you wrote “important,” “critical,” “must,” or “really.” Over five and the emphasis has flattened.
6. Prompt is one wall of prose
No headers, no labels, no whitespace. The model has to infer structure, and inference is unreliable on long inputs. Anthropic’s own testing reports that structured prompts produce noticeably more consistent outputs than the same text unstructured.
How to spot it: there are no ## Background, ## Constraints, or ## Output labels anywhere.
Before you change anything
- Save your current prompt and the current bad output side by side.
- Re-run the 60-word version that worked earlier. Does it still work? This isolates a prompt-shape problem from a model problem.
- Count words per section: task vs. background vs. constraints vs. output spec.
- Write the actual deliverable in one sentence, without rereading the prompt.
- Decide which paragraphs of background would not change the answer if removed.
Shortest path to fix
Step 1: Lift the goal to line 1
TASK: Decide whether to migrate from Postgres to DynamoDB
for the workload below. Pick one. Defend in 3 sentences.
[context follows]The first imperative wins. Make sure yours is correct.
Step 2: Section the body
## Task
<one sentence>
## Context
<bulleted, only the load-bearing facts>
## Constraints
- <each one a single line>
- <if any conflict, say which wins>
## Output format
- decision: postgres or dynamodb
- reason: max 60 wordsLabels dramatically improve parsing on long inputs. For Claude specifically, wrapping each section in named XML tags such as <task>, <context>, and <output_format> is the documented best practice.
Step 3: Cut redundant constraints
Read each constraint and ask: “would a reviewer actually check this?” If not, cut it. Soft preferences fight hard rules; cutting the soft ones strengthens the hard ones.
Step 4: Add one positive example, remove three sentences of rules
One sample of “correct output” is worth a paragraph of rules. If you have rules describing how the output should look, replace them with a single example showing exactly that. Anthropic and OpenAI both recommend a few labeled examples over long prose descriptions of format.
Step 5: Put the output schema last
The last block in the prompt carries the highest recency weight. Use it for the structural spec:
[everything else]
OUTPUT (return only this):
{ "decision": "...", "reason": "..." }If you are calling the API rather than a chat box, do not describe the JSON shape in prose at all. Pass it as a real schema. As of June 2026, OpenAI’s GPT-5.5 Structured Outputs (response_format: { type: "json_schema", json_schema: {...}, strict: true }) and Anthropic’s tool-use / structured-output schemas both enforce the format server-side via constrained decoding, so the model literally cannot emit a non-conforming token. That is far more reliable than asking nicely in the prompt. OpenAI’s guidance is explicit: with Structured Outputs there is “no need for strongly worded prompts to achieve consistent formatting” — drop the schema description from the prompt and let the API supply it.
Step 6: Move large reference text into a tagged block
If you have 800 words of reference material, fence it so the model treats it as input data, not as instructions to engage with:
<reference>
... 800 words of policy ...
</reference>
TASK: <one sentence; for Claude, a query placed AFTER the reference works best>Anthropic’s long-context guidance is specific here: once an input passes ~20K tokens, put long documents at the top of the prompt and the query at the end. In their tests, putting the query after the documents improved response quality by up to 30% on complex, multi-document inputs (Anthropic long-context tips). When wrapping multiple documents for Claude, the documented pattern is <document index="1"> containing a <source> and a <document_content> subtag, so each chunk is clearly delimited.
How to confirm the fix
- A stranger reads only the first three lines and correctly identifies the deliverable.
- The count of “important / critical / must” is under three.
- The background section is no more than 2x the task + constraints sections.
- The output matches the schema you specified, not a generic essay shape.
- Running the same prompt three times produces three outputs of the same shape.
If it still fails
- Compress further. The goal is the smallest prompt that still produces the right answer, not the biggest prompt with all detail.
- Split into multiple turns: turn 1 plans, turn 2 executes, turn 3 verifies. Shorter, focused turns beat one giant turn because each stays near the high-attention edges.
- Ground the answer in quotes. For long reference inputs, Anthropic recommends asking the model to first quote the relevant passages (e.g. “find the relevant quotes, put them in
<quotes>tags, then answer using only those quotes”). Forcing a retrieval step before the answer pulls the load-bearing facts out of the low-attention middle before reasoning starts. - Switch from the chat UI to the API with structured output (JSON schema enforcement, tool use), where the format is guaranteed instead of suggested.
- If the prompt genuinely cannot be shorter, move to a model with stronger long-context behavior. As of June 2026, Claude Opus 4.7 and Gemini 3.1 Pro tend to hold detail across long inputs better than the in-app ChatGPT context on Plus, which is capped well below the API’s window. Note the limits: independent multi-needle testing (Chroma’s “context rot” study of 18 frontier models) finds effective context for GPT-5.5 and Claude Opus 4.7 sits roughly in the 200K–400K band, with only Gemini’s deep-reasoning mode holding quality near the full 1M window.
Prevention
- Keep prompts under 600 words unless reference text is genuinely necessary.
- Default template: TASK first, CONTEXT second, OUTPUT FORMAT last.
- Use section headers (or XML tags for Claude) once a prompt exceeds 200 words.
- For repeated workflows, save a template; do not improvise structure each time.
- Audit long-lived prompts quarterly. They accumulate constraints that no longer matter.
- Re-read the first three lines before sending. A stranger should know what to produce.
FAQ
Does a longer prompt always produce a worse answer?
No. Up to roughly 500 well-structured words, more relevant detail usually helps. The degradation starts when extra words dilute priority, repeat emphasis, or bury the task in the low-attention middle of the prompt. A long prompt that is cleanly sectioned (task at top, reference tagged, schema at bottom) can outperform a short, vague one. Structure, not raw length, is the variable that matters.
Is this the same as hitting the context window limit?
No, and that is a common confusion. A 1,400-word prompt uses around 1,900 tokens, nowhere near any model’s window. The “lost in the middle” attention bias degrades quality long before you run out of context. As of June 2026, even 1M-token models like Claude Opus 4.7 and Gemini 3.1 Pro show measurable accuracy loss well before the window fills, so a fuller prompt is not a better prompt.
Where should I put the most important instruction?
Line 1 or the final block. Models attend most to the start (primacy) and end (recency) of a long input. Put the single deliverable first and the output schema last. Reserve the middle for reference data the model can look up, not for rules it must remember.
Does this differ between ChatGPT, Claude, and Gemini?
The U-shaped attention pattern affects all of them, but Claude responds especially well to named XML tags, and Anthropic documents that placing the query after the documents can lift quality by up to 30%. For ChatGPT, note that the full 1M-token window is reserved for the $200 Pro plan as of June 2026; the in-app Plus context is much smaller, so over-long prompts get truncated or degraded sooner.
Should I describe my JSON format in the prompt or use the API?
If you are building anything programmatic, use the API’s structured-output feature instead of describing the schema in prose. OpenAI’s GPT-5.5 Structured Outputs and Anthropic’s tool schemas enforce the shape server-side. OpenAI explicitly recommends removing the schema description from the prompt when Structured Outputs is on. In a plain chat box with no API, put a minimal example schema in the final block of the prompt.