You attached three files: the current product spec, last month’s draft of the same spec, and a Slack thread debating one section. The model produced a summary that pulled from all three as if they were equally valid. It cited a requirement from the draft (which was rejected in the final spec) and ignored a change in the current version because the Slack thread argued against it. The model has no way to know which document is authoritative. It does not read file modification dates, it does not infer “rejected” from your folder names, and it has no built-in notion of canonical. By default it blends, and blending “current” + “rejected draft” + “internal debate” produces a hybrid that exists nowhere in reality.
Fastest fix: wrap each source in a labeled block with a role and a date, then state a one-line precedence rule, then require a per-claim citation. Three small additions and the averaging stops. The rest of this page shows the exact tags to use and how to confirm it worked.
Why the model “averages” instead of choosing
This is not random. Two well-documented behaviors drive it, and naming them tells you which fix to reach for.
1. No provenance signal. Unless you label them, the model sees three blobs of similarly structured text with no metadata. It cannot distinguish “ratified spec” from “draft we killed.” There is no temporal field, no author, no status — so it treats all three as inputs of equal weight.
2. Position bias, not authority. When sources compete, the model leans on where text sits in the prompt rather than how authoritative it is. The well-studied “lost in the middle” effect (the U-shaped attention curve documented across frontier models through 2026) means content at the very top and very bottom of a long prompt gets the most weight, while material in the middle is under-weighted. So if your rejected draft happens to be pasted last, it can dominate the answer purely because of position. Newer models (Claude in particular) decay more slowly than older ones, but as of June 2026 no production model has eliminated position bias — you cannot rely on the model to “just know” the newest doc is the real one.
The fix for #1 is labels and precedence. The fix for #2 is structure and placement: put the canonical source where the model attends most, and tell it explicitly which label wins.
Which bucket are you in
| Symptom in the output | Likely cause | Go to |
|---|---|---|
| Facts from an old version appear | No temporal/status label; model can’t tell new from old | Steps 1–3 |
| Output favors a third-party blog over your own spec | First-party vs third-party not marked | Step 4 |
| You can’t tell which source any claim came from | No per-claim citation required | Step 5 |
| Docs and code disagree, model picked the wrong one | No “which wins” rule for docs vs code | Step 6 |
| Two sources agree but it cites the weaker one | No “cite the canonical one” instruction | Steps 2, 5 |
Before you change anything
- List every source attached to your prompt.
- For each, decide its role:
canonical,supplementary,context-only, or remove entirely. - Identify temporal order: which is newest, which is current, which is rejected.
- Decide your precedence rule, e.g. “current spec wins over draft; spec wins over Slack; first-party wins over third-party.”
- Plan to require a citation per claim so you can audit which source supplied each fact.
Information to collect
- All sources you attached, in the order you pasted them.
- The output that pulled from the wrong source (keep it as a before/after baseline).
- Your team’s actual precedence — who decides what is canonical.
- Metadata for each source: date, author, status.
- The model and any system prompt in play (the system prompt is where precedence rules belong; see the FAQ).
Shortest path to fix
Step 1: Label every source with name, date, and role
Use a structured wrapper so the model has metadata to reason over. For Claude, the documented multi-document convention is <documents> containing one <document> per source, each with a <source> tag and a <document_content> tag. Add your own role/status/date attributes — XML tag and attribute names are not fixed, so descriptive ones the model can read are exactly what Anthropic recommends.
<documents>
<document index="1" role="CANONICAL" date="2026-05-15" status="ratified">
<source>product-spec.md</source>
<document_content>
... current spec content ...
</document_content>
</document>
<document index="2" role="REJECTED_DRAFT" date="2026-04-01" status="superseded">
<source>spec-draft.md</source>
<document_content>
... old draft, kept for context only ...
</document_content>
</document>
<document index="3" role="DEBATE" date="2026-04-20" status="discussion">
<source>slack-debate.txt</source>
<document_content>
... slack thread ...
</document_content>
</document>
</documents>Now every fact carries a traceable origin and a status the model can act on.
Step 2: State precedence in one line
Put this immediately after the documents block (or, better, in the system prompt — see the FAQ):
Precedence rules:
- Prefer CANONICAL over REJECTED_DRAFT over DEBATE.
- If CANONICAL and DEBATE conflict, CANONICAL wins.
- REJECTED_DRAFT is for context only; never cite it as authoritative.
- If sources agree, cite the CANONICAL one.
- If you find a conflict, surface it explicitly rather than silently picking a side.That last line matters. Telling the model to name conflicts instead of resolving them quietly is what turns “silent blend” into “auditable decision.”
Step 3: Put the canonical source first and the question last
Because of the position bias above, ordering is a real lever, not a cosmetic one. Two placement rules, both backed by Anthropic’s long-context guidance:
- Long source documents go near the top, above your instructions and question.
- Your actual question goes at the very bottom, after all the documents. In Anthropic’s tests, putting the query at the end (rather than the top) of a long multi-document prompt improved response quality by up to 30%.
If you can only keep one source, keep the canonical one and delete the rest — fewer sources means fewer blending mistakes.
Step 4: Mark first-party vs third-party
<document role="FIRST_PARTY_CANONICAL"><source>our-spec.md</source>...</document>
<document role="THIRD_PARTY_CONTEXT"><source>competitor-blog.html</source>...</document>
Rule: For factual claims about our product, prefer FIRST_PARTY.
Use THIRD_PARTY only for industry context, never for facts about our own product.Without this, the model can favor the third-party source simply because marketing copy tends to use more confident, declarative language than an internal spec.
Step 5: Require a citation per claim, grounded in quotes
This is the single highest-leverage step. First make the model quote its evidence, then write — Anthropic’s documented “ground your response in quotes” technique, which cuts noise and exposes blending:
Step A: Before answering, extract the verbatim quotes you will rely on. Put each in
<quotes> tags with its source, e.g. <quote source="product-spec.md">...</quote>.
Step B: For every factual claim in your answer, cite the source inline:
"[claim] [source=product-spec.md, section=2.3]"
If a claim cannot be tied to a quote, mark it [UNCITED] so a reviewer can verify or cut it.Quote-first does two things: it forces the model to anchor on real text rather than a fuzzy average, and it makes silent blending visible — a claim with no quote behind it shows up as [UNCITED].
Step 6: For code vs docs, declare which wins
When documentation disagrees with code:
- For behavior questions ("what does it actually do?"): the code is authoritative.
- For intent questions ("what was this supposed to do?"): the docs are authoritative.
- Always cite which one you used, and flag the discrepancy.How to confirm it’s fixed
Re-run the prompt and check, in order:
- Every factual claim in the output carries a citation (or an explicit
[UNCITED]tag). - Citations point to the canonical source, not to drafts or debate.
- The specific outdated fact from your before-baseline no longer appears.
- A teammate can open the cited source, find the cited section, and confirm the claim.
- Re-running the prompt twice produces the same source distribution — if a fact flips between sources run to run, your precedence rule is too weak; tighten it.
If it still fails
- Delete the secondary sources entirely. If the model only needs the canonical doc, give it only the canonical doc.
- Move precedence into the system prompt. System/developer-level instructions sit above user-message content in both Anthropic’s and OpenAI’s instruction hierarchy, so a precedence rule there is harder to override than one buried among the documents.
- Use retrieval (RAG) with metadata in every chunk. Chunk each source with its
role,date, andstatusattached so provenance survives retrieval. - Split into two passes. First prompt extracts facts from the canonical source only; second prompt does analysis over those extracted facts. The blending opportunity never arises.
- For high-stakes work, audit every citation manually before scaling the prompt.
Prevention
- Default to declaring precedence in one block on every multi-source prompt.
- Use filenames that encode role and date (
spec-canonical-2026-05.md) so the label is obvious before you even open the file. - Keep an explicit “outdated, do not use as authoritative” list for sources you retain only for comparison.
- Require a per-claim citation in any analytical prompt.
- In RAG setups, attach source metadata to every chunk, not just the document header.
- Treat any production multi-source prompt without a precedence rule as a known risk and fix it on sight.
FAQ
Should precedence rules go in the system prompt or the user message? Prefer the system prompt. In both Anthropic’s and OpenAI’s instruction hierarchy, system/developer instructions outrank anything in the user turn, so a precedence rule placed there is less likely to be overridden by confident-sounding language inside a document. Keep the documents and the question in the user turn; keep the “CANONICAL wins” rule in the system prompt.
Do I really need XML tags, or is plain “Source 1 / Source 2” enough?
Plain labels work for two short sources. For longer or more sources, structured tags (<document>, <source>, <document_content>) reduce cross-contamination because the model can see exactly where one source ends and the next begins. Anthropic explicitly recommends this wrapper for multi-document prompts; the tag names themselves are not magic, but consistent boundaries are.
Why did the model prefer the older or rejected document? Usually position, not judgment. Because of the “lost in the middle” / recency bias, a source pasted last (or first) gets weighted more heavily regardless of its status. Put the canonical source near the top, the question at the bottom, and state the precedence rule explicitly so authority — not position — decides.
The model still blends two sources that agree but cites the wrong one. How do I force the canonical citation? Add the explicit instruction “If sources agree, cite the CANONICAL one” (Step 2) and require quote-first grounding (Step 5). When the model has to attach a verbatim quote and a source name to each claim, it can no longer cite a source it didn’t actually read the fact from.
How do I handle a genuine conflict where I don’t yet know which source is right? Tell the model to surface the conflict rather than resolve it: “If two sources disagree on a fact, do not pick one silently — list both claims, their sources and dates, and stop.” You make the call, not the model.
Related reading
- Prompt lacks context hierarchy
- Conflicting instructions weaken output
- Model fills missing details
- AI hallucinated facts
- Long background hides task
External references: Anthropic — long context prompting tips and Anthropic — use XML tags to structure prompts.
Tags: #Troubleshooting #Prompt #Prompt quality #Prompt engineering