Most AI-assisted drafts ship with at least one bad citation: a real URL that doesn’t say what you claimed, a real-sounding paper that doesn’t exist, or a quote pulled from a secondary blog that mangled the original. This workflow gives editors, researchers, and content marketers a four-pass routine that catches all three failure modes before publish, with about 5 minutes of work per citation on the first run and 1 minute once you have templates.
What this covers
A four-pass AI workflow — existence, accuracy, provenance, recency — applied to every citation in a draft. The output is a clean “provenance ledger” you ship alongside the final piece, plus a list of citations that need to be re-researched or removed. This is not about catching plagiarism. It is about catching the four ways AI-assisted research silently lies to you.
Who this is for
Editors, researchers, students, content marketers, policy analysts — anyone shipping work with citations they didn’t personally verify. If you wrote every citation yourself by reading the source, you don’t need this. If an AI drafted any part with citations, or a co-author handed you a deck and said “trust me”, you do.
When to reach for it
Before publishing AI-assisted research, journal-club notes, blog posts, white papers, or briefs that lean on cited claims. Also before forwarding any deck or memo that quotes sources you cannot vouch for personally. The cost of one fabricated citation in a published piece is usually higher than the entire audit cost.
Before you start
- Get your draft in plain text or Markdown. PDFs hide citations in footnotes and break paste.
- Have at least two tools available: a chat model (Claude, GPT) for parsing and a web-grounded model (Perplexity, Gemini with search, GPT with browsing) for live URL checks.
- Decide your bar up front: “all citations must be primary unless explicitly noted” is a much higher bar than “all citations must exist”. Pick one and stick with it.
- Block 30-90 minutes the first time. Subsequent runs on similar pieces are much faster.
Step by step
- Build the ledger. Paste the full draft into a chat model and prompt: “Extract every citation into a table with columns: claim (one sentence), source name, URL, page or section, claim type (statistic, quote, definition, attribution).” Save this table.
- Pass 1 — existence. Paste the citation list into Perplexity or a search-grounded model. Prompt: “For each row, confirm the URL resolves and the source exists. Flag any that 404, redirect to a homepage, or appear to be fabricated. Cite your search result for each verdict.”
- Pass 2 — accuracy. For each existing source, prompt: “Fetch the source and quote the specific passage that supports the claim. If the passage does not exist, say so explicitly. Do not paraphrase — quote.” Compare the quoted passage to your draft’s claim word by word.
- Pass 3 — provenance. Prompt: “Is this source primary (original data, study, or first-hand account), secondary (analysis of primary sources), or tertiary (summary of secondary)? If it is not primary, identify the primary source it ultimately points to.” Weak provenance becomes a re-research candidate.
- Pass 4 — recency. Prompt: “Has the cited claim been updated, contradicted, retracted, or superseded since the source was published? Search for newer publications by the same authors or on the same topic in the last 12 months.”
- Score and decide. For any citation that fails 2 of 4 passes: re-research or remove. Do not patch — re-research. Patching a weak citation usually means swapping in a slightly less weak one and shipping anyway.
- Produce the ledger. Save the cleaned citation list with verdicts as your provenance ledger; ship it alongside the final draft. If you publish on a content site, link it from the article footer.
Prompt templates
Pass 1: "Here are 12 citations from a draft. For each, confirm
the URL resolves to a real source and the source actually exists.
Return: row | resolves Y/N | source exists Y/N | one-line evidence.
Do not guess. If unsure, say UNSURE and explain."Pass 2: "For each cited source below, fetch the page and quote
the exact passage that supports the claim. If no such passage
exists at the URL, return NO MATCH and quote the closest thing
you found. Maximum 30 words per quote."First-run exercise
- Pick one short draft with 5-8 citations.
- Run all four passes on every citation, even ones you wrote yourself. The point is to learn what each pass catches and how much it costs.
- Tally results: how many failed Pass 1? Pass 2? You will likely find Pass 2 (accuracy) is the most common failure, not Pass 1 (existence).
- Time the run. The first one is slow; the third one is fast.
Quality check
- Did the AI quote actual passages or paraphrase? Paraphrase means it didn’t actually fetch the source — re-run with a model that has browsing enabled.
- Are flagged “fabricated” sources actually fabricated, or just behind a paywall? Paywalled sources are real but unverifiable; mark them separately.
- Did any quote in your draft survive Pass 2 without modification? If yes, that quote is safe. If you edited it for flow, you may have changed its meaning — restore the original.
How to reuse this workflow
- Save the four prompts as a single text file with one variable: the citation list.
- Build a per-publication style note: “we require primary sources for statistics, secondary acceptable for definitions.” Bake that into Pass 3.
- Rerun the workflow before any reprint, anthology, or syndication. Sources rot — links break, papers get retracted, statistics get revised.
Recommended workflow
List all citations -> existence pass -> accuracy pass -> provenance pass -> recency pass -> re-research failures -> provenance ledger shipped alongside the draft.
Common mistakes
- Trusting that “the AI wrote a citation, so the citation exists.” LLMs invent plausible-sounding papers constantly; existence is exactly what Pass 1 catches.
- Skipping the accuracy pass — the URL exists but doesn’t say what the draft claims. This is the single most common failure and the easiest to ship past without noticing.
- Treating a secondary citation as primary because the secondary source said so. The secondary may have mangled the primary.
- Patching a failed citation with a slightly weaker one from the same search result. If the strongest source you can find is weak, that is a signal your underlying claim is wrong, not just your sourcing.
- Not checking recency on statistics. “67% of X” from 2019 is often 51% by 2026 and the article reads as dated or wrong.
- Pasting a 20,000-word draft into one prompt. Chunk by section; long contexts degrade citation extraction accuracy.
FAQ
- Does this catch AI-fabricated sources?: Yes. Pass 1 (existence) is exactly the fabrication catch. Most LLM hallucinations fail at this stage; a small minority invent URLs that resolve to unrelated real pages, which Pass 2 catches.
- How long does this take?: About 5 minutes per citation the first time; about 1 minute once you have a template prompt and you batch by domain.
- Which models are best for each pass?: Existence: Perplexity or any search-grounded model. Accuracy: a model with browsing (Claude with web tool, GPT with browsing, Gemini). Provenance: any reasoning model with the source contents pasted. Recency: search-grounded model again.
- What about paywalled sources?: Mark them as “real but unverifiable” in the ledger. Ask the AI to verify via abstract, preprint, or institutional summary. Do not ship a claim whose primary source you cannot read at all.
- Can I automate this in a script?: Yes, but only the existence pass cleanly. Accuracy and provenance need a human in the loop — the failure modes are too subtle to fully delegate.
- What if I cited the same source 10 times?: Run accuracy once per source-passage pair, not once per source. A source you trust for claim A may not support claim B.
Related
- AI fact-check workflow
- Perplexity basics
- AI multi-source synthesis
- AI research workflow tutorial
- AI Competitive Research Tutorial: 5 Competitors in 30 Minutes
- AI Historical Archive Research Tutorial: Primary Sources First
- AI Market Sizing Tutorial: TAM/SAM/SOM From Top-Down + Bottom-Up
- AI Systematic Literature Review Tutorial Without Hallucination