Asking an AI to “write a literature review” is how people get a four-page essay with seven made-up citations and one accidental retraction. A real systematic review still needs you to define inclusion criteria, hit the actual databases, screen, and extract. AI helps with triage, extraction, and synthesis — never with citation generation. This tutorial walks the loop that keeps the rigor and cuts the busywork roughly in half.
What this covers
A real systematic-review workflow with AI in three roles only: screening abstracts against your inclusion criteria, extracting structured data from full texts you have actually downloaded, and helping draft the synthesis where every claim points back to a numbered paper. The workflow assumes PRISMA-adjacent discipline, not just “skim and summarize.”
Who this is for
PhD students writing the review chapter, postdocs producing a meta-analysis, evidence-synthesis teams in medicine, policy, or ed-research, and any consultant who needs a defensible “what does the literature say” section. Less useful for casual reading — for that, the AI paper-reading workflow is faster.
When to reach for it
When you have a defined research question, access to the right databases (PubMed, Scopus, Web of Science, ACM, Semantic Scholar), and a body of literature that is too large to read linearly. Not the right tool when the field has fewer than 20 candidate papers — read those by hand — or when the review will be submitted to a journal that bans AI-assisted screening; check author guidelines first.
Before you start
- Write your PICO or equivalent. Population, intervention, comparator, outcome — or your field’s equivalent framing. Without it, inclusion screening drifts into vibes.
- Decide your databases and search strings in advance. The AI does not run the database search; you do.
- Decide what gets extracted from every included paper: design, sample, method, primary result, effect size, limitations. Lock the columns of your extraction sheet before you start screening.
- Pick a long-context model: Claude Sonnet 4.6 or Opus 4.7, GPT-5.5, Gemini 3 Pro. Smaller models lose mid-paper detail during extraction.
Step by step
- Run the database search yourself. PubMed, Scopus, Semantic Scholar — whatever your field uses. Export the hits as a RIS or CSV. The AI does not search; it screens. This separation is the whole reason the review is defensible.
- Title and abstract screening with AI as a second reviewer. Paste 20-50 abstracts at a time with your inclusion criteria. Ask “for each, output INCLUDE, EXCLUDE, or UNCLEAR, with a one-line rationale referencing my criteria.” Treat UNCLEAR as automatic include for full-text review.
- Reconcile against your own first-reviewer pass. Disagreements are signal — they expose ambiguous criteria. Most reviews need 1-2 criteria rewrites at this stage.
- Download full texts for the included set. This is non-negotiable. You cannot extract data from a paper you have not downloaded. Build a folder where the filename is
firstauthor_year_id.pdf. - Run extraction one paper at a time. Upload the PDF. Ask: “Extract design, sample size, intervention, comparator, primary outcome, effect size with CI, and the single biggest limitation the authors acknowledge. Return as a row of pipe-separated values matching this column order.” For a related single-paper workflow, see the AI paper-reading workflow — the extraction step here is the structured cousin of pass 2 there.
- Spot-check 20 percent of extracted rows against the source. Open the PDF. Find the number. If the AI got effect size or sample size wrong, you have a calibration problem — switch models or shorten the prompt.
- Synthesis draft. Cluster included papers by design or by intervention. Ask the AI to draft a paragraph per cluster, citing only by your numeric IDs. Never let the AI invent author names or years.
First-run exercise
Pick a sub-question of your real review — narrow enough that 10-15 papers cover it. Run the full loop end-to-end on that sub-question first. Time each phase. Most teams find screening compresses the most, extraction modestly, and synthesis the least. Use the per-phase timing to budget the full review; the sub-question result also tests whether your inclusion criteria are crisp enough.
Quality check
- Every cell in the extraction sheet matches a sentence or table in the source PDF — spot-check 20 percent.
- Disagreements between you and the AI during screening were logged, not silently overridden.
- No citation in the final synthesis is invented — every numeric ID resolves to a paper in your downloaded folder.
- Effect sizes report confidence intervals or “not reported” — never a single number with no spread.
- The synthesis paragraphs have a clear shape: established, contested, gap. If everything reads “established,” you are flattering the field.
How to reuse this workflow
- Save your inclusion criteria, extraction columns, and the screening prompt as a
review_template.md. New question, new search string, same scaffolding. - Build a personal model-calibration log. Which model got effect sizes right at what rate, across which fields. This compounds across reviews.
- Keep the screening reconciliation log. Reviewers asking “how did you handle ambiguous cases” want to see this artifact.
Recommended workflow
PICO question → search string → database hits → AI second-reviewer screening → reconcile → download full texts → structured extraction → 20 percent spot-check → cluster and synthesize → cite by numeric IDs. Plan one week for a 100-paper review with AI assistance, versus three weeks linear.
Common mistakes
- Asking the AI to “find the relevant papers” — it cannot replace your database search and will invent citations.
- Skipping the spot-check on extraction — confident-sounding errors enter the sheet and survive into the meta-analysis.
- Letting the AI cluster the papers without your judgment — clusters end up by surface topic rather than by mechanism.
- Treating UNCLEAR as exclude during screening — you lose the borderline papers that are usually the most interesting.
- Using a short-context model on long papers — the back half gets summarized away.
- Forgetting to record your prompt versions. Reviewers will ask which prompt produced which screening pass.
FAQ
- Will my journal accept AI-assisted reviews?: Check author guidelines. Most accept AI for screening and extraction if disclosed; few accept AI-generated prose without revision. Disclose what you used.
- Which model for extraction?: Long-context preferred. Claude Sonnet 4.6 or Opus 4.7 for nuanced fields; Gemini 3 Pro for very long PDFs.
- How many papers per screening batch?: 20-50 abstracts. Beyond 50, the model starts averaging the criteria.
- What about non-English papers?: AI translation helps for screening but is risky for extraction. For included non-English papers, get a human translation of the methods section.
- Should I use specialized tools like Rayyan or Covidence?: Yes, alongside AI. AI is a screening assistant, not a workflow tool — Rayyan and Covidence handle the audit trail.
Related
- AI paper-reading workflow
- Literature review prompts
- AI citation check workflow
- AI research workflow guide
Tags: #lit-review #Research #Tutorial