Crawl Stats shows Googlebot’s crawl requests jumping sharply — say from 50/day to 300+/day — which looks like SEO progress. Over the same period, Performance impressions are flat or down.
Fastest path: the spike almost never means new traffic is coming. Crawl is the input, indexing is the middle, impressions are the output, and the chain only pays out when all three move together. Open three Search Console reports side by side — Settings -> Crawl stats, Indexing -> Pages, and Performance — and find the step where the numbers stop climbing. That step is your real problem. In most cases it is the middle: new pages are crawled but stuck at “Crawled - currently not indexed,” so crawl goes up and indexed pages do not.
Symptoms
- Crawl Stats total crawl requests up sharply (e.g., 50 -> 300/day)
- Pages report shows Indexed nearly flat or only a slight increase (about +5%)
- Performance impressions flat or down
- The “Crawled - currently not indexed” count is also rising
The pipeline in one line
Crawl -> Index -> Impressions. Crawling a URL is necessary but not sufficient: Google still has to decide the page is worth indexing, and then the indexed page still has to target a query people actually search. If Google does not consider new pages worth indexing, the pipeline clogs at the middle and nothing reaches the output.
A quick read on where you are stuck, comparing the three reports over the same 28-day window:
| Crawl | Indexed | Impressions | What it means |
|---|---|---|---|
| up | up | up | Full pipeline working — this is the normal, healthy case |
| up | up | flat | Indexing is fine; your queries have little or no search volume |
| up | flat | flat | Stuck at the indexing decision (quality / duplication) |
| up | down | down | Old pages demoted plus new pages too thin — net loss |
Common causes
1. New pages are crawled but not indexed (quality bar not met)
The most common case. You published 30 new articles, Googlebot crawled all of them, but each is thin (under ~500 words, no unique data, screenshots, or first-hand testing) and they all land in “Crawled - currently not indexed.” Crawl goes up by 30; indexed goes up by 0.
Since Google’s 2025 quality and helpful-content systems tightened, pages that read as machine-generated and lack demonstrated first-hand expertise are routinely held at this status, and thin AI-spun pages can even suppress crawl demand for the rest of the site.
How to confirm: Search Console -> Indexing -> Pages -> open the “Crawled - currently not indexed” reason and check its trend over the last 4 weeks. If the curve is climbing in step with the crawl spike, this is your root cause.
2. Crawl is concentrated on faceted / duplicate URLs
Settings -> Crawl stats -> click into "By file type" / "By response" rows
to see example URLs. Heavy crawl on /tag/, /search?, or /?utm_= = wasted budget.Googlebot has a finite daily request capacity for your site. If it spends that on a thousand /tag/?page=N or parameter-permuted URLs, your real articles wait in line. Crawl Stats does not have a literal “by URL type” view — to see which paths are eating the budget, click into a grouping row (for example a 200 response or an HTML file type) to expand its example URLs, or analyze your server access logs directly.
3. Indexed pages do not target queries with volume
The index count went up, but for topics nobody searches:
- Internal tech docs indexed — no one searches for them
- Template-generated “City X + keyword Y” combos indexed — zero search volume
- Lots of “About our company X” content indexed, while users search industry-generic queries
4. Old indexed pages got demoted, offsetting new gains
New pages indexed +50, but older pages were demoted by a core or helpful-content update and lost -100, for a net of -50.
How to confirm: Performance -> set date range to 28 days, compare to the previous period, and filter to pages that existed 30+ days ago. If the overall line is down even though you added pages, the decay is on the old pages.
5. Crawl spike from a robots reset (post-fix re-crawl)
If you recently changed robots.txt — especially from block to allow, or removed a stray noindex — Google re-crawls heavily to catch up on history. That is a backlog drain, not new demand, and it does not produce new impressions on its own.
Shortest path to fix
Step 1: Find where the pipeline clogs
Open the three reports above for the same 28-day window and match your pattern to the decision table near the top. The row you land on tells you which step below to jump to.
Step 2: See where the crawl is going
In Settings -> Crawl stats, the data is grouped four ways: By response (200, 301, 404, 5xx…), By file type (HTML, image, JS, CSS…), By Googlebot type, and By purpose (only two values: Discover = first-time fetch, Refresh = re-fetch). Click any row to see example URLs. A spike that is almost entirely Refresh is Google re-checking pages you already have, not finding new ones — consistent with cause 5.
If you find heavy crawl on parameter or auto-generated URLs:
Disallow them in robots.txt so the budget routes to real content:
User-agent: *
Disallow: /*?utm_
Disallow: /*?sort=
Disallow: /*?filter=
Disallow: /tag/
Disallow: /searchNote: robots.txt stops crawling, not indexing. To keep already-indexed junk URLs out of the index, leave them crawlable and add a noindex meta tag (or remove them and serve a 410) — a blocked URL can still be indexed from links without its content.
Step 3: New pages — indexed or rejected?
In the Pages report, filter to your new URLs:
- Mostly Indexed -> indexing is fine; your problem is query volume, go to Step 4
- Mostly Crawled - currently not indexed -> quality / duplication issue, go to Step 5
Step 4: Confirm the target queries have real volume
New pages indexed but 0 impressions usually means nobody searches the term:
Open Google Keyword Planner (or Search Console's own query data on
similar pages). Enter the new page's main keyword. Read monthly volume.If it is under ~10 searches/month, that is why there are no impressions. Re-target toward terms people actually search.
Step 5: Add depth to the new pages
If “Crawled - currently not indexed” is the growing bucket:
- Bring genuinely useful word count up (aim for 800+, but never pad)
- Add original screenshots, tables, and first-person testing notes
- Add 3+ relevant internal links from and to the page
- See Low content depth issue
Then request reprocessing: use URL Inspection -> Request Indexing for a handful of priority URLs, and make sure they sit in your XML sitemap. Do not mass-spam Request Indexing — it is rate-limited and will not override a quality decision.
Step 6: Audit old-page traffic decay
Performance -> date range 28 days, compare to previous 28 days
-> sort by Page -> find URLs down 30%+Typical decay causes:
- Core update demotion
- Stale content (especially year-anchored titles like “Best … of 2023”)
- Competitors published newer, deeper content
Fix: refresh the content, add current data, and re-link it internally.
Before you panic, rule out reporting lag
The Pages and Performance reports run on a processing delay, and Google occasionally publishes data anomalies. As recently as late 2025 the Page indexing report froze for nearly a month (data stuck around Nov 21, 2025 until it caught up on Dec 18), and Google’s Data Anomalies page documented an impressions logging error that ran from May 2025 into late April 2026. Always check that page and give a fresh change 1-2 weeks before concluding the numbers are flat. A crawl spike followed by a few quiet days is also normal: Googlebot settles back to a steady rate once a catch-up is done.
Easy to misdiagnose
- Treating crawl as traffic: crawl is necessary, not sufficient. Nothing ranks until it is both indexed and matched to a real query.
- Expecting impressions to scale with crawl: the extra crawl might be entirely low-value URLs.
- Panicking at the spike itself: first check whether it is a post-fix re-crawl (mostly Refresh) or budget bleeding into low-value paths.
Prevention
- Route crawl budget to high-value URLs:
noindexor remove low-value ones, and keep parameter/faceted paths out of the crawl. - Never assume “more pages published = more impressions.” Every page must clear the unique-value bar before it indexes.
- Monitor all three numbers — crawl, index, impressions — together. An anomaly in any one of them is worth a look.
- Refresh older content on a schedule; do not only ship new pages.
FAQ
Q: Googlebot is hammering my server — can I throttle it in Search Console?
A: Not anymore. Google removed the manual crawl-rate limiter tool from Search Console on January 8, 2024. To slow Googlebot now, serve 500, 503, or 429 responses to its requests for a short period (a couple of hours up to 1-2 days) and it backs off automatically; do not keep returning errors longer than that or URLs can drop from the index. If you cannot serve errors, file Google’s special request to reduce crawl rate. For most small sites the real issue is too little crawling, not too much.
Q: Should crawl rate scale with traffic? A: Roughly, but with lag. New crawls pass through an indexing decision and then a ranking evaluation, so expect 4-12 weeks before crawl growth shows up as impressions — if it ever does, since low-value crawl never converts.
Q: Is a crawl spike followed by a few-day drop normal? A: Yes. Googlebot returns to a steady rate after a catch-up. What is worth investigating is a sustained spike with no movement in indexed pages or impressions.
Q: My new pages are stuck at “Crawled - currently not indexed” — does Request Indexing fix it? A: It nudges Google to re-evaluate, but it cannot override a quality decision. Fix the underlying thinness first (Step 5), then request indexing for a few priority URLs. Mass-requesting is rate-limited and does not help.
Q: I blocked the junk URLs in robots.txt but they are still indexed. Why?
A: robots.txt blocks crawling, not indexing — a disallowed URL can still be indexed from inbound links, just without its content. To remove it, let Google crawl it and serve a noindex tag (or a 410), then it drops on the next crawl.
Related
- Page indexed but zero impressions
- Discovered - currently not indexed
- Crawled - currently not indexed
Tags: #SEO #Google #Search Console #Indexing #Troubleshooting #Crawl budget #Impressions