Your site has been live for a week or two and site:yourdomain.com returns nothing. Before you assume the worst, know that the site: operator is a sampled, incomplete view of Google’s index — it routinely hides new and low-traffic URLs. The authoritative check is URL Inspection in Google Search Console, so start there.
TL;DR — the fastest path
- Open Search Console -> paste your homepage URL into the URL Inspection bar at the top.
- Read the verdict. “URL is on Google” means you’re indexed (the
site:operator was just hiding it). “URL is not on Google” tells you exactly why —noindex, blocked by robots.txt, “Discovered/Crawled — currently not indexed”, or “Page with redirect”. - Click Test Live URL, then Request Indexing for that one URL. This is a hint, not a command — Google does not guarantee or speed up indexing because you asked.
- If you have no Search Console property yet, set one up first (it is the only place that tells you the real reason). See submit a sitemap in Search Console.
If URL Inspection already names a reason, jump to the matching cause below. If it just says the page is fine but site: is empty, you are almost certainly in the new-domain waiting period (cause 6) — give it 7–14 days.
Which bucket are you in?
| URL Inspection / report says | Most likely cause | Go to |
|---|---|---|
| ”Blocked by robots.txt” | Disallow: / is live | Cause 1 |
| ”Excluded by ‘noindex’ tag” | meta robots or X-Robots-Tag header | Cause 2 |
| ”Alternate page with proper canonical tag” | Canonical points elsewhere | Cause 3 |
”URL is on Google” but site: is empty | Sampling; you’re actually indexed | None — verify in cause 11 |
| ”Discovered — currently not indexed” | Crawl queued, often thin content | Causes 6, 7 |
| ”Crawled — currently not indexed” | Crawled but judged low-value | Cause 7 |
| Nothing in Search Console at all | No property / no sitemap | Cause 5 |
1. robots.txt is blocking everything
Visit https://yourdomain.com/robots.txt and confirm it does not contain:
User-agent: *
Disallow: /This single line blocks every crawler from every page. It is the most common and most catastrophic mistake, and it usually ships by accident from a staging config. A blank robots.txt, or one with only specific Disallow lines, is fine.
There is a subtle trap here: if robots.txt blocks a URL, Google can never fetch the page, which means it can never see a noindex tag on that page either. So a robots.txt block and a noindex tag are mutually exclusive fixes — solve robots.txt first.
2. Pages carry a noindex directive
noindex can hide in two places. Check both.
In the HTML. Right-click an article -> View Source -> search (Ctrl/Cmd+F) for:
<meta name="robots" content="noindex">In the HTTP response header. A meta tag is not the only way to set noindex; a reverse proxy, CDN, or framework can add an X-Robots-Tag header that View Source never shows. Check it from a terminal:
curl -I https://yourdomain.com/your-article/ | grep -i x-robotsNo output means no X-Robots-Tag header, which is the desired state. If you see X-Robots-Tag: noindex, that is your problem. When both a meta tag and a header are present, the most restrictive directive wins, so a stray noindex header overrides an index meta tag.
If noindex is present anywhere, nothing on that page gets indexed. Common causes:
- An Astro/Next/Hugo template that ships
noindexin dev or preview mode, left on after launch. - A theme or plugin adding
noindexsilently. - WordPress: Settings -> Reading -> “Discourage search engines from indexing this site” checkbox left ticked. Untick it, save, then re-request indexing.
3. Canonicals pointing to the wrong URL
A <link rel="canonical"> tells Google which URL is the “real” one. If article A’s canonical points to article B, Google indexes B and drops A — and in Search Console you’ll see A flagged as “Alternate page with proper canonical tag.”
- Each page should normally canonical to itself (its own clean, absolute URL).
- Bilingual sites: each language version canonicals to itself, then use
hreflangto link the two. Do not point the Chinese page’s canonical at the English one, or only one language gets indexed.
4. The site renders entirely in JavaScript
Disable JavaScript in your browser and reload one article. If the page goes blank, Googlebot can see a blank page too. Google does render JS, but it does so on a delayed second pass, which slows or skips indexing for new sites.
Fix: serve real HTML with static generation (SSG) or server-side rendering (SSR). Astro defaults to SSG, so the content is in the HTML on first load and is indexable out of the box. Confirm with View Source: your article text should be present even before JS runs.
5. Sitemap was never submitted (or shows zero)
In Search Console, open Indexing -> Sitemaps and confirm your sitemap reads “Success.” If it says “Submitted” but the discovered-page count stays at zero, that is a specific failure state — see Sitemap submitted but not indexed. For setting Search Console up from scratch, the submit sitemap walkthrough covers it end to end. A sitemap is not required for indexing, but for a new site it is the fastest way to hand Google your full URL list.
6. The new-domain waiting period
Brand-new domains routinely take time before pages appear, while Google decides you are not spam. As of June 2026, Google’s own guidance (John Mueller) is that good content on a new site is usually picked up within about a week, but the realistic range is anywhere from a few hours to a few weeks. There is no button that skips this.
While you wait, the two most common in-between states in Indexing -> Pages are Crawled — currently not indexed and Discovered — currently not indexed. Each has a different fix, so read the one that matches your report.
7. Thin or duplicate content
This is the cause that has grown most. As of 2025–2026, content quality — not technical access — is the dominant reason new pages stall at “Crawled — currently not indexed.” Google crawled the page, judged it low-value, and declined to index it. Guidance:
- Homepage: at least 300 words of real, specific content (not just a logo and a nav bar).
- Articles: enough genuine depth to answer the query better than what already ranks; roughly 800+ words for a real how-to, but length is a proxy for usefulness, not the goal.
- Never publish copied, spun, or raw machine-translated content. Near-duplicate pages compete with each other and often get dropped.
8. Weak internal linking
Crawlers follow links to discover pages. An article reachable only from the sitemap is a weak signal. Every page should be linked from at least a few others — its category page, related-article links, and breadcrumbs. Orphan pages (zero internal links pointing to them) are a frequent cause of “Discovered — currently not indexed.”
9. DNS or HTTPS problems
- No HTTPS: a valid TLS certificate is effectively expected; HTTP-only sites are heavily suppressed.
- Flaky DNS or intermittent 5xx errors: if Googlebot hits timeouts or server errors on repeated crawls, it backs off and crawls less. Check Settings -> Crawl stats in Search Console for a spike in failed requests.
10. Manual actions or security issues
Open Security & Manual Actions in Search Console. A manual action (for spam, hacked content, or sneaky redirects) can suppress an entire site. If anything appears, fix the cited issue and click Request Review — reviews can take days to weeks.
11. How to confirm it’s fixed
Don’t trust site: to confirm a fix; it lags and samples. Instead:
- In URL Inspection, click Test Live URL on a previously-blocked page. The “Indexing allowed?” line should now read Yes, and “User-declared canonical” should match the page’s own URL.
- Click Request Indexing. Note the daily cap: as of June 2026 the URL Inspection tool allows roughly 10–12 manual indexing requests per property per day (Google does not publish an exact number).
- Re-check Indexing -> Pages after a few days. Pages should move from the gray “Not indexed” group into the green “Indexed” group.
- The genuine confirmation is the page appearing for a
site:yourdomain.com/your-exact-url/query, or — better — for its real title.
FAQ
How long should I wait before worrying? For a brand-new domain, give it 7–14 days after submitting a sitemap before treating it as a problem. If URL Inspection says the page is indexable and you’re past two weeks with zero indexed pages, start working causes 1–3 and 7.
Search Console says “Crawled — currently not indexed.” Is that a bug? No. It means Google fetched the page and chose not to index it, almost always a content-value judgment as of 2026. Improve the page’s depth and uniqueness, add internal links to it, then request indexing again. See Crawled — currently not indexed.
Does clicking “Request Indexing” make Google index my page faster? It’s a hint, not a command. It can nudge a single URL into the crawl queue, but it does not guarantee or speed up indexing, and spamming it does nothing. Fix the underlying cause first.
My page looks fine in the browser but Search Console says it’s blocked. Why?
Two usual culprits: a noindex in the X-Robots-Tag HTTP header (invisible in View Source — check with curl -I), or a robots.txt rule blocking the path. Use Test Live URL to see exactly what Googlebot fetches.
Do I even need a sitemap? Not strictly — Google can find pages by following links. But for a new site with few backlinks, a sitemap is the fastest way to hand Google every URL at once, and it gives you a clean coverage report.
I’m on a free host. Can that block indexing?
Hosting tier doesn’t block indexing by itself, but some free/preview hosts add a noindex header or serve a holding page. Check the X-Robots-Tag header and confirm your real content is served on the production domain, not a *.preview or *.vercel.app subdomain you’d rather not index.