AI AI Tools Guidebook
Troubleshooting

What "URL is Unknown to Google" Really Means

Search Console says URL unknown. The URL exists, but Google has never seen it. Next steps.

Published: Author: AI Productivity Guide Team 🌐 查看中文版本

URL Inspection saying “URL is unknown to Google” looks scary but means something simpler than you’d think: the URL exists on your site, but Google has never encountered it. Not refused, not noindexed, not blocked by robots.txt — it has just never entered Google’s discovery pipeline.

This is the most common state for freshly published articles. Usually no panic needed — it’ll flip to “Crawled” within 3-14 days. But a few causes never resolve on their own.

Common causes

Google discovers URLs through three main pipes: sitemap, internal links, external links. Zero internal links + not in sitemap = permanent unknown.

How to confirm:

# Site-wide internal link search
rg 'href="/my-new-url/?"' src/
# Zero lines = no internal links

2. Not in sitemap.xml

The publish script forgot to add the new URL to the sitemap, or your sitemap is hand-maintained and someone forgot to edit it.

How to confirm:

curl -s https://yourdomain.com/sitemap.xml | grep -c "/my-new-url"
# Want >= 1

3. Sitemap was never submitted

Search Console’s Sitemaps section is empty or shows “Couldn’t fetch.” Google doesn’t know you have a sitemap at all.

How to confirm: Search Console → Indexing → Sitemaps. Check whether you’ve submitted, whether the status is “Success,” and whether “Discovered URLs” matches your actual article count.

4. Brand new site, Google hasn’t crawled the whole site yet

The domain only went live a few days ago. Site-wide crawling isn’t done. Even URLs in the sitemap may take a week to enter “Known.”

5. URL blocked by robots.txt

If Disallow: /article/, Google sees the URL in the sitemap but won’t fetch — and URL Inspection will display “Excluded by robots.txt,” not unknown.

If you see unknown, robots.txt isn’t the cause.

6. Special characters or encoding issues in the URL

Non-ASCII paths (Unicode characters in slugs or path segments) must be properly percent-encoded in the sitemap, or Google may not be able to fetch them.

How to confirm: URLs in the sitemap should be in percent-encoded form (e.g. /%E6%96%87%E7%AB%A0/...), not raw non-ASCII characters.

Shortest path to fix

Step 1: Tell Google via URL Inspection

Open Search Console → top search box → paste the full URL → wait for the result → click “Request indexing.”

This is the fastest “hey Google, here’s a URL” signal. Within ~10 minutes it flips from unknown to “URL is on Google” or “Crawled - currently not indexed.”

Daily quota is ~10 URLs. Save it for important pages.

Step 2: Add the URL to sitemap.xml

If you’re on Astro / Next / Hugo with auto-generated sitemaps, new pages enter automatically. If hand-maintained:

<url>
  <loc>https://yourdomain.com/articles/my-new-url/</loc>
  <lastmod>2026-05-21</lastmod>
  <changefreq>monthly</changefreq>
  <priority>0.8</priority>
</url>

After deploy, go to Search Console → Sitemaps and resubmit (or wait for the next ping).

Step 3: Add internal links from homepage / index / related

# Verify
rg 'href="/articles/my-new-url/?"' src/
# Want >= 3

At minimum:

  • Homepage “latest articles” section
  • /articles/ index page
  • 2-3 related articles’ “Related reading” section

Step 4: Submit the sitemap if you never did

Search Console → Indexing → Sitemaps → type sitemap.xml → submit.

Status should flip to “Success” within 1-2 minutes; “Discovered URLs” appears. If “Couldn’t fetch,” curl -I https://yourdomain.com/sitemap.xml should return 200.

Step 5: Wait 3-14 days

New sites without backlinks: 10-14 days is normal. During this window:

  • Re-check URL Inspection weekly; status migrates unknown → URL on Google
  • Don’t Request indexing the same URL daily — it doesn’t help and burns quota

Easy to misdiagnose

  • “Unknown” doesn’t equal “rejected”: Google has never seen it. Rejection isn’t even possible yet.
  • Expecting instant indexing after submission: from unknown to indexed usually takes 3-14 days. Established sites: 1-3 days. Brand-new sites: 2-4 weeks.
  • Thinking sitemap resubmission speeds things up: one submission is enough. Repeat submissions are no-ops.
  • Daily Request indexing wastes quota: once per URL is enough; save the rest for other pages.

Prevention

  • New-article publish script does three things: write the file, add to sitemap, add internal links from homepage / index
  • Build-time check: every article is referenced from at least 3 other files
  • Sitemap is always auto-generated; never hand-maintained
  • Within 24h of publishing an important URL, manually Request indexing once

Tags: #SEO #Google #Search Console #Indexing

Related Articles