On most sites, internal search result pages are the single biggest source of low-quality URLs leaking into Google. They are dynamically generated, infinite in number, near-duplicate of one another, and updated faster than crawlers can keep up. Google’s own documentation is direct about it: by default, internal search results should not be indexed. Here is the rule, the two exceptions where indexing actually wins traffic, and the exact steps to clean up search URLs that have already leaked in.
TL;DR
- Default to
noindex. Add<meta name="robots" content="noindex, follow">(or theX-Robots-TagHTTP header) to the search template. Let Google crawl freely so it can read the directive. - Do not
Disallow: /searchin robots.txt before thenoindexhas cleared. A blocked page is a page whosenoindexGooglebot can never see, so it stays indexed as a bare URL-only listing. - Two exceptions only: curated landing pages built from high-value queries, and stable e-commerce filter combinations with real demand. Everything else gets
noindex. - To clean up fast: add
noindex, then use the Search Console Removals tool with a URL prefix for a roughly six-month temporary hide while thenoindexpropagates. Without a recrawl, Google warns deindexing can “take months.”
Why search URLs leak in
An internal site search returns whatever the user typed: /search?q=react+hooks, /search?q=react%20hooks, /search?q=foo+bar+baz+qux. Every unique query is a new URL. Even on a small site, a crawler can discover thousands of these from internal links, sitemaps, and external referrers. Most have thin content (just a list of titles), duplicate each other (same query, different encoding), or carry garbage from spam queries. Google has told webmasters to keep these out of the index since 2007, and as of June 2026 that guidance is unchanged.
How to spot the problem
- The Search Console Page indexing report shows
/search?q=...URLs under “Crawled — currently not indexed” or “Duplicate, Google chose different canonical.” site:yoursite.com inurl:searchreturns hundreds of pages you never intended to publish.- A spammer has been pinging your search endpoint with junk queries, and those URLs are getting picked up.
- Your sitemap accidentally lists
?q=URLs because the URL-discovery script crawled the site.
Quick verdict
Default: noindex all internal search result pages. Add a robots noindex to the template, and let Google crawl freely so it can read the directive. Do not Disallow in robots.txt first. That prevents Google from seeing the noindex, and you can end up with URL-only listings. The exceptions are below.
The two exceptions
There are two cases where letting search pages into the index actually wins traffic.
Exception 1: curated landing pages. If you can pre-generate a small set of high-value queries as real pages — /search/react-hooks, /search/python-async — those are not really search results. They are landing pages with unique titles, descriptions, and edited copy. Treat them as full articles. The URL pattern looks search-like, but the content is curated.
Exception 2: e-commerce category-like queries. If your “search” is really a filter on a product catalog with stable inventory (/search?category=running-shoes&size=10), and there is real demand for that combination, you may want it indexed. But only the combinations that match real search intent, not every possible filter permutation.
Everything else: noindex.
Implementing noindex correctly
Use one of these. Google states both have the same effect, so pick whichever your stack makes easy.
<!-- In the search template, conditional on the page being a real query -->
<meta name="robots" content="noindex, follow"># Or via HTTP header (cleaner for non-HTML responses)
X-Robots-Tag: noindex, followMake sure the directive renders before any redirect or canonical tag. Crawlers fetch the page, read noindex, and drop it. Keep follow so links to real articles in the results still pass equity. The catch: Google only acts on the directive the next time it recrawls the URL, and the official docs warn that for low-priority pages “it may take months for Googlebot to revisit.” A passive noindex alone can leave thousands of search URLs lingering in the index for a long time.
Cleaning up URLs that already leaked in
If thousands of search URLs are already indexed, combine a permanent signal (noindex) with a fast temporary hide (the Removals tool). Here is how the three available levers compare:
| Method | Speed | Scope | Permanent? |
|---|---|---|---|
noindex on the template | Slow (waits for recrawl, can be months) | Per-URL, all matching pages | Yes, once recrawled |
| Removals tool, “Temporarily remove URL” | ~24 hours to hide | Single URL or a URL prefix like /search? | No — expires after ~6 months |
Disallow: /search in robots.txt | Saves crawl budget immediately | Pattern-wide | No — does not deindex on its own |
A clean cleanup sequence:
- Add
noindex, followto the search template so the durable signal is in place. - In Search Console, open Removals → Temporary Removals → New Request, choose “Remove all URLs with this prefix,” and enter your search path (for example
https://yoursite.com/search?). This hides every matching URL within about 24 hours. - Note the limit: the temporary removal lasts roughly six months. The
noindexyou added in step 1 is what makes the removal stick after that window expires, so do not skip it. - Watch the Page indexing report shrink over the next few weeks as Google recrawls and honors the
noindex. - Only after the
noindexhas cleared, optionally addDisallow: /search?to robots.txt to save crawl budget on future queries.
That order matters. Disallowing first locks Google out before it can read the noindex, which is the most common way sites accidentally cement these URLs in place.
Sitemap and internal link hygiene
A noindex on the page is the policy layer. Sitemaps and internal links are the discovery layer. If they keep pointing at search URLs, crawlers keep fetching them — wasted budget even when nothing ends up indexed.
- Audit your sitemap generator. If it walks the site looking for links, it will pick up
?q=URLs from the search box’s example queries and ship them in the sitemap. Filter the pattern out at build time. - Reduce in-template links to search. A persistent search box that submits to
/search?q=is fine; a “popular searches” list that hardcodes 10 example URLs into the footer is not. - After cleanup, run
site:yoursite.com inurl:searchweekly for a month to confirm the count is dropping. If it plateaus, find what still links to the URLs and remove the link.
Spam query protection
If your search endpoint reflects user input into the page title or H1, you have a free SEO injection vector. Spammers ping your search with their target keyword (often in Russian, Japanese, or pharma terms), and the resulting URL becomes a thin page on your domain with their keyword in the title — exactly the artifact they want indexed.
Two defenses, both cheap. First, escape and truncate the user query before rendering — never let it appear in <title> or H1 as raw text. Second, even with noindex on the search template, rate-limit the endpoint at the CDN. A single IP hitting /search?q= 500 times an hour is not a real user; return 429 and stop crawlers from chasing those URLs entirely.
Common mistakes
Disallow: /search?in robots.txt withoutnoindexfirst. Google cannot crawl the page, cannot read thenoindex, and the URL stays indexed as a URL-only listing.- Linking heavily to internal search from the homepage, header, or footer. Every link is a crawl invitation. If you
noindex, also reduce the internal link count. - Including
?q=URLs in your sitemap because a crawler-based generator picked them up. Filter them out at sitemap build time. - Forgetting that internal search referrers also appear in Search Console performance reports — those queries are searches on your site, not on Google.
- Allowing user-submitted queries to be reflected raw in the page title (
Search results for "spam-phrase-here"). Spammers exploit this for SEO injection.
FAQ
- Should I
noindexorcanonicalsearch results?:noindex. Canonical is for “this is the same content as another URL.” Search results are not duplicates of one master page; they are thin pages that should not be in the index at all. - How long until the pages actually drop?: With only a
noindex, Google has to recrawl each URL first, and the official docs say that can take months for low-priority pages. To force it, add the URL to URL Inspection and Request indexing, or use the Removals tool prefix to hide everything within roughly a day. - What does the Removals tool’s “six months” mean?: The temporary removal hides the URLs for about six months, then they can reappear unless a durable signal (
noindexor a 404/410) is in place. The Removals tool is for speed;noindexis what makes the change permanent. - What about pagination of search results?: Same rule.
noindexevery page. The pagination itself is also noise. - My site search is powered by Algolia / Meilisearch / client-side JS. Does any of this matter?: If the URL changes (
?q=) when a user searches, then yes — Google still sees the URL and tries to crawl it. If your search is entirely client-side with no URL change, you have nothing to noindex. - Will noindexing search hurt my SEO?: No. These pages were unlikely to rank for anything competitive. Removing them improves crawl efficiency and lifts the average quality of your indexed URLs.
- What about facet / filter URLs on an e-commerce site?: Same logic. Noindex by default. Whitelist a small set of high-traffic combinations as real pages with hand-written copy.
- Will Bing handle this the same way?: Bing follows the same
noindexandrobots.txtsemantics. The behavior is consistent across the major crawlers.