Infinite-scroll listing pages (homepage, blog index, product category) feel great to use but have a structural SEO problem: Googlebot does not scroll, click, or trigger scroll events. It loads the page, renders it once, snapshots the result, and leaves. Anything that only appears after a user-driven scroll is invisible to it.
Fastest fix (TL;DR): keep the infinite-scroll UX, but give every “page” of results its own crawlable URL with a ?page=n query parameter, server-render each of those URLs so the full list of 20 (or your page size) is in the raw HTML, self-canonical each page to itself, and list every page URL in your sitemap. Then re-submit the sitemap in Search Console. The infinite scroll stays for humans; the ?page=n URLs are how Google discovers items 21 and beyond.
This is exactly what Google recommends in its pagination and incremental-page-loading docs: “Give each page a unique URL. For example, include a ?page=n query parameter, as URLs in a paginated sequence are treated as separate pages by Google.”
Typical symptoms:
- Only the first ~20 articles ever get discovered; the rest sit in “Crawled - currently not indexed” or never appear in Search Console at all
- Articles 21+ are reported as “URL is unknown to Google” even though they’re live and linked in the UI
/blog?page=2,/blog?page=3are never crawled because they don’t exist as real, server-rendered pages
Which bucket are you in?
Run two quick checks, then jump to the matching cause.
| Check | Result | Most likely cause |
|---|---|---|
curl -sL https://yoursite.com/blog | grep -c "<article" | Returns ~20 (your first page size) but the live page shows hundreds | Scroll loading is JS-only (Cause 1) |
| Scroll past screen 1, watch the address bar | URL stays /blog forever | No crawlable pagination URLs (Cause 2) |
curl -sL "https://yoursite.com/blog?page=2" | grep -c "<article" | Returns 0 | ?page=2 exists but renders client-side only (Cause 3) |
Your “next page” link is /blog#page=2 | — | Hash/fragment URL (Cause 4) |
?page=2 returns 20 in curl, but Search Console says “unknown to Google” | — | Not in sitemap / not linked (Cause 5) |
Common causes
1. Scroll loading is JS-only, and Googlebot doesn’t scroll
// Typical React: trigger fetch when a sentinel scrolls into view
useEffect(() => {
const observer = new IntersectionObserver(([e]) => {
if (e.isIntersecting) loadMore();
});
observer.observe(sentinelRef.current);
}, []);Googlebot won’t run this loop, because it doesn’t generate scroll events. Its rendered snapshot contains only the initial 20 <article> tags. Google’s own lazy-loading guidance is explicit: load content “whenever it is visible in the viewport… The methods mentioned don’t rely on user actions, such as scrolling or clicking, to load content, which is important as Google Search does not interact with your page.”
So IntersectionObserver is fine for lazy-loading images that already exist in the viewport, but it is the wrong mechanism for loading new list items that only the user can reveal by scrolling.
How to confirm:
curl -sL "https://yoursite.com/blog" | grep -c "<article"
# e.g. 20 means Google's HTML snapshot only contains 20 items2. No crawlable pagination URLs
Many SPAs use a “Load more” button (or auto-scroll) that fetches data but never changes the URL — you stay on /blog permanently. Google has no way to request “page 2” because that URL doesn’t exist as a distinct address.
How to confirm: scroll past the first screen and watch the address bar. If it never changes from /blog, there are no pagination URLs to crawl.
3. Content first appears in a client-only render
Even if /blog?page=2 exists as a route, if that page also fetches its list inside useEffect (or any client-only data hook), Googlebot’s rendered HTML is still an empty list. You need server-side rendering (SSR) or static generation (SSG) so the 20 items are present in the initial HTML payload.
How to confirm: curl -sL "https://yoursite.com/blog?page=2" | grep -c "<article" returns 0 even though the page looks full in a browser.
4. Pagination URL uses a hash fragment (#page=2)
Google ignores everything after the #. From its perspective, /blog#page=2 and /blog#page=3 are all the same URL: /blog. The docs state it plainly: “Don’t use URL fragment identifiers (the text after a # in a URL) for page numbers in a collection. Google ignores fragment identifiers.” Use a query parameter (?page=2) or a path segment (/blog/page/2) instead.
5. Pagination URLs aren’t discoverable
Even if /blog?page=2 is server-rendered and returns a 200, Google won’t find it unless something points to it: a sitemap entry, or a real <a href> link from a crawlable page. Buttons wired only with JavaScript don’t count — Google follows <a href> links, not click handlers.
Shortest path to fix
Step 1: Give every infinite-scroll page a crawlable paginated URL
Keep the infinite-scroll UX, but update the URL as new pages load. Google explicitly endorses the History API for this in its lazy-loading docs (“Using the History API to update displayed URLs when chunks load”):
// Update the URL without a full reload as page 2 comes into view
window.history.pushState({}, '', '/blog?page=2');/blog?page=2 must also be a real, directly accessible URL that renders the same list on its own (SSR/SSG), so that a user who shares or refreshes the link — and Googlebot — gets the right content.
Step 2: Server-render the pagination URLs
Next.js (App Router):
// app/blog/page.jsx
export default async function BlogPage({ searchParams }) {
const page = parseInt(searchParams.page) || 1;
const PER_PAGE = 20;
const posts = await db.posts.findMany({
skip: (page - 1) * PER_PAGE,
take: PER_PAGE,
orderBy: { publishedAt: 'desc' },
});
return (
<>
{posts.map(p => <article key={p.id}><a href={`/blog/${p.slug}`}>{p.title}</a></article>)}
<nav>
{page > 1 && <a href={`?page=${page - 1}`}>Prev</a>}
<a href={`?page=${page + 1}`}>Next</a>
</nav>
</>
);
}In Astro, use getStaticPaths (or Astro’s built-in paginate() helper) to generate /blog/page/1, /blog/page/2, etc. as static files at build time.
Step 3: Add real prev/next links (an <a href>, not a button)
The single most important thing is that each page links to the next and previous pages with crawlable <a href> links — a numbered <nav> at the bottom is ideal:
<nav aria-label="Pagination">
<a href="/blog?page=1">1</a>
<a href="/blog?page=2">2</a>
<a href="/blog?page=3">3</a>
<a href="/blog?page=4" rel="next">Next</a>
</nav>You can still emit <link rel="prev"> / <link rel="next"> for other crawlers and accessibility, but note that Google stopped using these as an indexing signal in 2019 (“Google no longer uses these tags, although these links may still be used by other search engines”). Don’t rely on them — rely on the visible <a href> links above.
Step 4: List all pagination URLs in the sitemap
// scripts/generate-sitemap.mjs
const totalPosts = await db.posts.count();
const totalPages = Math.ceil(totalPosts / 20);
for (let i = 1; i <= totalPages; i++) {
urls.push(`https://yoursite.com/blog?page=${i}`);
}After deploy, re-submit the sitemap in Search Console (Sitemaps → enter the path → Submit). Google typically begins crawling the new URLs within a few days; for ecommerce catalogs you can also feed product URLs via a Merchant Center feed.
Step 5: Self-canonical every page
Each pagination URL must canonical to itself, not to page 1. Pointing /blog?page=2 at /blog tells Google “page 2 is a duplicate of page 1,” which is the fastest way to keep items 21+ out of the index. Google’s docs: “Don’t use the first page of a paginated sequence as the canonical page. Instead, give each page its own canonical URL.”
<!-- on /blog?page=2 -->
<link rel="canonical" href="https://yoursite.com/blog?page=2" />Step 6: Verify it’s fixed
Two checks — one local, one in Search Console.
# 1. Raw HTML for page 2 should already contain the full list
curl -sL "https://yoursite.com/blog?page=2" | grep -c "<article"
# Expect 20 (or your page size), not 0Then in Search Console, open URL Inspection, paste https://yoursite.com/blog?page=2, click Test Live URL, then View Tested Page → HTML, and confirm the rendered HTML contains all 20 items (Google: “Check the rendered HTML to make sure your content is in the rendered HTML”). If the list is there, click Request Indexing. Repeat-spot-check one or two deeper pages (e.g. ?page=5).
Prevention
- Every listing = infinite-scroll UX plus a crawlable
?page=n(or/page/n) URL set; the two coexist, they aren’t alternatives - Sitemap generation automatically appends new pagination URLs whenever content count crosses a page boundary
- Pagination URLs use query params or path segments — never a
#page=2hash fragment - Every page self-canonicals; nothing canonicals to page 1
- A weekly
curlspot-check confirms each pagination URL’s HTML still contains the full list (catches the day someone refactors the list back to client-only fetching)
FAQ
Will Googlebot eventually render the JavaScript and see the scrolled-in items?
Google does render JavaScript, but it renders the page once without scrolling, clicking, or firing scroll events — so content that only loads in response to a user scroll stays invisible. Rendering is also queued and not guaranteed on every crawl. Don’t depend on it; put the items in crawlable ?page=n URLs.
Is a “Load more” button better or worse than auto-scroll for SEO?
Slightly better, but only if the button is (or sits beside) a real <a href="?page=2"> link. A button wired purely with a JavaScript click handler is just as invisible to Google as auto-scroll. The reliable pattern is: button for UX, <a href> pagination links in the HTML for crawlers.
Do I still need rel=“next” / rel=“prev”?
Not for Google — it dropped them as an indexing signal back in 2019. They cost nothing and other search engines plus some assistive tech still read them, so emitting them is harmless, but they will not fix your indexing problem on their own. Crawlable <a href> links and self-canonicals do the actual work.
Should I build a single “View all” page instead of pagination?
You can, and Google says users often prefer single-page content — but only if that page loads fast and renders all items server-side. For a large catalog or blog with hundreds of items, a “View all” page is usually too heavy, so paginated ?page=n URLs are the safer default.
How long until items 21+ get indexed after I deploy the fix? Discovery usually starts within a few days of re-submitting the sitemap, but full indexing of every page can take a few weeks depending on your crawl budget and site authority. Use URL Inspection → Request Indexing on the most important pages to nudge it, and watch the Pages report in Search Console for the “Crawled - currently not indexed” count to fall.
Related
Tags: #SEO #Indexing #Troubleshooting