AI AI Tools Guidebook
Troubleshooting

Infinite Scroll Pages Don't Get Indexed

First 20 items render, the next 100 load via JS — Googlebot never sees them.

Published: Author: AI Productivity Guide Team 🌐 查看中文版本

Infinite-scroll listing pages (homepage, blog list, product category) have great UX but a fatal SEO problem: Googlebot doesn’t scroll. It opens the page, takes the initial HTML, and leaves. Items 21+ loaded via JS are invisible. Result:

  • Only the first 20 articles get discovered
  • Articles 21+ remain “URL unknown to Google” even though they exist
  • Pagination URLs (/blog?page=2, /blog?page=3) won’t be crawled if they don’t exist as static pages

The fix’s core idea: add an SEO-fallback pagination URL set alongside the infinite scroll.

Common causes

1. Scroll loading is JS-only, Googlebot doesn’t scroll

// Typical React: trigger fetch when sentinel scrolls into view
useEffect(() => {
  const observer = new IntersectionObserver(([e]) => {
    if (e.isIntersecting) loadMore();
  });
  observer.observe(sentinelRef.current);
}, []);

Googlebot won’t run this observer. Its HTML snapshot only has the initial 20 <article> tags.

How to confirm:

curl -sL "https://yourdomain.com/blog" | grep -c "<article"
# e.g., 20 = Google only sees 20

2. No crawlable pagination URLs

Many SPAs’ “Load more” button never updates the URL — you’re permanently on /blog. Google has no way to visit “page 2” because the URL doesn’t exist.

How to confirm: Watch the address bar as you scroll past the first screen. Still /blog? No pagination URLs.

3. Content first appears in client-only render

Even if /blog?page=2 exists, if that page also fetches data in useEffect, Google still sees an empty list. You need SSR / SSG.

4. Pagination URL uses hash (#page=2)

Hash URLs (fragment identifiers) aren’t crawled. To Google, everything after # is the same page.

5. Pagination URLs aren’t in the sitemap

Even if /blog?page=2 is server-rendered, Google won’t discover it without a sitemap entry or links from other pages.

Shortest path to fix

Step 1: Give every infinite-scroll page a paginated fallback URL

UX stays infinite-scroll, but the URL changes too. As you scroll to load page 2:

// Use History API to update URL without reloading
window.history.pushState({}, '', `/blog?page=2`);

And /blog?page=2 must be a real accessible URL, independently SSR / SSG-rendered.

Step 2: Server-render the pagination URLs

Next.js:

// app/blog/page.jsx (App Router)
export default async function BlogPage({ searchParams }) {
  const page = parseInt(searchParams.page) || 1;
  const PER_PAGE = 20;
  const posts = await db.posts.findMany({
    skip: (page - 1) * PER_PAGE,
    take: PER_PAGE,
    orderBy: { publishedAt: 'desc' },
  });
  return (
    <>
      {posts.map(p => <article key={p.id}><a href={`/blog/${p.slug}`}>{p.title}</a></article>)}
      <nav>
        {page > 1 && <a href={`?page=${page - 1}`}>Prev</a>}
        <a href={`?page=${page + 1}`}>Next</a>
      </nav>
    </>
  );
}

In Astro, use getStaticPaths to generate /blog/page/1, /blog/page/2, etc. as static files.

Step 3: Add rel=“next” / rel=“prev” (Google de-emphasized these post-2019 but they still help)

<link rel="prev" href="/blog?page=1" />
<link rel="next" href="/blog?page=3" />

Or simpler: a <nav> at the bottom with prev/next/numbered page links (real <a href>, not button + JS).

Step 4: List all pagination URLs in the sitemap

// scripts/generate-sitemap.mjs
const totalPosts = await db.posts.count();
const totalPages = Math.ceil(totalPosts / 20);
for (let i = 1; i <= totalPages; i++) {
  urls.push(`https://yourdomain.com/blog?page=${i}`);
}

After deploy, resubmit sitemap in Search Console. Google crawls them one by one within days.

Step 5: Handle canonicals correctly

Each pagination URL self-canonicals. Don’t point /blog?page=2 canonical to /blog — that prevents page 2 content from being indexed.

<!-- on /blog?page=2 -->
<link rel="canonical" href="https://yourdomain.com/blog?page=2" />

Step 6: Verify

# Static HTML should show the page N list
curl -sL "https://yourdomain.com/blog?page=2" | grep -c "<article"
# Want 20 (or your page size)

In Search Console URL Inspection, enter /blog?page=2 → “Test live URL” → check the screenshot for a complete list.

Prevention

  • Every listing design = infinite-scroll UX + paginated URL SEO fallback, both coexisting
  • Sitemap generation auto-includes new pagination URLs when content is added
  • Pagination URLs use query params (?page=2) or path segments (/page/2) — server-rendered either way
  • Never use #page=2 hash URLs
  • Weekly curl spot-check of pagination URLs to confirm the HTML contains the full list

Tags: #SEO #Indexing #Troubleshooting

Related Articles