JavaScript rendering and whether your content is actually seen

This is the single most common structural cause of "our pages are not indexed and we cannot work out why." The content exists, the URL returns 200, the page renders perfectly for a human — and the initial HTML response contains a loading skeleton.

What each crawler actually does

Treating "crawlers" as one category is where the reasoning goes wrong. Their JavaScript behaviour differs enormously, and the gap has widened as AI answer engines have become significant traffic sources.

Google's two-phase model is the source of the false confidence. The first pass reads the HTML; rendering is queued separately and may happen minutes or weeks later. For a site with modest crawl demand, that render queue is where content goes to be forgotten.

Test what is served, not what is displayed

Opening a page in a browser tells you nothing, because the browser runs the JavaScript. You need to see the raw response before any script executes.

# The definitive check: what does the server actually return?
curl -s https://example.com/product/123 > raw.html

# Does the primary content exist in it?
grep -c "Add to basket" raw.html
grep -oE "<title>[^<]*</title>" raw.html
grep -oE '<link rel="canonical"[^>]*>' raw.html

# How much real text is there, with scripts and styles stripped out?
sed -e 's/<script[^>]*>.*<\/script>//g' \
    -e 's/<style[^>]*>.*<\/style>//g' \
    -e 's/<[^>]*>/ /g' raw.html | tr -s ' ' | wc -c

If that character count is a few hundred on a page that visibly contains a thousand words, the content is client-rendered and at risk. Cross-check with Search Console's URL Inspection tool, which shows both the raw HTML and Google's rendered version — the difference between them is precisely your exposure.

1. 01Fetch the raw HTML with curl and confirm primary content, title, description and canonical are present.
2. 02Run URL Inspection in Search Console and compare the HTML tab against the rendered screenshot.
3. 03Disable JavaScript in the browser and reload — a rough but fast approximation of a non-rendering crawler.
4. 04Check the Coverage report for "Crawled — currently not indexed" clustering on client-rendered templates.

The strategies, and what each costs

The decision is per route, not per application. A marketing page and an authenticated dashboard have entirely different requirements, and forcing one strategy across both is what creates the problem.

Note the last row. Client rendering is not wrong — it is wrong for content you want found. An internal dashboard behind a login has no indexing requirement, so its rendering strategy is purely a product decision.

The hybrid trap

Modern frameworks make it easy to render a page on the server and then move the important part to the client by accident. A single component that fetches its data in a browser-side effect can empty the part of the page that mattered.

// Server-rendered shell, client-rendered content. The product
// description — the thing you want indexed — is not in the HTML.
"use client";
export function ProductDetail({ id }) {
  const [product, setProduct] = useState(null);
  useEffect(() => {
    fetch(`/api/products/${id}`).then(r => r.json()).then(setProduct);
  }, [id]);

  if (!product) return <Skeleton />;        // <- what the crawler sees
  return <article>{product.description}</article>;
}

// Fetch on the server instead. The description ships in the HTML,
// and no loading state is ever rendered to a crawler.
export async function ProductDetail({ id }) {
  const product = await getProduct(id);
  return <article>{product.description}</article>;
}

Dynamic rendering, and why it is a last resort

Serving prerendered HTML to crawlers and client-rendered pages to humans works, and Google permits it provided both receive equivalent content. It is also a permanent second code path that drifts from the first, and a prerender service that fails silently takes your indexing with it.

It is a legitimate stopgap for a legacy single-page application that cannot be migrated soon. It is not a design to choose deliberately for a new build, because modern frameworks make server rendering the cheaper option.

The rule worth holding

Whatever a page is meant to be found for — its primary content, its title, its description, its canonical — belongs in the initial HTML response. Everything else can arrive later. A page that depends on a render queue is a page whose indexing you do not control.