GPTBot, ClaudeBot, and PerplexityBot don't execute JavaScript. Verified 2026 data on what AI crawlers actually see on a client-rendered page, and how to fix it.
Googlebot can run your JavaScript. GPTBot, ClaudeBot, Meta-ExternalAgent, and PerplexityBot can't. Vercel's own engineering team checked this directly against its network traffic: GPTBot generated 569 million requests in a single recent month, and across that traffic Vercel found no evidence of JavaScript execution, even though GPTBot downloads your .js files on about 11.5% of its requests. It just never runs them.
That gap is not a robots.txt problem or a missing-markup problem. It is a rendering problem, and it means a client-heavy single-page app can rank on Google, show up in Gemini, and still be functionally invisible to ChatGPT, Claude, and Perplexity, three products a growing share of your prospective readers now use instead of a search box. This post covers why the gap exists, what these crawlers actually receive from a JS-rendered page, how to check your own site, and how to close it without rewriting your frontend.
Googlebot processes a JavaScript page in three phases: crawl, render, index. Every other major AI crawler stops after phase one. That single architectural difference is the whole story, and it is why the same page can rank in Google while returning nothing to ChatGPT.
Google's own documentation lays out the pipeline plainly: Googlebot first crawls a URL and parses the raw HTML for links, queues the page for rendering, then hands it to "an evergreen version of Chromium" that executes the JavaScript before the final HTML gets indexed. Google is explicit that the render step is not instant: "The page may stay on this queue for a few seconds, but it can take longer than that," depending on how busy Google's rendering resources are at the time, per Google's JavaScript SEO basics guide.
That queue is expensive. Running a full headless browser at Google's scale, for every JS-heavy page on the web, is a capability only a company with Google's crawling infrastructure has built and maintained for over a decade. Most other crawlers, including every AI crawler covered here, skip it entirely and index the raw HTML your server returns on the first fetch.
Vercel's engineering team put this in one sentence after analyzing crawler traffic across its network: "none of the major AI crawlers currently render JavaScript," naming OpenAI's GPTBot, OAI-SearchBot, and ChatGPT-User, Anthropic's ClaudeBot, Meta's Meta-ExternalAgent, ByteDance's Bytespider, and Perplexity's PerplexityBot in the same finding. Every one of those crawlers fetches HTML once and moves on. There is no queue, no headless browser, no second pass.
OpenAI's own bot documentation backs this up by omission: it describes what GPTBot, OAI-SearchBot, and ChatGPT-User are each for, training, search citations, and user-triggered fetches, without claiming JavaScript execution for any of them. Perplexity's documentation does the same for PerplexityBot: fetch, index, respect robots.txt and published IP ranges, nothing about running scripts. When a vendor documents exactly what a crawler does and stays silent on rendering, that silence is the answer.
Google-Extended, the crawler that feeds Gemini, is not a separate bot with its own infrastructure. Google's documentation on common crawlers states that "crawling is done with existing Google user agent strings," meaning Google-Extended is a robots.txt control token layered onto Googlebot's own crawl, not a standalone fetcher. Because it rides Googlebot's infrastructure, Gemini's training and grounding pipeline inherits full JavaScript rendering, the same headless Chromium pass described above.
That is the whole reason a client-rendered SPA can look fine on Google and Gemini while going dark everywhere else. Google-Extended gets the rendered DOM because it is Googlebot in a different hat. GPTBot, ClaudeBot, and PerplexityBot get whatever your server sent before your framework's hydrate() call ever runs.
A GPTBot request against a client-rendered React or Vue app gets exactly what curl gets: the initial HTML payload, usually a near-empty <div id="root"></div> plus your <head> tags and a pile of <script> references. No headline, no body copy, no product description. Everything your framework injects at runtime after the JavaScript executes simply does not exist from the crawler's point of view.
This is not a theory about crawler behavior, it is a measured outcome. Vercel's analysis of its own network traffic found no trace of GPTBot executing a script anywhere in it, and the pattern holds for Anthropic's crawler too: Claude's crawler requests JavaScript files on 23.84% of its fetches, and images on 35.17%, without executing a single one. Both crawlers are pulling the same assets a browser would, they just never run the part that turns those assets into visible content.
Scale that against how much of this traffic exists at all. GPTBot generated 569 million requests and Claude's crawlers 370 million across Vercel's network in a recent one-month window, against 4.5 billion for Googlebot. Add Apple's crawler at 314 million requests and PerplexityBot at 24.4 million, and the four combined run at roughly 28% of Googlebot's volume. AppleBot is the outlier in that group, worth calling out precisely because it breaks the pattern: Vercel's data shows it renders JavaScript through a browser-based crawler similar to Googlebot's, so it sits outside the gap this post is about. Strip AppleBot out and GPTBot, Claude, and PerplexityBot alone, all three confirmed non-rendering, still account for a large and fast-growing slice of your crawl traffic reading a version of your page that may contain none of your actual content.
The traffic keeps shifting toward these crawlers, too. Independent analysis of an SPA-heavy site's server logs found ChatGPT-User making 3.6x more requests than Googlebot (133,361 versus 37,426) across 69 sites and more than 78,000 pages, with OpenAI's combined crawlers running at 3.8x Googlebot's volume. The same dataset put ChatGPT-User's average response time at 11ms against Googlebot's 84ms, and a 99.99% success rate against Googlebot's 96.3%, with Googlebot alone absorbing 624 blocked (403) responses and 480 not-found (404) errors chasing stale URLs. ChatGPT-User is fast, persistent, and everywhere; it is just not looking at your rendered DOM.
And the crawlers behind that traffic keep reshuffling. A monthly analysis of Cloudflare Radar's AI crawler data shows ClaudeBot's share of identified AI crawler traffic jumping from 12.1% to 20.0% in a single month, a 66% relative increase that made it the #2 AI crawler outright, ahead of Meta-ExternalAgent at 10.2% and GPTBot at 9.6% individually. ClaudeBot alone now out-crawls Meta and GPTBot combined. If your rendering gap is only costing you GPTBot visibility today, it is a growing problem, not a static one.
The most telling detail in Vercel's dataset is that these crawlers request your JavaScript bundles anyway. GPTBot pulls .js files on about 11.5% of its fetches; Claude's crawler pulls them on 23.84%. They are not ignoring your scripts out of indifference, they are fetching the file and discarding it unread, the same way you might download a PDF you never open. That behavior only makes sense if these crawlers have no execution engine at all, just an HTTP client and a parser for the raw response.
Worth separating from all of this: none of it is a permission problem. Google is explicit that appearing in its own AI features requires no special file or markup: "You don't need to create new machine readable files, AI text files, or markup to appear in these features. There's also no special schema.org structured data that you need to add," per Google's AI features documentation. The gap covered in this post is upstream of markup entirely, it is whether the crawler can see your text at all before it gets to judging your structured data. Access control is a separate layer, covered in our robots.txt for AI bots guide; this post is about what happens after a crawler is let in the door.
Don't guess. The fastest way to know whether GPTBot, ClaudeBot, or PerplexityBot can read your content is to request the page as they would and read the raw response, the same thing they get.
Run this against any page you care about:
curl -sS -A "GPTBot" https://yourdomain.com/your-page/ | less
curl -sS -A "ClaudeBot" https://yourdomain.com/your-page/ | less
curl -sS -A "PerplexityBot" https://yourdomain.com/your-page/ | lesscurl never executes JavaScript, which is exactly what makes it accurate here: it shows you the same HTML a non-rendering crawler receives. Search the output for your headline, your main body copy, and your key facts. If they are missing and all you see is a shell <div> and a list of <script src> tags, that is your rendering gap, confirmed against real output instead of assumed from a framework's reputation.
We run this exact check against our own blog before every post goes out, this one included. curl -sS -A "GPTBot" https://www.trylyra.ai/blog/javascript-rendering-ai-crawlers/ returns the full article body and headline, because Lyra statically generates every post at build time and ships it as plain server-rendered HTML, no client-side hydration standing between GPTBot and the words on the page. Run that same command yourself and you should see the sentence you're reading right now in the raw response.
Open the same page in Chrome, open DevTools, and inspect the Elements panel, which shows the DOM after your JavaScript has run. Compare it against the curl output side by side. Anything present in DevTools but absent from the curl response is content a human sees and GPTBot, ClaudeBot, and PerplexityBot do not. It's the same raw-response-versus-rendered-DOM comparison SEOs have long used to debug what Googlebot sees, applied here to a set of crawlers that never get past the raw-response side of that comparison at all.
On a client-rendered React or Vue page, this comparison usually plays out the same way: the curl response comes back with an empty <main> or <div id="root"> and a stack of <script src> tags, while the DevTools Elements panel a few tabs over shows the full, rendered article. Seeing that gap side by side, on your own URL, is usually what convinces a team the problem is real. It's a five-minute check, not a research project.
The fix is not "remove JavaScript." It is making sure the meaningful content exists in the HTML your server sends on the first response, before any script runs, while keeping the interactive parts of your app exactly as interactive as they are today.
The pattern that works across frameworks is the same one behind server-side rendering generally: render the article body, headings, and key facts to static or server-generated HTML, and let JavaScript handle only the interactive layer on top, a comment widget, a filter, a live search box. GPTBot and ClaudeBot get your content because it is already in the HTML. Your users still get the interactivity because the client-side code runs exactly as it always did, it is just no longer the only place your content lives.
This is the same underlying variable covered in our Astro vs Next.js SEO comparison: how much of the page is server-rendered content versus client-hydrated shell. That post is about Core Web Vitals; this one is about a second, separate cost of getting that variable wrong, invisibility to an entire category of crawler that Core Web Vitals scores don't measure at all.
The right move depends on what you're running:
"use client" when it doesn't need interactivity, or fetching your actual article content inside a client component instead of a server one. Audit for that pattern first; it's the most common way a Next.js site accidentally recreates an SPA's rendering gap one component at a time.client:* directive. If a client-rendered gap exists on an Astro site, it's usually because a content section got wrapped in an interactive island that didn't need to be one.If you're generating pages programmatically at scale, thousands of URLs from a database or API, the rendering gap compounds: one missed server-render pattern in a template can make every page it generates invisible to GPTBot at once. Our programmatic SEO in Next.js and Astro guide covers building those pages so the content ships in the initial HTML from the start rather than retrofitting it across thousands of routes later.
Once you've shipped the change, run the exact same curl commands from earlier and confirm your headline and body copy now appear in the raw response, no browser, no rendering, just the HTTP payload. That's the whole verification loop: curl before, curl after, diff the two. If your content shows up in the unrendered HTML, GPTBot, ClaudeBot, and PerplexityBot will see it on their next fetch. If you'd rather have a second pair of eyes run that diff against your own repo, tell us about your blog and we'll walk through what GPTBot actually sees on it today.
Fixing the crawler's access to your content is the prerequisite, not the finish line. Once a page is readable, whether it actually gets picked as a citation depends on how clearly it answers the question someone asked, which is what our guide to ranking in ChatGPT and our engine-by-engine playbook for getting cited by ChatGPT, Perplexity, and Claude both cover. And once you've made the fix, AI citation tracking is how you confirm it actually moved the needle instead of just trusting that it did.
Fixing a JavaScript rendering gap only matters if the content GPTBot and ClaudeBot can now see is also worth citing, and that's the other half of what Lyra writes: server-renderable, source-backed posts shipped as a pull request you review.
FAQ
No. Vercel's analysis of its own network traffic, where GPTBot generates hundreds of millions of requests a month, found zero evidence of JavaScript execution. GPTBot downloads .js files on about 11.5% of its requests but never runs them, so it only ever sees the raw HTML your server returns.
No. ClaudeBot requests JavaScript files on 23.84% of its fetches and images on 35.17%, but Anthropic's crawler does not execute any of it. If your content only appears after a client-side render, ClaudeBot never sees it.
Perplexity's own documentation describes PerplexityBot as a fetch-and-index crawler with no mention of JavaScript execution, and independent crawler analysis places it alongside GPTBot and ClaudeBot as HTML-only. Treat it the same way: raw HTML in, nothing else.
Because Googlebot and every major AI crawler are built differently. Googlebot fetches your page, queues it for a headless Chromium render, executes your JavaScript, and indexes the result. GPTBot, ClaudeBot, Meta-ExternalAgent, and PerplexityBot skip the render step entirely and index whatever HTML your server sent on the first request.
Google-Extended has no separate crawling infrastructure of its own: Google's documentation confirms it crawls using existing Googlebot user-agent strings, and Gemini's crawling runs on that same Googlebot infrastructure, including full JavaScript rendering. That makes it the one AI-linked crawler that can see a client-rendered page the way Google Search does.
Built by the tool you're reading about
Lyra finds the topics worth ranking for, writes them in your repo's voice, fact-checks every claim, and opens a pull request scored and ready to merge. You review and hit merge. Want to see what she'd write for you? Tell us about your blog and the founder will walk through it with you.
Keep reading
AI content pipeline security means guarding against prompt injection: hostile source data hijacking an AI writer to leak keys or publish false claims.
Multi-agent content review pairs a generator with an independent critic and judge so mistakes get caught before a human ever opens the pull request.
Claude API cost per blog post, broken down into research, draft, review, and iteration tokens, at 2026 Anthropic pricing, to budget the BYOK model.