What AI Crawlers and Googlebot See When Your Website Is Personalized

Our blog on why AI Overviews are changing what “good” landing page content looks like made the case that specific, localized content is easier for AI engines to cite than a page that tries to talk to everyone. A fair follow-up question, and one we hear a lot: if the page changes depending on who visits, which version does the crawler get?

It depends on the crawler. For most of them, the answer is the default version, which makes the default the most important piece of a personalized page. It's also the piece people tend to spend the least time on.

What Googlebot Sees

Googlebot renders JavaScript, so in theory it can see content that changes after the page loads. In practice, it has a location problem. Google's documentation on locale-adaptive pages says Googlebot's default IP addresses appear to be in the USA and that it sends requests without a language preference. Because of that, Google says it might not crawl, index, or rank every version of a page that changes by visitor location. Googlebot does crawl from some IP addresses outside the US too, and Google's guidance is to treat it like any other visitor from whatever country it appears to be in.

What does that mean? A variant built for people near Missoula will almost never be seen by Googlebot, because Googlebot isn't in Missoula. It sees what a US visitor with no other signals sees. If you're counting on a localized variant to help that page rank, don't. For content you need indexed by locale, Google's longstanding advice is still to give each version its own URL.

What AI Crawlers See

AI crawlers are a different story. Vercel and MERJ analyzed crawler traffic across Vercel's network, including OpenAI's GPTBot and Anthropic's ClaudeBot, and found that none of the major AI crawlers rendered JavaScript. They sometimes download JavaScript files, but they don't run them. Googlebot, by comparison, does full-page rendering.

GeoFli is built by Pintler Group, a digital marketing agency here in Missoula, and it works through a script on your page. That's what makes it easy to install without a developer, but it also means a crawler that doesn't run scripts gets the page as it exists before any personalization happens. That's your default.

One caution: the Vercel research dates to late 2024, and crawlers change. Treat it as a reason to test your own pages, not a rule that will always hold.

Why Your Default Matters More Than Your Variants

If AI engines mostly see the default, the default needs to be a complete page rather than a placeholder you meant to come back to. For a visitor we know nothing about, it should still:

• Say what you do and who it's for in the first screen, in plain text.
• Name every location or service area you cover, not just the primary one.
• Carry the facts that are true everywhere: how to book, what services you offer, typical hours, pricing ranges, the FAQ.
• Match your structured data. Schema markup should describe what's actually on the page.

Think of the variants as the polished layer for the people who match them, and the default as what everyone else gets, crawlers included. It's easy to end up with a carefully tuned Missoula variant sitting on top of a default that says little more than “Welcome to our website.” Guess which one the crawler reads.

Where Cloaking Comes In

Google's spam policies define cloaking as showing different content to users and to search engines with the intent to manipulate search rankings and mislead users. Intent is part of the definition. Showing a visitor in Billings the Billings hours is aimed at helping that visitor, not at gaming a ranking.

Where people get into trouble is the logic behind the swap. A few ways to stay on the right side of it:

• Personalize on the visitor, never on the requester. Don't write rules that look for Googlebot or GPTBot and serve them something else. Googlebot should be treated like any visitor from wherever it appears to be.
• Keep the same offer in every version. Different hours, address, phone number, testimonial, and headline are fine. 
• Don't hide from visitors what you show crawlers. If the default carries text that no human ever sees, that's a problem.

How to Check What Crawlers See on Your Site

You can run this in about fifteen minutes:

1. View the page source (right-click, then View Page Source, not Inspect). This is the HTML before any script runs, which is close to what a non-rendering AI crawler gets. Search it for your key phrases.
2. Turn JavaScript off in your browser and reload. Whatever's left is your default.
3. Use the URL Inspection tool in Google Search Console. Run a live test and open the tested page to see the rendered HTML Googlebot gets.
4. Check the variants from the visitor side, using a VPN or a colleague in another city, to make sure the right version shows up for the right audience.

Then compare. The default and the variants should read like the same business saying slightly different things to different people. If the default is thin, fix that first.

Personalization Doesn't Replace Location Pages

If you want to rank for “urgent care in Missoula,” you need a Missoula page with its own URL and its content in the HTML. Personalization is what you layer on top of that, tailoring the homepage, service pages, and landing pages once a visitor arrives. Our post on IP-based redirection and SEO covers the 301s, hreflang tags, and location-specific URLs side of this. Beyond Location: 7 Signals You Can Use to Personalize a Website covers what else you can personalize against besides where someone is.

The Takeaway

Googlebot mostly sees a US visitor's version of your page, and AI crawlers mostly see the version that exists before any script runs. Either way, the default is what gets read. Build it as a real page, keep the offer consistent across every version, and give any content you want indexed by location its own URL. Personalization can then do what it's supposed to do: make the page better for the person who's actually on it.