The short answer: each AI assistant cites pages it can fetch and trust, but they find them in different ways. ChatGPT search uses OpenAI's OAI-SearchBot plus third party search providers. Claude's web search relies on Claude-SearchBot for indexing and Claude-User for live fetches. Gemini and Google AI Overviews both lean on Google's own search index, crawled by Googlebot, while the Google-Extended setting only controls Gemini training and grounding, not AI Overviews. To get cited, allow those crawlers in robots.txt, make sure you're properly indexed in Google and Bing, and publish clear, specific, quotable pages that answer the questions your customers actually ask.

I'm Rustam, and I run Frame the Pixel, a web design and WordPress studio in Kuala Lumpur. Since I started building AI ready websites, the most common question I get is "how do I get ChatGPT to recommend us?" Nobody can promise that. What you can do is remove every technical reason an AI system would skip your site, and give it pages worth quoting. Here's how, assistant by assistant.

How AI assistants choose what to cite

All four work in roughly the same two stages:

Retrieval. When a question needs current or specific information, the assistant runs a search, either against its own index or a partner's, and fetches a handful of pages.
Answering. The model reads those pages, writes an answer and links the ones it relied on.

A model may also "know" about your business from training data, but that knowledge is old and usually uncited. Citations come from retrieval. So the practical goal is simple: be in the indexes these tools search, be fetchable, and be the clearest answer on the page they fetch.

ChatGPT

How it finds sources

OpenAI says ChatGPT search uses third party search providers together with content from its own crawling and partners. When ChatGPT searches, it shows inline citations and a list of sources.

Which crawlers matter

OpenAI's crawler documentation lists three you should know:

OAI-SearchBot is used to surface sites in ChatGPT's search features. OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. This is the one to allow if you want citations.
GPTBot collects content that may be used to train OpenAI's models. Blocking it does not remove you from search. It's a separate decision.
ChatGPT-User fetches a page when a user's request needs it. OpenAI notes robots.txt rules may not apply to it because the request is user initiated, and that it isn't used to decide what appears in search.

OpenAI also says changes to robots.txt can take about 24 hours to take effect for search.

What helps

Because ChatGPT also draws on third party search results, a site that is well indexed in mainstream search engines has a head start. I'd set up Bing Webmaster Tools alongside Google Search Console and submit your sitemap to both. It costs nothing.

Claude

How it finds sources

When Claude searches the web, it fetches pages and cites them in its answer. Anthropic describes three separate robots in its crawler help article.

Which crawlers matter

ClaudeBot collects public web content that may be used for model training.
Claude-SearchBot indexes content to improve search results. Anthropic says disabling it may reduce your site's visibility and accuracy in user search results.
Claude-User fetches pages when a person asks Claude a question. Blocking it prevents Claude retrieving your content for those requests.

Anthropic says its bots honour robots.txt, including the non standard Crawl-delay directive, and warns that blocking by IP address may not work reliably because it stops the bot reading your robots.txt.

What helps

Allow Claude-SearchBot and Claude-User. Whether you allow ClaudeBot for training is your call and is independent of the other two.

Gemini

How it finds sources

Gemini grounds many answers in Google Search, which means it draws on the same index that powers Google's results. If you're not in Google's index, you're very unlikely to be cited by Gemini.

Which crawlers matter

There is no separate Gemini crawler. Pages are fetched by Googlebot. What Google does provide is Google-Extended, a product token you can use in robots.txt. Google's list of common crawlers explains that Google-Extended lets you control whether content Google crawls can be used to train future Gemini models and for grounding in Gemini Apps and certain other Google AI products. It does not affect your inclusion or ranking in Google Search.

So, if you block Google-Extended, you may reduce how Gemini uses your content. If you want Gemini to be able to ground answers in your pages, don't block it.

Google AI Overviews (and AI Mode)

How it finds sources

AI Overviews are part of Google Search. Google's guidance on AI features and your website is refreshingly plain: there are no extra requirements. To be eligible as a supporting link, a page must be indexed and eligible to appear in Search with a snippet. Google also says you don't need any special AI files or markup.

Which crawlers matter

Only Googlebot and the normal Google index. Google-Extended does not control AI Overviews. The controls that do apply are the ordinary search ones: nosnippet, data-nosnippet, max-snippet and noindex, which limit how your content appears in Search generally.

What helps

Everything that helps in classic search. Pages that rank well, answer the question directly and are trusted tend to be the ones that get linked. If you're worried AI Overviews are already eating your clicks, see has Google's AI Overviews cut your traffic?.

A quick summary of the crawlers

ChatGPT search: allow OAI-SearchBot. GPTBot is training only. ChatGPT-User is user fetches.
Claude: allow Claude-SearchBot and Claude-User. ClaudeBot is training.
Gemini: Googlebot fetches. Google-Extended controls training and Gemini grounding.
AI Overviews: Googlebot and the Google index. Normal snippet controls apply.

Step 1: Let the right crawlers in

An open wooden garden gate in a clipped hedge in morning light, representing letting AI crawlers in through robots.txt

Open your robots.txt file at yourdomain.com/robots.txt. Many WordPress security plugins, CDNs and hosts now block AI bots by default, sometimes without telling you. If you want citations, a minimal setup that allows search crawlers while keeping training optional might look like this:

User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Googlebot
Allow: /

Then check your firewall or CDN bot settings, because a block there overrides robots.txt. Look at your server logs for these user agents to confirm they're actually getting through.

Step 2: Get indexed properly

Verify the site in Google Search Console and Bing Webmaster Tools.
Submit an XML sitemap to both.
Check the Page indexing report for important pages marked "Crawled, currently not indexed" and fix thin or duplicate content.
Make sure key content is in the HTML, not only loaded by JavaScript after interaction or hidden behind tabs that require clicks.

Step 3: Make your facts easy to quote

Hands filing blank index cards into a wooden card catalogue drawer, like structuring business information so AI can cite it

AI answers are built from short passages. A page that buries the answer in paragraph eight loses to one that answers in the first two sentences. On your key pages:

Start with a direct answer to the question the page targets.
Use clear H2 and H3 headings phrased the way customers ask.
State facts plainly: what you do, where, for whom, your hours and how to contact you.
Keep your name, address and phone the same everywhere online.
Add LocalBusiness structured data and Organization markup. It won't force a citation, but it removes ambiguity about who you are.
Add an FAQ section with real questions from your enquiries.

Step 4: Give AI something worth citing

Assistants prefer sources that add something: original experience, specific numbers with sources, worked examples, clear comparisons. A generic "10 tips" page that repeats what's everywhere else has no reason to be cited over the dozen pages it copied.

For a small business that usually means writing about the questions only you can answer well. A Penang clinic explaining exactly what happens at a first appointment. A UK accountant explaining a specific deadline. This is the same advice as good SEO, which is no coincidence.

Step 5: Be mentioned elsewhere

Models judge trust partly by what other sources say. Reviews on your Google Business Profile, mentions in local news or industry directories, and accurate information on the platforms your customers use all help an assistant feel confident naming you.

What not to bother with

Tricks aimed at AI. Hidden text, keyword stuffing for "best X in Y" and fake reviews are spam in Google's eyes and easy for models to ignore.
Paying for "guaranteed AI rankings". Nobody can guarantee what an assistant says.
Special AI files as a shortcut. Google says none are needed for AI Overviews. I cover the most talked about one in what is llms.txt and does your website need one?.

Keep the site healthy

None of this works if the site is slow, broken or erroring when a crawler arrives. Bots give up on pages that time out. If your WordPress site is struggling, start with why is my WordPress site slow and the WordPress maintenance checklist.

How I can help

My AI ready websites service covers the technical side: crawler access, indexing, structured data and page structure that AI systems can read and quote. My search engine optimisation service covers the content and authority work that earns citations over time. I work with businesses in Malaysia, the UK, Europe and beyond, 9am to 5pm Malaysia time (GMT+8). Get in touch here or email contact@framethepixel.com.

FAQ

Can I pay to be cited by ChatGPT or Google AI Overviews?

No. Citations in answers are not sold. Ads in some AI products are separate and labelled.

Does blocking GPTBot stop ChatGPT citing my site?

No. GPTBot is for training. ChatGPT search uses OAI-SearchBot, and OpenAI treats them as independent settings.

Does blocking Google-Extended remove me from AI Overviews?

No. Google says Google-Extended doesn't affect Search. AI Overviews use the normal Google index, so only search controls such as nosnippet or noindex change that.

How long does it take to get cited by AI?

It depends on how quickly you're indexed and how competitive the question is. Fix access first, then expect it to take as long as normal SEO work, usually months rather than days.

How do I know if AI tools are citing my business?

Ask the assistants the questions your customers ask and look at the sources. Check your analytics for referrals from chatgpt.com, claude.ai and gemini.google.com, and check your server logs for the crawlers above.