How to check if AI can find your website

Before you start

You need a browser and your website address. That is all. Work through the steps in order: each one rules out a cause before the next.

If you want the machine-readable parts done for you, the free checker covers steps 3, 5 and 6 in a few seconds. The rest need a person.

Step 1: ask the assistants about you by name

Open ChatGPT, Claude, Gemini, Perplexity and Copilot. In each, ask:

What is [your business name] in [your town]? What do they sell?

Use a fresh chat, and sign out or use a private window if you can, so your own history does not help it.

What to look for:

• Does it know you exist? If it says it has no information, or describes a different business, the assistant cannot find or cannot read your site.
• Are the facts right? Wrong address, old prices or a discontinued service mean it is reading something out of date.
• Where did it look? Most assistants show their sources. If your own website is not among them, ask yourself why a directory was a better source than you.

Step 2: ask what your customers would ask

This is the test that matters. Write three questions a customer might type before they have heard of you. Do not include your name.

• "Who is a good [what you do] in [your area]?"
• "What is the best [your product type] for [a specific need]?"
• "Alternatives to [your best-known competitor]?"

Ask each assistant. Write down who gets named and which websites are cited. Ask each question two or three times, because answers vary.

If you appear, good. If not, open the cited pages. They show you what the assistant found useful: a clear description, a price, a comparison, a review. That is your to-do list, written by the assistant itself.

Step 3: read your robots.txt

Go to yourdomain.com/robots.txt. You are looking for two things.

A blanket block:

User-agent: *
Disallow: /

That tells every crawler to stay out. It is sometimes left over from when a site was being built.

AI crawlers named and blocked:

User-agent: GPTBot
Disallow: /

Blocking GPTBot or ClaudeBot only opts you out of training, which is a fair choice. Blocking OAI-SearchBot, Claude-SearchBot or PerplexityBot takes you out of those assistants' search answers. The full list of names and what each does.

If you see rules you did not write, a plugin, your host or your content delivery network added them. On two of my own domains, Cloudflare was adding AI-crawler blocks to a robots.txt that I had written to welcome them.

Step 4: check for blocks robots.txt does not show

A site can have a perfect robots.txt and still turn crawlers away. Firewalls and bot protection work at a different level.

• Look at your CDN or security dashboard. Cloudflare, for example, has settings for AI bots, managed robots.txt and bot fight mode. Check each.
• Look at your security plugin. Some block lists include AI crawlers.
• Look at your server logs, if you can get them. Search for OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot. A status of 200 next to them is good. 403 or 429 means they are being refused.

A quick test from a terminal, if you are comfortable with one. It asks for your home page while announcing itself as OpenAI's crawler:

curl -I -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" https://yourdomain.com/

HTTP/2 200 is what you want. This only shows how your site treats that name. Real crawlers come from known addresses, and some firewalls check those as well, so a pass here is encouraging, not conclusive.

Step 5: check you are in the search indexes

Assistants find pages through search. Go to Bing and Google and search:

site:yourdomain.com

• Plenty of results: you are indexed.
• Few or none on Bing: common, and worth fixing, since ChatGPT and Copilot draw on Bing. Set up Bing Webmaster Tools and submit your sitemap. How.
• Few or none on Google: check Google Search Console for the reason.

While you are there, check you have a sitemap at yourdomain.com/sitemap.xml, and that robots.txt mentions it.

Step 6: read your page the way a machine does

Right-click your home page and choose "View page source". This is what a crawler receives.

• Can you find your main text? Search the source for a sentence from your home page. If it is not there, your content is added by JavaScript after the page loads. A 2024 study by Vercel and MERJ found the major AI crawlers it tracked did not run JavaScript, so they would see an empty page.
• Is there a title and a description? Look for <title> and <meta name="description". They should say what you do, not just your name.
• Is there structured data? Search for application/ld+json. If you find it, read it and check the facts. What to add.
• Does the page ask not to be indexed? Search for noindex. If it is there and you did not intend it, that is your problem right there.

Step 7: check what llms.txt returns

Go to yourdomain.com/llms.txt. Three things can happen.

• A text file describing your site. Fine. Check it is still true.
• A "not found" error. Also fine. The file is optional.
• Your home page, or a normal-looking page. This is a fault: your server is saying "found it" for a file that does not exist. Three of my own sites did this. Fix the server so a missing file returns a proper 404.

More on llms.txt and whether it matters.

What the results mean

What you cannot check yourself

Some of it is out of reach without logs, tools or time:

• what each crawler actually received from your server over the past month;
• whether your facts match across every directory and profile;
• how often you appear across dozens of questions, tracked over time.

That is where a hand check helps.

Frequently asked questions

How do I know if ChatGPT can see my website?

Ask ChatGPT, with search switched on, what your business is and what it sells. If it answers accurately and cites your site, it can see you. If not, check that robots.txt allows OAI-SearchBot and ChatGPT-User, that no firewall blocks them, and that Bing has indexed your pages.

Why does AI give wrong information about my business?

Usually because it found an out-of-date source: an old directory listing, a previous address on a profile, or a stale page on your own site. Assistants repeat what they read. Find the wrong fact, correct it at the source, and make every listing agree.

Is there a free tool to check AI visibility?

Yes. The checker on this site tests robots.txt rules for AI crawlers, structured data, llms.txt, your sitemap and meta tags, free. It cannot tell you what assistants say about you. For that you ask them, as in steps 1 and 2.

My site ranks well on Google. Can AI still miss it?

Yes. A site can rank in Google while blocking OpenAI's or Anthropic's crawlers, or while being missing from Bing, which ChatGPT and Copilot draw on. Good Google rankings help with Google's own AI features and do not guarantee the rest.

How often should I check?

Once a month is enough for most businesses, and after any change to your hosting, security settings or website platform. Those changes are what most often break crawler access without anyone noticing.

Sources

• OpenAI, Overview of OpenAI crawlers. Checked 10 November 2026.
• Anthropic, Does Anthropic crawl data from the web?. Checked 10 November 2026.
• Perplexity, Perplexity crawlers. Checked 10 November 2026.
• Vercel and MERJ, The rise of the AI crawler, 17 December 2024. Crawler behaviour may have changed since.
• My own checks on 11 websites, 10 November 2026: the case study.