# What is llms.txt, and does it actually work? > What llms.txt is, how to write one, and what the evidence says about whether ChatGPT, Claude or Google read it. An honest guide with a working example. By Reece Chown, Agent SEO HQ. Published 11 November 2026. Canonical URL: https://agentseohq.com/articles/what-is-llms-txt **The short answer:** `llms.txt` is a Markdown file at the root of your website that gives AI agents a short, curated guide to your site. It is a proposal, not a standard any AI company has committed to. Google says its search does not use it, and a 2026 Ahrefs study found 97% of these files received no requests in a month. It takes ten minutes to write and does no harm, so add one, but do it last. ## What llms.txt is `llms.txt` is a plain text file, written in Markdown, that lives at `https://yourdomain.com/llms.txt`. It tells an AI agent what your site is and where the useful pages are, without the navigation, adverts and scripts that surround them in HTML. It was proposed by Jeremy Howard in September 2024, and the specification at llmstxt.org was revised in August 2026. The proposal says the file is mainly meant for use at "inference" time: when an agent is helping someone and wants to understand a site quickly, as opposed to when a model is being trained. A common mix-up: llms.txt is not robots.txt for AI. It grants and denies nothing. | Aspect | robots.txt | llms.txt | |---|---|---| | Purpose | Says which crawlers may fetch which pages | Describes the site and points to its best pages | | Format | Its own simple rule format | Markdown | | Status | An internet standard (RFC 9309) | A community proposal | | Followed by AI companies? | Yes, by the major crawlers, with exceptions for user-triggered fetches | No public commitment from any of them | To control access, use robots.txt. [That guide is here](https://agentseohq.com/articles/ai-crawlers-robots-txt). ## The format The specification is short. In order: 1. An `# H1` with the name of the site or project. This is the only required part. 2. A blockquote with a short summary. 3. Optional paragraphs or lists with more background. No headings in this part. 4. `## H2` sections containing lists of links, each written as `[name](url)`, optionally followed by a colon and a note. 5. By convention, a section called `## Optional` for links an agent can skip if it is short of space. Here is a complete example for an imaginary accountancy firm: ```markdown # Harbour Accounts > Harbour Accounts is an accountancy firm in Leeds for small design > studios and freelancers. Fixed monthly fees from £95 plus VAT. Key facts: - Services: bookkeeping, year-end accounts, VAT returns, payroll. - Area: Leeds and West Yorkshire in person, the rest of the UK remotely. - Not offered: audits, and personal tax for non-clients. - Contact: hello@harbouraccounts.example or 0113 496 0000. ## Pages - [Services and prices](https://harbouraccounts.example/services): what each monthly plan includes - [About the team](https://harbouraccounts.example/about): who we are and our qualifications - [Contact](https://harbouraccounts.example/contact): how to book a first call ## Guides - [VAT for freelance designers](https://harbouraccounts.example/guides/vat-for-designers): when to register and what to charge ## Optional - [Privacy policy](https://harbouraccounts.example/privacy) ``` The specification also suggests offering a Markdown copy of each important page at the same address with `.md` added. Many sites publish an `llms-full.txt` as well: the full text of the site in one file. That name is a widespread convention, not part of the specification itself. ## Does anything actually read it? This is where an honest article parts company with most of the ones you will find. Here is the evidence as of November 2026. **Google says no.** Its guidance on generative AI features states: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search." Making one for other services is fine, it adds, but it will not affect your visibility in Search. **OpenAI and Anthropic have not said yes.** Neither company's crawler documentation mentions llms.txt. Anthropic publishes an llms.txt for its own documentation, which shows it finds the format useful for developers, not that Claude's search consults yours. **The largest study found very little traffic.** In June 2026 Ahrefs looked at 137,210 websites. About 28% had an llms.txt. Of those files, 97% received no requests at all during May 2026. Where files were fetched, 96% of requests came from bots, but AI bots were only about a fifth of that. SEO audit tools fetched llms.txt more often than any single AI crawler. The authors called the file "largely decoration" for AI search today. One caveat they give themselves: their sample skews towards SEO-aware sites, so real adoption is probably lower than 28%. **Coding tools are the exception.** Developer tools and coding agents do look for llms.txt and llms-full.txt when pointed at documentation. In the Ahrefs data, Anthropic's Claude Code was among the more active fetchers. If you publish software documentation, the file earns its keep. My own experience fits. I have added llms.txt to 11 of my own sites. I cannot show you that it changed a single answer, and I will not pretend otherwise. ## So why add one? - **It is cheap.** Ten minutes for a small site. - **It is harmless.** Google says it has no effect on Search, in either direction. - **Agents are changing fast.** A file that is ignored today may be read next year, and coding agents already read it. - **Writing it is useful in itself.** If you cannot describe your business in a blockquote and six bullet points, your home page probably cannot either. The risk is not the file. The risk is paying someone for the file and thinking the job is done. If an agency leads with llms.txt, ask what else is on the list. ## How to write a good one 1. **Start with facts an assistant would need to answer a question about you**: what you sell, who for, where, prices, how to buy, what you do not do. 2. **Keep it true.** Copy prices and facts from the live site. A stale llms.txt is worse than none, because it is a clean, confident source of wrong information. 3. **Link to your best pages, not all of them.** It is a guide, not a sitemap. 4. **Generate it if you can.** On my sites, llms.txt is built from the same content as the pages, so the two cannot drift apart. This site's [llms.txt](https://agentseohq.com/llms.txt) is made that way. 5. **Serve it properly.** It should return status 200 with a `text/plain` or `text/markdown` content type. ## One mistake to check for today Open `yourdomain.com/llms.txt` in a browser. If you see your home page, or a "not found" page that looks like a normal page, your server is answering "yes, here it is" to a file that does not exist. Three of my own sites did this. A machine has to read the whole response to discover it has been handed the wrong thing. Either publish a real file or make the address return a proper 404. The [free checker](https://agentseohq.com/check) tests for this. **Want to know how AI assistants see your website?** Send me your address and I will check it by hand: what the assistants can read, what they say about you, and what to fix first. It is free, and you get a fixed quote before any paid work. [Get a free Agent SEO check](https://agentseohq.com/contact) or [try the instant checker](https://agentseohq.com/check). ## Frequently asked questions ### Is llms.txt an official standard? No. It is a community proposal published at llmstxt.org by Jeremy Howard in September 2024. It has no standards body behind it, and no major AI company has publicly committed to reading it for search answers. ### Does Google use llms.txt? No. Google's guidance says you do not need to create machine-readable AI text files to appear in Google Search, including its AI features, and that doing so will not affect your visibility there. ### Does ChatGPT read llms.txt? OpenAI has not said that ChatGPT's search uses it, and its crawler documentation does not mention the file. Ahrefs' 2026 study did record some requests for llms.txt from OpenAI's crawlers, but they were a small share of a small total. ### What is the difference between llms.txt and llms-full.txt? llms.txt is a short guide with links to your most useful pages. llms-full.txt is a common companion file containing the full text of the site in one Markdown document. Only llms.txt is defined in the proposal. ### Will llms.txt hurt my SEO? No. Google says the file has no effect on Search. The only real risk is letting it go out of date, so that it states old prices or facts. ## Sources - [llmstxt.org](https://llmstxt.org/), the llms.txt proposal. Published 3 September 2024, revised 10 August 2026. - Google Search Central, [Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide). Checked 10 November 2026. - Ahrefs, [We analyzed 137K sites: 97% of llms.txt files never get read](https://ahrefs.com/blog/llmstxt-study/), Louise Linehan, 15 June 2026. - OpenAI, [Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots). Checked 10 November 2026. - Anthropic, [Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler). Checked 10 November 2026.