What is llms.txt — And Does It Actually Work in 2026?

llms.txt is one of the most talked-about ideas in AI SEO — and one of the most oversold. The pitch is tidy: add a simple text file and help AI models find your best content. The evidence in 2026 is far less flattering. This guide explains what llms.txt is, how it differs from robots.txt, what the data actually shows about whether it works, and the one place it genuinely earns its keep. For the tactics that do move AI citations, see what AEO is.

What is llms.txt?

llms.txt is a proposed standard — introduced in September 2024 by Jeremy Howard of Answer.AI — for a Markdown file at yourdomain.com/llms.txt that lists your most important pages, each as a link with a short description, so a large language model can find and understand your key content quickly. A companion idea, llms-full.txt, goes further and concatenates the full text of those pages into a single file. Think of it as a curated, machine-friendly table of contents you publish for AI, written in plain Markdown rather than the XML of a sitemap.

Is llms.txt the same as robots.txt?

No — they are near-opposites in purpose, and only one is actually honoured at scale. robots.txt tells crawlers what not to access; llms.txt tries to suggest what an LLM should read.

robots.txt is a restrictive, respected standard; llms.txt is an aspirational, curational one. Publishing llms.txt does not restrict anything and, on current evidence, rarely causes anything to happen.

The asymmetry shows up in our own audit data: of the 909 websites SeoVision's free audit has analyzed as of August 17, 2026, 151 block at least one major AI crawler in robots.txt — and 128 of those block four or more. The respected file is actively used to shut AI out, while the aspirational one waits for a reader. If AI visibility is the goal, checking what your robots.txt already blocks is worth more than writing an llms.txt.

Does llms.txt work?

For AI search and citations, there is no evidence that it does. In May 2026 Ahrefs analysed roughly 137,000 domains and found that 97% of llms.txt files received zero bot requests — and of the requests that did arrive, AI answer and retrieval bots made up only about 1%; most came from SEO audit tools and generic crawlers. Adoption of the file has climbed sharply, but the engines it is meant for are, by and large, not reading it. No major provider — OpenAI, Anthropic, Google or Perplexity — has confirmed using llms.txt to build answers or choose citations. So the file can exist, be perfectly formatted, and still do nothing for your visibility in ChatGPT or Perplexity.

Do AI crawlers use llms.txt?

Mostly not — at least not the ones that generate answers. The traffic that hits llms.txt files today comes overwhelmingly from SEO tools, generic web crawlers and profiling bots, not the answer engines. The one audience that plausibly does use it is AI coding assistants and agentic developer tools: when a coding agent or a documentation-aware assistant needs to orient itself in a codebase or a product's docs, a clean llms.txt can genuinely help it find the right pages. That is a real but narrow use — developer docs, API references, coding agents — not a lever for marketing citations.

In the Ahrefs data, the requests that did land on llms.txt files came mostly from SEO audit tools — the single biggest category — followed by unidentified bots, general web crawlers and tech-profiling services. The answer engines that would actually turn a read into a citation were a rounding error. That distribution is the tell: the file is being fetched by tools that scan everything, not by the assistants it was written for.

Does Google support llms.txt?

No, and Google has said so plainly. Google representatives have confirmed that Search does not use llms.txt and there are no plans to; John Mueller likened the idea to the old meta keywords tag — a signal that sounds helpful but that search does not consume. Google's own AI-features guidance, updated in mid-2026, states directly that you do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI features, because Search itself does not use them. Mueller has also noted that llms.txt turned up on a few Google properties only because an internal content system added it and some teams had not removed it — not because Search reads it.

Why does the idea sound right but fall flat?

The intuition behind llms.txt is reasonable: sitemaps help search engines, so a Markdown map should help language models. The flaw is that a standard only works when the consumers commit to reading it. Sitemaps earned their status because Google, Bing and others publicly support and act on them. llms.txt has no such commitment — the major answer engines have not agreed to read it, so publishing one is like leaving instructions for a reader who never opens the envelope. Google has even pointed to a different, Google-backed direction for agent interaction (WebMCP) rather than llms.txt, which tells you where the platform's attention actually is. Until an engine you care about announces support and demonstrates it in your server logs, llms.txt remains a proposal in search of an audience.

How do I create an llms.txt file?

If you want the cheap optionality anyway, it takes minutes. Create a plain-text file named llms.txt at your domain root. Start with an H1 that names your site, add a short blockquote summary, then use H2 sections (for example: Docs, Products, Guides) with a Markdown link and a one-line description for each key page. Keep it short and current, list only genuinely important pages, and — if you serve developers — consider an llms-full.txt with the full docs text. That is the whole job; there is no verification step in a console, because no major engine officially consumes it. Do keep the maintenance cost in mind, though: a hand-curated file drifts out of date the moment you restructure your site, and a stale llms.txt that points at moved or deleted pages is worse than none — so only take it on if you will actually keep it current.

So should you bother?

Treat llms.txt as cheap optionality, not a citation strategy. If you publish documentation, an API, or developer content, adding it is a low-effort bet that may help coding agents and agentic tools navigate your site — worth doing. For everyone hoping it will get them cited in AI answers, it will not, and time spent perfecting it is time taken from the things that actually move citations: strong, rankable content, credible third-party coverage, and answer-ready structure. Do the real work first — see how to do AEO and the best GEO tools — and add llms.txt only as a five-minute afterthought if your content genuinely suits it.

Concretely, the levers that do move AI citations are the unglamorous ones: rank for the questions your buyers ask, because answer engines overwhelmingly retrieve and cite pages that already perform in classic search; earn credible third-party coverage, because reviews and round-ups shape what models believe about you far more than any file on your own domain; and structure your pages so a model can lift a clean, attributable line. A text file the engines ignore is not on that list, and no amount of formatting will move it there.

A worked example

A SaaS company with heavy product documentation publishes llms.txt listing its setup guides and API reference. Weeks later, its server logs show the file fetched occasionally by coding-assistant and crawler user-agents — and its citations in ChatGPT and Gemini are unchanged, because those came from its ranking pages and its reviews, not the file. The lesson in miniature: llms.txt helped a developer tool orient itself and did nothing for marketing visibility. That is exactly the scope to expect — useful in its lane, irrelevant outside it.

How we measured

The robots.txt figures are first-party: 909 real websites analyzed by SeoVision's free audit as of August 17, 2026, whose robots.txt files we parse against the major AI crawler user-agents. The llms.txt adoption and bot-traffic evidence discussed above is Ahrefs' published study of roughly 137,000 domains (May 2026 traffic), and the Google position comes from Google's public statements and documentation — both linked below, so you can check where our data ends and third-party data begins.