What Is llms.txt, and Does It Actually Help AI Search?
llms.txt is a Markdown file at your site root that gives AI tools a curated map of your most important pages. Here's how it works, how it differs from robots.txt and sitemap.xml, and an honest verdict on whether it moves AI citations.
llms.txt is a plain-text Markdown file at your site's root (yoursite.com/llms.txt) that gives AI systems like ChatGPT, Perplexity, and Gemini a curated map of your most important pages. Think of it as a sitemap written for language models instead of crawlers. Evidence that it improves AI citations is still thin: no major AI provider has confirmed it parses the file, so treat it as a cheap addition, not a fix.
How llms.txt works
The format was proposed at llmstxt.org in 2024. The file is plain Markdown with a simple shape:
- An H1 with the site or product name. Required.
- A short blockquote summarizing what the site is.
- Optional paragraphs with anything a model should know up front.
- H2 sections of links, each a Markdown list of
[Page title](url): one-line description.
A minimal example:
# Northwind Coffee
> Northwind roasts single-origin coffee in Portland and ships across the US.
## Products
- [Subscriptions](https://northwind.example/subscribe): Weekly or monthly beans, cancel anytime.
- [Wholesale](https://northwind.example/wholesale): Pricing and terms for cafés.
## Guides
- [Brewing guide](https://northwind.example/brew): Ratios and grind sizes for pour-over, French press and espresso.
The idea is that an AI tool fetching your site gets a short, clean summary instead of wading through navigation, scripts and cookie banners to work out which pages matter.
llms.txt vs robots.txt vs sitemap.xml
| File | Audience | What it does |
|---|---|---|
| robots.txt | All crawlers | Allows or blocks crawling, per bot and per path |
| sitemap.xml | Search engines | Lists every URL you want indexed |
| llms.txt | AI tools and agents | Curated summary of your most important pages |
The important difference: robots.txt is enforced by well-behaved bots, and llms.txt is only a suggestion. If your robots.txt blocks GPTBot or PerplexityBot, no llms.txt file will get you cited. Check that first.
Does llms.txt improve ChatGPT or Perplexity visibility?
Not proven yet. A late-2025 SERanking study of roughly 300,000 domains found no measurable citation lift from publishing one, and Google's AI search guidance doesn't mention it. OpenAI, Anthropic, Perplexity and Google have not said their answer engines use it as a citation signal.
Where it can help today is with AI coding tools and agents that fetch documentation on demand. Several developer-docs platforms publish llms.txt for exactly that reason.
So the honest verdict: publish one, because it takes minutes and costs nothing, then spend your real effort on the things with evidence behind them. In order:
- Let the AI search bots in. Allow OAI-SearchBot, ChatGPT-User, PerplexityBot and Google's crawlers in robots.txt, and make sure your CDN isn't challenging them.
- Write answer-first pages. Put a direct, two or three sentence answer at the top of each page that targets a question. Engines lift those.
- Add structured data. Organization, Article and FAQPage JSON-LD help engines trust who you are and what the page answers.
- Get mentioned elsewhere. Third-party reviews, comparisons and forum threads are what AI engines cite most for "best X" questions.
We cover each of these in 8 fixes that move brand visibility in AI search.
How to add llms.txt to your site
- WordPress: Yoast and Rank Math can both generate one for free. The free AuditAE WordPress plugin generates a live llms.txt that updates as you publish, alongside robots.txt checks and AI crawler logs.
- Next.js, Astro and other code sites: add a static file to
public/llms.txt, or a route that builds it from your content so it never goes stale. - Webflow, Wix, Shopify, Squarespace: support varies by platform and plan. We compared them in the best website builders for AI search.
Keep it short. List the pages that answer the questions buyers ask, not every URL on the site. That's what sitemap.xml is for.
See one in practice
AuditAE publishes its own at auditae.app/llms.txt: product pages, pricing, per-engine guides and the blog, each with a one-line description. Its AI agent can generate and maintain the same file for your site as part of its citation-gap fixes.
Not sure whether llms.txt is even the problem? The free AI search audit scores any site 0–100 on robots.txt bot rules, llms.txt, schema, server-rendered text and freshness, and tells you which fix to do first.
FAQ
llms.txt is a Markdown file published at a site's root that gives AI systems like ChatGPT, Perplexity, and Gemini a curated map of a site's most important pages, similar to a sitemap but written for LLMs instead of crawlers.
Not proven yet. No major AI provider has confirmed they parse llms.txt as a ranking or citation signal, and a late-2025 SERanking study of roughly 300,000 domains found no measurable citation lift. It's a low-cost addition worth doing, but crawlability and clear, answer-first content still matter more for AI visibility.
No. robots.txt tells crawlers which URLs they may fetch. llms.txt doesn't allow or block anything; it only points AI tools at the pages you consider most important. If robots.txt blocks an AI bot, llms.txt can't undo that.
Yoast and Rank Math can both generate one for free, and the free AuditAE plugin generates a live llms.txt at your site root that updates as you publish. You can also upload a hand-written llms.txt file to your site root with your host's file manager.
Aaron is the founder of AuditAE. He has run AI-visibility audits for SEO agencies and in-house brand teams, and writes about how generative answer engines are reshaping the practice of search marketing.
Related reading
- 12 min readDo llms.txt Files Help AI Search? What They Do, What They Don't, and How to Use OneAn llms.txt file can help AI agents understand your site faster, but it is not a proven AI search ranking factor. Here is what it does, what it does not do, and how to use one.
- 10 min readHow to Improve Brand Visibility in AI Search Engines: 8 Fixes You Can Ship This MonthAI engines already have an opinion about your brand. Here is how to check what it is, and the eight fixes, in order, that change it — with the WordPress path for each one.
Run the citation report on your own brand.
See which prompts cite you on ChatGPT, Perplexity, Gemini, and Google AI Overviews — the first one’s free, no card.