As AI search tools have grown, so has a steady stream of advice about new files and formats that supposedly help your site get cited by them. The most common is llms.txt, a proposed file that gives large language models a clean, curated summary of a website. Some guides treat it as essential for AI visibility. Google says you don't need it. Meanwhile, Google's own Chrome team has started checking for it in Lighthouse. This guide explains what llms.txt actually is, what Google has said about it, where it's starting to show up, and whether it's worth adding to your site.

What Is llms.txt?

llms.txt is a proposed standard for a plain-text Markdown file placed at the root of a website, at yoursite.com/llms.txt. It was proposed in September 2024 by Jeremy Howard, co-founder of fast.ai. The idea is to give AI systems a concise, machine-friendly overview of a site so they don't have to parse complex HTML, navigation, scripts, and ads to find the useful content.

A typical llms.txt file contains:

  • An H1 heading with the site or project name
  • A short summary of what the site is, in a blockquote
  • Optional notes with context about the site
  • Sections of links to the most important pages, each with a short description

Here's a simplified example of what one might look like for a site like this one:

# TopWebPositions

> Free SEO and website audit tools, plus practical guides on technical SEO, content, link building, and local SEO.

## Tools
- [SEO Audit Tool](https://topwebpositions.com/tools/seo-audit/): 40+ point on-page and technical audit
- [Indexability Checker](https://topwebpositions.com/tools/indexability-checker/): checks noindex and X-Robots-Tag

## Guides
- [What Is Technical SEO?](https://topwebpositions.com/seo_insights/posts/what-is-technical-seo.html): plain-English overview

Some sites also publish a longer companion file, often named llms-full.txt, that contains the full text of key pages in Markdown.

llms.txt Is Not robots.txt

The names are similar, but the files do completely different jobs:

  • robots.txt tells crawlers which URLs they may and may not fetch. It's a long-established standard that major crawlers, including AI crawlers, check. The beginner's guide to robots.txt covers how it works.
  • llms.txt doesn't grant or deny access to anything. It's a voluntary guide to your content. An AI system that ignores it loses nothing, and it can't be used to block AI crawlers.

If your goal is to control whether AI companies can crawl your content, robots.txt is the tool. The upcoming guide on whether to block AI crawlers covers those decisions.

What Google Has Said About llms.txt

Google Search's position is clear. Its documentation on AI features such as AI Overviews and AI Mode says you don't need to create new machine-readable files, AI text files, or markup to appear in those features, and that there are no additional requirements or special optimizations needed. Google's Search team has also said publicly that it isn't using llms.txt and has no plans to support it, and has compared it to the long-ignored keywords meta tag.

That matters because Google's AI features draw on its regular search index. The same things that help your pages rank, such as crawlability, clear structure, and genuinely helpful content, are what help them get cited. The guide to ranking in Google AI Overviews covers those factors.

Where llms.txt Is Showing Up

The picture isn't entirely one-sided. In May 2026, Chrome's Lighthouse auditing tool added an llms.txt check to a new set of agentic browsing audits, which evaluate how well a site works for AI agents that browse on a user's behalf. The documentation describes llms.txt as an emerging convention. If your site has no llms.txt at all, the audit is marked not applicable, because the file is optional. It mainly flags server errors when the file is requested.

So the current situation is split. Google Search says the file isn't needed to appear in its AI features, while Chrome's developer tooling treats it as a reasonable convention for AI agents. Neither position says llms.txt affects rankings.

Beyond Google, there's little public evidence that the major AI search systems actually read llms.txt files when deciding what to cite. Some AI coding assistants and developer tools do use llms.txt files, particularly for software documentation, which is where the format has been adopted most widely.

Should Your Site Have One?

For most sites, llms.txt is a low-priority, low-cost addition. Here's a practical way to decide.

Reasons to add one

  • You publish technical documentation that developers use with AI coding tools. This is where llms.txt has the clearest real-world use.
  • It's cheap. A basic file takes under an hour to write and needs updating only when your main pages change.
  • It's a useful exercise. Writing a one-paragraph summary of your site and choosing your ten most important pages forces clarity about what your site is for.
  • Agent tooling is moving. If AI agents become a meaningful source of visits, having a clean summary ready costs little.

Reasons not to prioritize it

  • Google says it doesn't need it for AI Overviews or AI Mode.
  • There's no evidence it improves rankings or AI citations in major AI search products.
  • It can drift out of date. A stale file pointing at moved or deleted pages is worse than none.
  • It doesn't fix real problems. If AI systems aren't citing you, the cause is almost always content, authority, or crawlability, not a missing summary file.

A reasonable middle path: if you have a few spare hours after your technical foundations and content are solid, add a simple llms.txt and review it when you update your sitemap. Don't spend money on it or delay real work for it.

How to Create an llms.txt File

If you decide to add one, keep it simple:

  • Create a plain-text file named llms.txt in your site's root directory.
  • Start with # Your Site Name, followed by a one- or two-sentence summary in a blockquote line beginning with >.
  • Add a few sections with ## headings, such as Tools, Guides, or Products.
  • Under each, list your most important pages as Markdown links with a short description after a colon.
  • Link only to live, canonical, indexable pages, the same standard you'd use for your XML sitemap.
  • Confirm the file loads at /llms.txt and returns a 200 status. The Redirect & Header Checker shows the exact status code.

Make sure robots.txt doesn't block the file, and serve it as plain text.

What Actually Improves AI Visibility

If your real goal is to be cited by AI search tools, spend most of your effort where the evidence points:

  • Make sure AI crawlers can reach your content. Check your robots.txt rules for the crawlers you want to allow with the Robots.txt Tester.
  • Structure content for extraction. Clear headings, direct answers near the top of sections, and concise definitions are easier for any system to quote. The Heading Structure Checker shows your page outline at a glance.
  • Build topical depth and trust. AI systems tend to cite sources that cover a topic thoroughly and credibly. The guide to topic clusters and pillar pages covers how to build depth deliberately.
  • Earn visibility beyond Google. Some AI search products rely on other indexes. The guide to optimizing for ChatGPT search covers what matters there.

Finally, measure instead of guessing. Without data, it's impossible to know whether any change, llms.txt included, affected how often AI tools mention you. OmniSEO tracks your brand's visibility across ChatGPT, Perplexity, Google AI Overviews, and other AI platforms, so you can see whether you're being cited and compare yourself with competitors. The OmniSEO review covers what it does and who it's for. Disclosure: TopWebPositions may earn a commission if you sign up through this link, at no extra cost to you.

The Bottom Line

llms.txt is a reasonable idea with limited adoption. Google Search says you don't need it to appear in its AI features, there's no evidence it affects rankings or citations in major AI search tools, and it doesn't control crawler access the way robots.txt does. It's cheap to add and harmless if kept current, and Chrome's tooling now recognizes it. Treat it as an optional extra, not a strategy. The fundamentals of crawlability, structure, and genuinely useful content are still what get you found, by search engines and AI systems alike.