What is llms.txt?

By Nik Andreev, Founder & SEO specialist · Last reviewed · Editorial policy

llms.txt is a proposed Markdown file, usually served at /llms.txt, that gives AI models a summary of a site and a curated list of links to its most useful content. Jeremy Howard published the proposal in September 2024. It is not a standard and controls nothing: models can read it, ignore it or never request it.

Full guide llms.txt explained: what it is and how to create one How to write one, the v2 changes, and the evidence on whether AI crawlers actually request it. Read the guide →

At a glance

Proposed by
Jeremy Howard, at llmstxt.org (September 3, 2024)
Status
Community proposal, not an IETF or W3C standard
Location
/llms.txt at the root, or a subpath such as /docs/llms.txt
Format
Markdown. Only the H1 with the site or project name is required
Google Search
Ignores it. Google says it will neither harm nor help rankings
Controls access?
No. It grants or blocks nothing

How to spot it

  • In your logs: requests for /llms.txt (or a subpath copy). The user agent tells you which bot, if any, fetched it, and verification tells you whether that claim is true.
  • The format: an H1 with the site name, an optional blockquote summary, then H2 sections of Markdown links with short notes.
  • What it is not: an opt-in or opt-out. To control access, use robots.txt or a WAF.
# Example Co

> Example Co makes billing software for small teams.

## Docs

- [Quick start](https://example.com/docs/start): Set up in five minutes

Files bots read at your site root

robots.txt The standard (RFC 9309) for telling crawlers which paths they may fetch. AI crawler The bots that would read llms.txt, if they choose to.

Quick answers

Does llms.txt help with Google rankings?

No. Google's guide to generative AI features says the file will neither harm nor help visibility or rankings in Search, because Google Search ignores it.

Is llms.txt the same as robots.txt?

No. robots.txt asks crawlers to stay out of paths. llms.txt only summarizes content for models that choose to read it.

How do I know if AI bots read my llms.txt?

Check your server or CDN logs for requests to /llms.txt and look at the user agents making them.

Sources