# llms.txt and Markdown for agents, in plain language

**llms.txt is a plain-text index at the root of a website that tells an AI assistant what the site is and where its important pages are; Markdown for agents is serving each page as clean Markdown, either when a tool asks for it or at a .md address, so an assistant reads the content in a tenth of the bytes.** Both make a site cheaper and more accurate for a machine to read once it has found you. Neither makes it rank: Google says in writing that no AI text file or markup is needed to appear in its AI features, and nobody who sells these as a ranking trick is telling the truth.

## What is llms.txt?

A proposal, not a standard, published at [llmstxt.org](https://llmstxt.org/): a Markdown file at
/llms.txt that opens with the site's name, a one-paragraph summary, and a list of
links to the pages an assistant would need, each with a line saying what it is. The idea is that a web page
is a poor thing to hand a language model, full of navigation, scripts and repetition, and a short curated
index is a better starting point. Blue Crane's own is at
[bluecraneworks.ca/llms.txt](https://bluecraneworks.ca/llms.txt) and says, in a few hundred words, what the company does,
what it charges and where each product lives.

Its job is accuracy. When an assistant is asked about a business and reads that file, it gets the facts
the business chose, in the words the business chose, instead of whatever it could scrape from a footer. A
stale llms.txt is therefore worse than none: it looks authoritative and is wrong. Ours is regenerated with
the site so it cannot drift.

## What is Markdown for agents?

Two separate mechanisms that people talk about as one. The first is ordinary web plumbing: a browser
asking for a page sends a header saying what formats it accepts, and an AI tool can send
Accept: text/markdown. A server that notices can return the same page as Markdown
instead of HTML. That is content negotiation as defined in [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html), the HTTP
standard, and the Markdown media type is registered in [RFC 7763](https://www.rfc-editor.org/rfc/rfc7763.html). The second
mechanism needs no header at all: the same page is also reachable at a .md
address, so /pricing has a twin at /pricing.md, which
is the convention the llms.txt proposal suggests.

Why both? Because the tools split. The coding assistants, Claude Code, Copilot, Cursor, send the header;
the consumer assistants people actually ask about local businesses, ChatGPT and Perplexity among them,
do not, according to the [field tracking at acceptmarkdown.com](https://acceptmarkdown.com/status). For that larger
group the only clean thing you can give them is a .md address that a person, a sitemap or an llms.txt
hands over. Cloudflare sells the first half as a paid feature; its
[documentation](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) describes exactly that, negotiation on the Accept header,
and does not create the .md addresses. Blue Crane builds both halves on every managed site, for free, and
measured the result on this domain: the homepage is 213 kilobytes as HTML and about 11 as Markdown, 95
percent smaller for a tool that asks.

|   | Accept: text/markdown | A .md address | llms.txt |
| --- | --- | --- | --- |
| What it is | The client asks for Markdown in the request | The same page at /page.md | One file listing the site's key pages |
| Defined by | [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html), [RFC 7763](https://www.rfc-editor.org/rfc/rfc7763.html) | The [llms.txt proposal](https://llmstxt.org/) | The [llms.txt proposal](https://llmstxt.org/) |
| Who uses it | Coding assistants: Claude Code, Copilot, Cursor | Anything, including ChatGPT and Perplexity, once handed the link | An assistant reading the site's index |
| Needs the client to opt in | Yes, it must send the header | No | No |
| Helps ranking | No | No | No. [Google says so in writing](https://developers.google.com/search/docs/appearance/ai-features) |

## Do these files help a site rank or get cited?

No, and it is important to say so plainly because they are sold as if they did. Google's guide to AI
features says that to appear in AI Overviews and AI Mode "you don't need to create new machine readable
files, AI text files, or markup", and that there are [no additional requirements](https://developers.google.com/search/docs/appearance/ai-features)
beyond being indexed and snippet-eligible. Bing's guidelines carry the general limit: GEO "does not guarantee grounding or citations." What the files change is what happens after an assistant has already
decided to read you: fewer tokens, less noise, and your facts in your words. That is worth having. It is
not a ranking.

## What can go wrong?

Three things, each of which we have seen live. A Markdown copy that says something different from the
page, an old price or a dropped paragraph, is the version the assistant will quote back to a customer, and
serving different content to different readers is also the definition of cloaking, which Google penalises.
A server that negotiates on the header without telling caches it did so can hand one visitor's Markdown to
the next visitor's browser; the fix is one response header and it is not optional. And an llms.txt written
once and never updated turns into a confident lie about the business. The rule for all three is the same:
generate the files from the same source as the pages, on every deploy, so they cannot disagree.

## Where do these sit in the order of work?

Last. First the crawlers have to be allowed in and actually get in, the words have to be in the HTML, and
nothing on the page may forbid citation; [the five-minute check](https://bluecraneworks.ca/articles/can-chatgpt-see-my-website)
covers that floor and it decides everything. Then the site has to be worth quoting and has to agree with
the [business listing](https://bluecraneworks.ca/articles/does-my-google-business-profile-matter-for-ai-answers). Then, and
only then, llms.txt and Markdown make a readable site cheap to read. On a Blue Crane managed site all of it
is built in from day one; on a site you already have, the one-time [AI Visibility Pass](https://bluecraneworks.ca/solutions/ai-visibility.html)
adds the same files and proves them.

## Can I use this?

Yes. Everything on this page is free to use: quote it, print it, hand it to your own web
person, or read it out to a client. Attribution is welcome and not required, and every claim
is linked to the vendor's own document so you can check it rather than take our word for it.
If a rule below changes at the source, the source wins, and we would rather hear about it than
keep a stale page up.

## Want this checked and fixed on your site?

The AI Visibility Pass runs every check in this article against your live site, fixes what it
finds, and hands you the proof. Fixed quote, written guarantee. On a managed website it is included
from launch.

[See the AI Visibility Pass](https://bluecraneworks.ca/solutions/ai-visibility.html)
[Managed websites, $100 a month](https://bluecraneworks.ca/solutions/websites.html)

## Next, from Blue Crane

- [**Can ChatGPT see my website?**The five-minute check anyone can run](https://bluecraneworks.ca/articles/can-chatgpt-see-my-website)

- [**The two settings that hide a site from AI answers**The directives that forbid citation](https://bluecraneworks.ca/articles/noarchive-nosnippet-remove-your-site-from-ai-answers)

- [**What AI visibility is, and why leads arrive this way now**The shift from ten links to one answer, in the vendors' own words](https://bluecraneworks.ca/articles/what-is-ai-visibility)

- [**The AI Visibility Pass**Every check in the article, run and fixed for you, under a written guarantee](https://bluecraneworks.ca/solutions/ai-visibility.html)

---

Source: https://bluecraneworks.ca/articles/llms-txt-and-markdown-for-agents-explained