Guide 02 · Evidence review
Does llms.txt actually work?
On the evidence available today, no. Ahrefs checked 137,000 sites that had published an llms.txt file and found that 97% of those files had never been requested by any crawler at all. Not read and ignored — never fetched.
What is llms.txt supposed to do?
It is a proposed convention: a plain-text file at the root of your domain, written for language models rather than for browsers, listing your important pages in a form a model can read without wading through navigation and markup. The analogy is robots.txt, and the name is deliberately borrowed.
The idea is reasonable. A clean, curated map of a site is genuinely easier to parse than a rendered page. Nothing about the proposal is silly, and that is exactly why it spread so quickly through GEO advice.
Do the engines read it?
No major engine has committed to reading it, and the server logs say they do not. That is the whole finding, and it is a stronger form of evidence than most GEO claims have behind them.
The distinction that matters is between “fetched and disregarded” and “never fetched”. If the files were being requested and not acting on, you could argue the weighting was low and might change. A request log with nothing in it says the convention has not been adopted at all. Adoption is a decision each engine makes and announces; none has.
robots.txt works because crawler operators agreed to request it. Publishing a file is one half of a convention. The other half is somebody asking for it, and that half is what the 97% figure measures.
Why do so many tools still score you on it?
Because it is trivially easy to check, and a checkable thing makes a tidy line in a report. A scanner can confirm whether a file exists at a fixed path in one request. Confirming whether a retrieval crawler can actually reach and parse your content takes real work.
Free AI-visibility checkers now list llms.txt among their scored visibility signals — one of the more careful ones names its three signals as robots.txt, llms.txt and a live crawler probe. Two of those three measure whether an engine can reach you. The third measures whether you have published a file that, on current evidence, nothing requests.
This is worth knowing if a report has told you your AI visibility is fine, or told you it is broken. Check which of the things it scored are things an engine actually asks your server for.
How long does llms.txt take to work?
There is no waiting period to be patient through, because there is no observed mechanism to wait on. If a file has not been requested, elapsed time does not change its effect. The question assumes a delay; the data suggests an absence.
You can settle this for your own domain in about a minute rather than taking my word for it. Search your access log for requests to the file:
grep llms.txt /path/to/access.logOn shared hosting the raw log is usually downloadable from cPanel under Raw Access. If the search comes back empty, no crawler has asked for the file since that log began, and nothing that reads it has seen your site. If it comes back with hits, I would like to see them — that would be a genuinely new data point and I will publish it with credit.
Should you publish one anyway?
You can. It costs ten minutes and it does no harm. If the convention is adopted later you are already covered, and ten minutes is a cheap option on that.
What I object to is the file being presented as a lever when it is a lottery ticket, and — more sharply — being placed at the top of a to-do list ahead of the things that decide whether you are retrievable at all. The order is the problem, not the file.
What should you do instead?
Make correct and reachable the things a retrieval crawler demonstrably does request: your TLS certificate, your robots.txt, your XML sitemap, and the HTML itself.
The free retrievability check tests exactly those, from a wordpress.org slug — and reports llms.txt at a weight of zero, for the reason above.
| File or property | Requested by retrieval crawlers? | Effect if broken |
|---|---|---|
| TLS certificate | Every request | Total |
| The page HTML itself | Every request | Total |
| robots.txt | Yes, by convention | Total |
| XML sitemap | Yes | Partial |
| llms.txt | 97% never fetched | None observed |
The first three are binary. Get one of them wrong and no amount of content work registers, because nothing arrives to read it. This site was invisible to every retrieval crawler for 152 days on an expired certificate, while its content was fine the whole time — which is a more useful cautionary tale than any llms.txt argument.
What would change my mind?
This rests on one large observational study and on the absence of any engine commitment. Both could change without notice and without a changelog.
Two things would move this page. An engine publicly committing to read the file, which would be an announcement rather than an inference. Or access logs showing a named retrieval crawler requesting llms.txt at a meaningful rate — one site’s hits could be a scanner, but a pattern across sites would not be.
The Ahrefs sample is large but it is one study, and it measures a moment. A convention with 97% non-adoption today is not permanently dead; it is unadopted. I would rather state that precisely than round it up into a verdict it has not earned.
Until one of those happens, the honest position is the one at the top: no observed effect, ten minutes to hedge, and nowhere near the front of the queue.
Want to know what is actually reaching your site?
Send me your wordpress.org slug. I will check what the retrieval crawlers get when they ask for your pages — certificate, robots rules, live response, and whether your content is in the HTML they receive. Free, no call, usually within a day.