Let's get the uncomfortable part out of the way first: the evidence that llms.txt meaningfully impacts AI visibility is, at best, inconclusive.
I know that's not what you want to hear after reading ten breathless blog posts about "the future of AI SEO." But I'd rather give you the honest picture than hype you into wasting time on false expectations.
Here's what we actually know.
What the Studies Show
The OtterlyAI Experiment
One of the more rigorous studies came from OtterlyAI, who tracked AI crawler behavior across websites with llms.txt files over a 90-day period.
The headline result: only 0.1% of AI crawler requests touched /llms.txt. The file received far fewer visits than the average content page. That means AI crawlers were visiting these sites but largely ignoring the llms.txt file.
The SE Ranking Analysis
SE Ranking analyzed 300,000 domains and found something even more deflating: domains with llms.txt files were not more likely to be cited by AI models. In fact, in some tests, the model performed better without the file.
That's worth sitting with for a moment. Not "slightly better with it." Not "no difference." In some cases, worse.
The Search Engine Land Test
Search Engine Land implemented llms.txt on their own site starting in March 2025. Over months of monitoring, they found no correlation between having the file and improved performance in AI-generated results. From mid-August to late October 2025, the file received zero visits from GPTBot, PerplexityBot, ClaudeBot, or Google-Extended bot.
Zero.
What the AI Companies Say
This is where it gets interesting.
Google has been the most blunt. Gary Illyes compared llms.txt to the old keywords meta tag — a standard that webmasters adopted enthusiastically but that search engines stopped caring about years ago. John Mueller echoed the sentiment.
OpenAI, Anthropic, and Perplexity haven't officially commented on whether their systems read llms.txt during inference.
No major AI provider has publicly stated they use llms.txt files. Not one.
So Why Is Anyone Still Talking About It?
Good question. Here's my honest assessment of why llms.txt still has momentum despite the evidence:
1. The Cost Is Near Zero
Adding an llms.txt file takes five minutes. Even if there's only a 5% chance it eventually matters, the expected value calculation works out. You're not investing engineering months — you're writing a Markdown file.
2. Adoption Creates Pressure
Over 800,000 websites have published llms.txt files. Cloudflare, Stripe, Anthropic, Vercel, and Coinbase all have them. When that many sites adopt a standard, AI providers face increasing pressure to support it. Network effects are real, even for file specs.
3. The Spec Is Still Young
robots.txt was proposed in 1994 and took years to reach universal adoption. llms.txt has existed for roughly a year. Judging its long-term impact now would be like reviewing a restaurant during construction.
4. It Has Indirect Benefits
The process of creating an llms.txt forces you to think critically about your content hierarchy. Which pages are actually important? What's your site really about? That exercise has value regardless of whether any AI ever reads the file.
My Honest Recommendation
Create one, but don't build a strategy around it.
Here's what that looks like in practice:
- Do spend 10-15 minutes creating a well-structured llms.txt
- Do include your most important 20-30 pages with clear descriptions
- Do update it when you publish major new content
- Don't spend money on llms.txt optimization tools
- Don't measure your AI visibility based solely on having one
- Don't let anyone sell you "llms.txt SEO services"
The real work for AI visibility is in your actual content quality, your structured data, your brand authority, and how your site appears in training data. An llms.txt file is a nice-to-have, not a strategy.
What Actually Moves the Needle for AI Visibility
If you want AI models to understand and cite your content, focus on things that demonstrably work:
- Clean, crawlable content: AI systems index your actual pages. Make them good.
- Structured data: Schema markup helps any system (AI or otherwise) understand your content.
- Authority signals: Being cited by other authoritative sources matters more than any file.
- Clean formats: Serving content as structured Markdown (instead of heavy HTML) makes it easier for AI to process. Tools like link.sc exist specifically because this is a real problem — most websites aren't LLM-friendly by default, and converting HTML to clean Markdown is a critical step in any AI data pipeline.
The Bottom Line
llms.txt is a reasonable, low-cost bet on a possible future standard. The data doesn't support it as a game-changer today. But the web moves fast, especially where AI is involved, and being early to a standard that eventually wins is worth a lot more than the five minutes it costs.
Just don't confuse "I published a file" with "I have an AI strategy."
Building an AI application that needs reliable web data? link.sc delivers clean, structured content from any URL — no llms.txt required. Try it free.