I talk to a lot of developers and site owners about llms.txt, and the same questions keep coming up. Rather than make you dig through ten different articles, here are the real answers to the things people actually want to know.
1. Do I Really Need an llms.txt File?
Short answer: You don't need one. Your site will work fine without it.
Longer answer: It depends on what "need" means to you. If you have documentation, a blog, or product pages that you want AI systems to represent accurately, an llms.txt gives you a low-effort way to signal what's important. The cost is five to fifteen minutes of your time. The potential upside is better AI representation of your brand as these systems evolve.
If your site is a personal blog with ten posts, it's probably not worth the bother. If you run an API product and developers are asking AI assistants about your endpoints, it's worth the effort.
2. Does Google Use llms.txt?
No, not currently. Google's Gary Illyes stated publicly in July 2025 that Google doesn't support llms.txt and isn't planning to. John Mueller compared it to the keywords meta tag.
That said, Google did include llms.txt in their Agent-to-Agent (A2A) protocol, which is an interesting signal. And Google's stance on standards has changed before. But right now, publishing an llms.txt won't do anything for your Google rankings or AI Overviews.
3. Does Any AI Provider Actually Read These Files?
Not officially. As of early 2026, no major AI provider — OpenAI, Anthropic, Google, Perplexity — has publicly confirmed they read llms.txt files during inference or training.
Some independent analyses have found very low llms.txt access rates from AI crawlers. The OtterlyAI study found only 0.1% of AI crawler requests touched the file over 90 days.
This could change. The standard is still young. But anyone claiming AI providers actively use llms.txt today is getting ahead of the evidence.
4. Is llms.txt the Same as robots.txt?
No. They solve completely different problems.
- robots.txt controls access — which bots can visit which pages
- llms.txt guides understanding — which pages are most important and what they contain
robots.txt is a bouncer. llms.txt is a tour guide. You probably want both, and they should be consistent with each other (don't block pages in robots.txt that you link to in llms.txt).
5. What Format Should llms.txt Be In?
Markdown. Specifically, it should follow this structure:
- An H1 heading with your site/project name (required)
- A blockquote with a summary (recommended)
- H2 sections with organized link lists (recommended)
- Each link optionally followed by a colon and description
No HTML. No JSON. No YAML. Just plain Markdown. It should be served as a raw file at yourdomain.com/llms.txt.
6. How Many Links Should I Include?
Between 20 and 50 for most sites. This isn't a hard rule, but it comes from a practical consideration: the file should be short enough for an AI to process quickly, and curated enough that every link earns its place.
If you have trouble cutting below 50, ask yourself: "Does this page help an AI understand what my business does and offers?" If the answer is "not really," cut it.
Don't list every blog post. Don't list support articles. Don't list legal pages (unless you're a legal company). Keep it tight.
7. What About llms-full.txt?
Some sites publish an llms-full.txt that includes the actual content of key pages, not just links. This is useful for API documentation or critical reference content because it gives AI everything in one file.
It's optional and not part of the original spec. If your key content totals under 50,000 tokens, it might be worth creating. If your docs are larger, stick with the standard llms.txt with links.
8. Can llms.txt Hurt My Site?
Extremely unlikely. The worst case scenario is that the file sits there doing nothing. It can't negatively impact your SEO because search engines don't use it. It can't break your site because it's just a static text file.
The only way it could theoretically hurt is if you include misleading descriptions that cause AI to misrepresent your products. But that's a problem with your content, not with the file format.
9. How Often Should I Update It?
Quarterly, or whenever you make major content changes. You don't need to update it every time you publish a blog post. But you should review it when:
- You launch a new product or major feature
- You restructure your documentation
- You publish a significant piece of content (like a definitive guide)
- You remove or redirect pages that are currently listed
- Your business focus or positioning changes
Set a calendar reminder. It takes five minutes per review.
10. What Should I Focus on Instead?
If you've got your llms.txt published and you're wondering what else to do for AI visibility, here's where to spend your time:
Content quality comes first. AI models cite good content. Write definitive, well-structured articles that become the authoritative source on topics in your space.
Structured data helps. Schema markup (JSON-LD) gives both search engines and AI models structured facts about your content. FAQ schema, How-To schema, and Product schema are particularly useful.
Clean content delivery matters. If your pages are JavaScript-heavy or cluttered with ads, AI systems struggle to extract the content. Making your content technically accessible is at least as important as any metadata file.
For developers building AI apps, the consumption side of this equation is equally important. When your application needs to fetch and understand web content, the same challenges apply in reverse — you need clean, structured data from messy web pages. That's the problem link.sc was built to solve, converting any URL to clean Markdown that LLMs can actually work with.
The Bottom Line
llms.txt is a small, sensible bet. Create one, keep it updated, and focus your real energy on making great content that's technically accessible. The AI discoverability landscape is going to keep evolving, and being early to the basics — a curated content map, clean page structure, structured data — positions you well no matter which direction things go.
Need clean web data for your AI applications? link.sc handles the messy part — fetching, cleaning, and formatting web content for LLMs. Get started free with 500 requests/month.