This article is based on empirical findings published by Search Engine Land, which tracked real websites before and after implementing llms.txt.
Key Takeaways
- llms.txt does not drive rankings or visibility. It does not make LLMs surface your content more often.
- Most websites see no impact. Adding llms.txt rarely changes traffic, crawl behavior, or AI referrals.
- Content quality and structure still decide everything. LLMs surface content that is useful, clear, and trusted.
What is LLMs.txt?
llms.txt is a proposed text-based file designed to help AI systems identify and consume important content on a website more efficiently. Instead of parsing full HTML pages with navigation, scripts, and design elements, an AI agent could use llms.txt as a curated entry point.
Conceptually, it resembles a sitemap, but with added human-readable context. The file typically lists priority URLs alongside short descriptions explaining what each page contains. What it does not do is signal quality, relevance, or authority.
Crucially, no major LLM provider has officially stated that llms.txt is used as a discovery, ranking, or prioritization signal. In many cases, server logs show that AI crawlers do not request the file at all.
Does LLMs.txt good for SEO & AEO?
For SEO, llms.txt has no measurable impact on rankings. It does not influence indexing, relevance scoring, backlinks, or internal link equity. Those factors remain unchanged regardless of whether the file exists.
For AEO (Answer Engine Optimization), llms.txt can only help after an AI system has already decided to consume your content. It does not influence whether your content is selected as a source in the first place. That decision depends on how well your content answers real questions and how easy it is to extract and trust.
In the Search Engine Land study, only two out of ten sites saw growth after adding llms.txt, and in both cases the growth correlated with other changes happening at the same time: new functional content, improved page structure, technical fixes, and external exposure. llms.txt was not the causal factor.
Should you use LLMs.txt for your website?
llms.txt can make sense for sites where AI agents are already heavily involved in how users interact with the product, such as developer tools, APIs, and technical documentation. In those cases, cleaner and more predictable content consumption can improve usability.
For most other websites, including publishers, ecommerce platforms, and service providers, llms.txt rarely changes outcomes. The limiting factor is not whether AI systems can find the content, but whether the content is actually worth surfacing.
If time and resources are limited, llms.txt should be treated as low priority compared to improving content usefulness, structure, technical accessibility, and credibility.
How to create an LLMs.txt file?
Creating llms.txt is straightforward from a technical perspective. The strategic challenge is deciding what deserves to be included.
- Define the purpose. Decide whether the file is meant to guide AI agents
through documentation, product information, or evergreen educational content. - Select priority pages. Include only pages that clearly represent your
most important concepts or functionality. - Add concise descriptions. Each URL should have a short explanation
that helps an AI system understand why the page exists. - Keep it curated. Overloading the file removes its value as a prioritization layer.
- Monitor usage. Check server logs to see whether relevant AI crawlers
actually request the file.
Summary
llms.txt is not a ranking factor, not a visibility lever, and not a shortcut to appearing in LLM answers. At best, it is optional infrastructure that may help AI systems consume content more efficiently if they already want it.
Websites that appear consistently in AI-generated answers do so because they publish clear, structured, and genuinely useful content, supported by credibility and accessibility. llms.txt can document that work, but it cannot replace it.
