What an llms.txt is and why it matters for AI visibility
The llms.txt is a simple Markdown text file in your website's root directory, reachable at /llms.txt. It gives large language models a curated overview of your most important content: a title, a short summary and a sorted list of relevant pages, each with a description. Where a normal website is built for humans, the llms.txt provides a machine-readable map for AI.
The background is a shift in search behaviour. Gartner predicts a drop of around 25 percent in classic search volume by the end of 2026. In Germany, AI overviews already appear in an estimated 15 to 25 percent of searches depending on the topic. 60 to 65 percent of those searches end without a click to a website (zero-click), and organic clicks drop by up to 38 percent where AI overviews appear (industry analyses, 2026). Around 30 percent of B2B research now runs through assistants like ChatGPT and Perplexity.
This shifts the contest from the link list to the AI answer. To appear in that answer, you have to provide your content in a way that machines can reliably find, understand and prioritise. The llms.txt is a young, proposed standard for exactly this purpose (documented at llmstxt.org).
llms.txt appears in current GEO and AI-SEO checklists but is not yet an established mainstream standard. The file is low-effort and forward-looking: it does no harm, organises your content and is future-proof as more AI systems evaluate the standard. For GEO in the DACH region there is currently a first-mover window of roughly 12 to 18 months.
robots.txt versus llms.txt in the AI era
Both files live in the root directory and address machines, but they solve different tasks. The robots.txt governs access: it tells crawlers what they may fetch and what not. The llms.txt governs orientation: it tells AI systems which content is important and how it connects.
| Aspect | robots.txt | llms.txt |
|---|---|---|
| Purpose | Access control (allow / disallow) | Content curation for AI |
| Format | Directives (User-agent, Allow, Disallow) | Markdown (title, summary, links) |
| Audience | All crawlers and bots | Large language models / AI crawlers |
| Maturity | Established standard | Young, proposed standard |
| Role | Opens the door | Hands over the map |
The two files do not replace each other, they complement each other. First the robots.txt opens the door for AI crawlers by allowing GPTBot, PerplexityBot, ClaudeBot and Google-Extended. Then the llms.txt helps to find the right rooms. A perfectly structured llms.txt is of little use if the robots.txt locks the AI crawlers out.
Best practices: content and structure of an llms.txt file
The proposed structure is deliberately plain: an H1 with the name, a short summary block and thematic sections with links and descriptions. Markdown keeps the file readable for humans and machines alike.
Reduce to the essentials
Only link canonical core pages, not every subpage. The llms.txt is a curation, not a full sitemap. The clearer the selection, the better the orientation for the AI.
Give every line context
Each link needs a short, factual description. That way the AI knows what a page is about without having to evaluate it in full.
Put a meaningful summary first
The quote block right under the heading sums up your offering in one or two sentences. These lines are often the first thing a model picks up.
Keep it current
Only point to pages that exist and are reachable. An llms.txt that points to deleted content sends the wrong signal. Plan a regular review.
Concrete llms.txt examples for shops, local services and SaaS
The structure stays the same, the focus shifts with the business model.
Online shop
Local service provider
SaaS provider
Combining llms.txt with Schema.org and sitemaps
The llms.txt unfolds its effect in combination. It provides the map, Schema.org provides the machine-readable facts per page, and the sitemap provides the complete inventory of all URLs. Only together do they form a consistent picture for AI systems.
| File | Answers the question | Level of detail |
|---|---|---|
| llms.txt | What matters here? | Curated, compact |
| Schema.org (JSON-LD) | What exactly does this page mean? | Structured facts per page |
| Sitemap (XML) | Which URLs exist? | Complete, without ranking |
In practice that means: the core pages linked in the llms.txt should be exactly the pages that also carry the strongest Schema.org markup. If you link your product pages in the llms.txt, maintain Product or Offer markup there; local service providers use LocalBusiness accordingly. That way the map points to pages the AI can also understand in detail. How to keep this markup correct over time is covered in the guide Keeping Schema.org current.
Avoiding mistakes: what you should not hand AI crawlers
An llms.txt can also do harm when poorly maintained. Avoid these patterns.
Dead links to deleted pages, references to thin or duplicate content, keyword stuffing instead of clear descriptions, sensitive or internal pages in the public file, and confusing it with robots.txt. The llms.txt blocks nothing, it only recommends. Access protection still belongs in robots.txt and in the server configuration.
- Do not link internal, protected or legally sensitive pages, the file is publicly accessible.
- No keyword lists, but factual descriptions that represent the content correctly.
- No references to non-canonical, thin or duplicate pages, that dilutes the signal.
- Do not leave outdated links in place, every removed page belongs out of the llms.txt too.
- Do not mistake the llms.txt for access protection, that is what robots.txt is for.
How to measure the effects of your llms.txt with Feed-AI
An llms.txt is a measure, not an end in itself. What counts is whether your AI visibility actually improves. That is exactly what Feed-AI makes measurable, by regularly checking how ChatGPT, Perplexity and Gemini answer realistic search questions.
Capture the baseline
Before implementation a baseline is measured: whether you are discovered in brand-free searches (discovery), whether the AI knows you by name (awareness) and what share of voice you hold against competitors.
Implement the measures
Create the llms.txt, allow AI crawlers in the robots.txt, maintain Schema.org on the core pages. These steps work together.
Track the effect over time
The before-and-after history shows whether the three metrics move. That turns an assumption into provable progress instead of gut feeling.
To be clear and fair: a single file does not change an AI answer overnight. Visibility comes from the interplay of allowed access, clear structure, solid content and time. Measurement ensures you can see the progress instead of optimising in the dark.
Frequently asked questions
What is an llms.txt file? +
The llms.txt is a simple Markdown text file in a website's root directory (at /llms.txt). It gives AI systems like ChatGPT, Perplexity and Gemini a curated, machine-readable overview of the most important content: title, short summary and a sorted list of relevant links with descriptions. It is a young, proposed standard and complements robots.txt and the sitemap.
Do ChatGPT, Perplexity and Gemini really read the llms.txt? +
Support is still emerging. There is no guarantee that every AI system already evaluates the file today. llms.txt is a forward-looking, low-effort signal: it does no harm, keeps the most important content ready for crawlers and is future-proof as more systems adopt the standard. Actual discoverability still comes from crawlable content and structured data.
Where does the llms.txt have to be located? +
In the root directory of the domain, at https://your-domain.com/llms.txt. Optionally, a more detailed llms-full.txt with full content can be added. The file should be served as text/plain or text/markdown.
Does llms.txt replace robots.txt or the sitemap? +
No. robots.txt controls access, the sitemap lists all URLs in full, and llms.txt curates the most important content with context for AI systems. The three files complement each other and do not replace one another.
How often should I update the llms.txt? +
Whenever the most important content changes: new core pages, changed products, new services or restructured areas. An outdated llms.txt that points to removed pages does more harm than good. A quarterly review is a good rule of thumb.
Does llms.txt instantly improve AI visibility? +
Not automatically and not overnight. llms.txt is a building block, not a switch. What matters is whether the AI finds and recommends your content at all. Whether your measures work can be tracked over time: whether you are discovered in brand-free searches, whether the AI knows you by name and what share of voice you hold against competitors.