What is llms.txt? The format, purpose, and practical limits

llms.txt is a proposed Markdown convention for guiding agents to useful information about a website or project. It can provide concise context and links to supporting content. It is not an access-control file, a universal AI search requirement, or a guarantee that an assistant will cite your business.

Treat it as a maintained information aid. The source pages, facts, and permissions still matter, and publishing the file does not prove how any particular client uses it.

Who proposed llms.txt, and why?

The proposal is authored by Jeremy Howard and was originally published in September 2024. The current site identifies a v2 revision modified in August 2026. It describes a concise guide that helps agents find useful content without treating a large human-oriented website as one undifferentiated document.

That purpose is practical for documentation and other information-rich sites. It does not establish a new ranking entitlement. A client may support the convention for navigation without using it to determine public search recommendations.

The distinction matters because adoption claims are often broader than the evidence. A company publishing its own documentation file is not the same as confirming that every consumer assistant uses other sites' files for ranking.

What goes into the file?

The current proposal requires a top-level site or project title. Further summary information and link sections are optional. Organize the information so a reader or client can understand what the site covers and where detailed material lives.

The file should identify real public information. Use accurate names and descriptions, keep links current, and avoid including material that should remain private. An agent-readable format does not change the sensitivity of the underlying content.

Version matters. A checker based on an older assumption may classify an optional omission as an error. Read the current proposal and state which rules your implementation uses.

A simple llms.txt example

This fictional example illustrates organization rather than a required template for every business.

# Example Studio

> Product documentation and public service information.

## Documentation
- [Getting started](https://example.com/docs/start.md): Setup and basic concepts.
- [Reference](https://example.com/docs/reference.md): Detailed product information.

## Optional
- [Background](https://example.com/about.md): Company context.

Replace every example destination and description with maintained public information. A generated file should be reviewed by someone who knows the site. Do not publish a sample just because its syntax looks plausible.

Where does the file live?

The root /llms.txt path is common. The current proposal also permits an appropriate subpath for a defined part of a site. Choose a scope that makes sense for the information and keep the related destinations consistent.

After publishing, inspect the real response at that URL. Check the returned text, redirects, and links. A framework route that serves the normal website shell is not necessarily serving the intended guide.

Do not replace ordinary navigation, search eligibility, or public page content with the file. It is a supplementary representation, not a reason to make the main site less usable or accurate.

How does llms.txt differ from robots.txt?

robots.txt expresses crawler policy for compliant agents. llms.txt provides proposed informational guidance. Neither is a replacement for authentication when content must remain private.

An llms.txt link cannot override a firewall restriction, a login requirement, or a crawler's applicable policy. Likewise, an allowed robots rule does not ensure that the content is useful or selected for an answer.

Keep training, search discovery, and user-triggered access distinct. Provider documentation defines those roles. The AI crawler audit guide explains how to investigate them.

How does it differ from sitemap.xml and llms-full.txt?

A sitemap lists URLs for supported discovery workflows. An llms.txt guide is curated and descriptive. Neither guarantees indexing, retrieval, or source selection.

llms-full.txt is used in some documentation setups for an expanded body of content. Do not assume every website needs it or every client supports it. Evaluate whether the extra representation has a useful maintained purpose.

Duplicating a large content estate into another file can create upkeep problems if it becomes stale. Keep the original public facts authoritative and coordinate updates across representations where they exist.

Does llms.txt improve AI visibility?

This article does not establish a universal ranking benefit. A file can be useful to clients that support the convention, but publishing it does not prove they fetched it, relied on it, or changed an answer because of it.

Google's AI search guidance says no special AI text file is required for its search features. Platform-specific search and crawler controls should be reviewed separately.

LLM Listed has discussed its own observations in an article about llms.txt and answer-engine visibility (our llms.txt observations). Treat observational company data as a claim to inspect with its method and limitations, not as settled policy from the AI platforms. A before-and-after change can have several explanations.

How do you create and maintain one?

Start with the public information that matters. Select useful, accurate source pages, write concise descriptions, and identify the person who will keep them current. Use the llms.txt generator for a draft if helpful.

Review the output for stale facts, unsupported claims, confidential references, and inappropriate links. Publish it through the normal website process and verify the actual response. You can check your llms.txt file for retrieval, structure, and sampled public links.

Maintain it after significant changes. New product documentation, removed services, changed URLs, and rebrands should trigger review. Technical syntax is only part of quality; the content still needs to be true and useful.

What should you prioritize before the file?

Prioritize the pages and facts that buyers need. A public guide cannot compensate for an unclear service, inaccessible specification, wrong price, or inconsistent business identity.

Use actual answer evidence to investigate visibility. A format check and a citation sample are different observations. The AI search guide provides the wider process, while LLM optimization explains the discipline.

Frequently asked questions

What is llms.txt?

It is a proposed Markdown guide to useful information about a site or project. It can provide context and links for clients that use the convention. It is not a universal ranking requirement, permission file, or guarantee of citations. Keep the facts and supporting public pages accurate regardless of whether you publish one.

Is llms.txt required for ChatGPT or Google AI search?

This article does not establish it as a requirement for ChatGPT search or citation. Google explicitly says special AI text files are not required for its AI search features. Use current provider documentation for access and participation questions, and avoid turning the existence of a guide convention into a promised ranking rule.

Does the file need a summary and several sections?

The current proposal requires an H1 title, while further context and link sections are optional. Add them when they help explain the site and guide the reader. A validator should distinguish a required-format failure from an editorial suggestion. Do not classify every missing optional section as proof that the file is invalid.

Can I generate llms.txt automatically?

A generator can help create a starting draft, but someone should review the facts, scope, and links before publication. Automated extraction may include outdated or unsuitable material. Keep a maintenance owner and inspect the actual published response. A generated file does not remove the need for source accuracy or ordinary access checks.

How can I tell whether the file helped?

Separate file requests, format checks, and actual answer outcomes. Logs may show a request without proving how the content was used. A before-and-after visibility change may have several causes. Use a declared method and evidence, and avoid claiming a causal ranking effect from the file's publication alone.

Create a useful guide, then inspect your visibility

Generate an llms.txt draft and review it carefully. For a separate brand-level starting point, run a free AI visibility scan.