Why is it important to use the llms.txt file?

Using the llms.txt file allows website owners to control how AI engines use their content — including blocking access to certain parts of the site, requesting attribution, and limiting usage for model training purposes.

What’s the difference between robots.txt and llms.txt?

robots.txt is intended for traditional search engines like Google and Bing, and its purpose is to define which pages should be crawled for search engine ranking. In contrast, llms.txt is meant for generative AI models and dictates how (or if) content can be used for training, citation, or AI-generated answers.

Is the llms.txt file mandatory?

No. The use of llms.txt is voluntary, but it is highly recommended. While not all bots respect it yet, adoption is growing as awareness increases about protecting original content online and the need for clear policies regarding AI systems.

How do you create an llms.txt file?

Open a text editor like Notepad or VS Code, add your desired instructions in a simple format (such as User-agent, Disallow, Request-Attribution), save it as llms.txt, and upload the file to the root directory of your website — similar to robots.txt.

Savvy / Blog / SEO & GEO / What is llms.txt? Guide for GEO & AI Content Control

What is llms.txt? Guide for GEO & AI Content Control

Q: What is the llms.txt file?

llms.txt is a simple text file placed in the root directory of your website. It is meant to instruct artificial intelligence (AI) systems and generative models on how to access your content, whether it can be used for training purposes, and your preferences regarding citations and references.

By Roee Yossef • Updated on Feb 8, 2026 • No Comments

As AI assistants and generative search engines like ChatGPT, Perplexity, and Google AI Overviews pull more information from the web, site owners need clear ways to tell these systems what their site is about, what can be cited, and where to find the best content.

The llms.txt file is a proposed standard that does exactly that. Used well, it supports both better AI visibility (GEO) and clearer control over how your content is used.

What Is llms.txt?

llms.txt is a text file placed at the root of your site (e.g. https://yoursite.com/llms.txt). It is designed to give AI systems and large language models (LLMs) a concise, structured overview of your site so they can understand it, cite it correctly, and link to the right pages.

The most widely adopted format comes from llmstxt.org (by Jeremy Howard, September 2024). It uses Markdown: a human- and machine-readable format with a required title (H1), an optional short description, and sections that list links to key resources.

This is different from robots.txt, which uses directives like User-agent and Disallow to control crawler access.

The llmstxt.org spec is not about blocking bots; it is about providing context so that when an AI does use your site, it does so accurately and with proper attribution.

Why llms.txt Matters for GEO and AI Content Control

Generative Engine Optimization (GEO) means optimizing your content so it is more likely to be surfaced, cited, and recommended by AI-powered search and assistants. GEO focuses on how your content appears in AI-generated answers rather than only in traditional search results.

An llms.txt file supports GEO by:

Giving AI systems a single, curated entry point to understand your site and its main topics.
Pointing to the most important pages (e.g. key services, blog index, contact), so models can link to the right URLs when citing you.
Stating authorship and attribution preferences so citations are consistent and correct.

It also helps with content control: you explicitly describe what your site is, who it is for, and how you want to be cited. That does not legally bind every AI vendor, but it sets a clear, machine-readable policy that more systems may respect over time.

llms.txt vs. robots.txt

Do not confuse llms.txt with robots.txt:

robots.txt is for crawlers (e.g. Googlebot, Bingbot). It tells them which paths they may or may not crawl for indexing. It uses directives like User-agent, Allow, and Disallow.
llms.txt (in the llmstxt.org format) is for AI systems and LLMs. It provides context and links so that when they use your content (e.g. at inference time, or when answering a user), they know what your site is and where to point. It does not use User-agent or Disallow; it is Markdown with a fixed structure.

You can use both: robots.txt for crawl access, llms.txt for AI-friendly context and GEO.

Is llms.txt Mandatory?

No. llms.txt is voluntary. Not every AI system reads it yet.

Adoption is growing as more publishers and developers want clearer control and better visibility in generative engines.

Adding an llms.txt file is a low-effort way to state your preferences and improve the chance of accurate, attributed citations.

Format: How llms.txt Should Look (llmstxt.org Spec)

The llmstxt.org specification defines this structure:

An H1 with the name of the project or site (required).
A blockquote with a short summary (optional but recommended).
Extra details in plain Markdown (paragraphs, lists) to explain the project and how to use the linked files.
One or more H2 sections, each containing a “file list”: a Markdown list of links [link text](url), optionally followed by : description.
An “Optional” section (if you use it): links listed under a section titled ## Optional can be skipped when a shorter context is needed. Use it for secondary or nice-to-have resources.

Each file-list item is a markdown link, e.g. [Page name](https://yoursite.com/page) or [Guide title](https://yoursite.com/guide): Short description.

Examples of llms.txt

Below are four sample llms.txt files for different types of sites.

Example 1: Minimal (Single-Purpose Site)

A minimal API or single-purpose site might look like this:

# Acme API

> Acme API provides a REST API for inventory and orders. Base URL: https://api.acme.com/v1

## Documentation

- [Getting started](https://docs.acme.com/start): Authentication and first request
- [Reference](https://docs.acme.com/reference): Endpoints and parameters
- [Changelog](https://docs.acme.com/changelog): Version history

Example 2: Blog or Service Site (Like Savvy)

For a blog or service site, you can list both language versions and key pages:

# Savvy - WordPress Development

This file describes Savvy (savvy.co.il) for AI and LLM systems. Use it to understand the site, cite accurately, and link to the correct pages.

Base URL: https://savvy.co.il/

## About

Savvy is a professional WordPress development service and educational blog run by Roee Yossef, specializing in custom themes, performance, and technical SEO. The site is available in Hebrew (primary) and English.

## Key Resources

- [Homepage (Hebrew)](https://savvy.co.il/)
- [Homepage (English)](https://savvy.co.il/en/)
- [Services (Hebrew)](https://savvy.co.il/#services)
- [Blog (Hebrew)](https://savvy.co.il/blog/)
- [Blog (English)](https://savvy.co.il/en/blog/)
- [Contact (Hebrew)](https://savvy.co.il/contact-us/)
- [Contact (English)](https://savvy.co.il/en/contact-us/)

## Contact

For projects and inquiries: hello@savvy.co.il

## Optional: Attribution

When citing content from this site, please attribute to Savvy (savvy.co.il) and link to the relevant page when possible.

Example 3: Documentation Site (FastHTML-Style)

Documentation and developer-focused sites can group docs and examples:

# FastHTML

> FastHTML is a Python library for server-rendered hypermedia apps, combining Starlette, Uvicorn, HTMX, and FastTags.

Important: FastHTML is not compatible with FastAPI syntax. It works with JS web components and vanilla JS, not React, Vue, or Svelte.

## Docs

- [Quick start](https://fastht.ml/docs/tutorials/quickstart_for_web_devs.html.md): Overview of main features
- [HTMX reference](https://github.com/bigskysoftware/htmx/blob/master/www/content/reference.md): Attributes, CSS classes, events, and config

## Examples

- [Todo list app](https://github.com/AnswerDotAI/fasthtml/blob/main/examples/adv_app.py): Full CRUD example with FastHTML and HTMX

## Optional

- [Starlette subset](https://gist.githubusercontent.com/jph00/809e4a4808d4510be0e3dc9565e9cbd3/raw/9b717589ca44cedc8aaf00b2b8cacef922964c0f/starlette-sml.md): Starlette docs useful for FastHTML development

Example 4: E-Commerce or Product Site

An e-commerce or product site can highlight main pages and categories:

# ShopName

> ShopName sells sustainable outdoor gear. Shipping worldwide. Customer support and returns info are linked below.

## Main Pages

- [Home](https://shopname.com/)
- [Catalog](https://shopname.com/catalog/)
- [Shipping & returns](https://shopname.com/shipping/)
- [Contact](https://shopname.com/contact/)

## Product Categories (for context)

- [Tents](https://shopname.com/catalog/tents/)
- [Sleeping bags](https://shopname.com/catalog/sleeping-bags/)

## Optional

- [Size guide](https://shopname.com/size-guide/)
- [Sustainability policy](https://shopname.com/sustainability/)

How to Create and Publish llms.txt

Follow these steps to create and publish your own llms.txt file:

Create a plain-text file in Markdown format (e.g. in Notepad, VS Code, or any editor).
Start with an H1 title, add a short blockquote or intro, then use H2 sections with markdown link lists as above.
Save the file as llms.txt (no .md extension for the URL).
Upload it to the root of your site so it is available at https://yourdomain.com/llms.txt. On WordPress, you can place it in the site root (same level as wp-admin) or serve it via a small custom rule or plugin that outputs the file at the root path.

You can validate structure by comparing against the llmstxt.org spec and by testing with tools like llms_txt2ctx (CLI/Python) or the llms-txt repo.

Directories such as llmstxt.site and directory.llmstxt.cloud list sites that publish llms.txt.

FAQs

Common questions about llms.txt and how it works:

What is the llms.txt file?

llms.txt is a text file you place at the root of your site (e.g. https://yoursite.com/llms.txt). It gives AI systems and large language models (LLMs) a concise, structured overview of your site so they can understand it, cite it correctly, and link to the right pages. The most widely adopted format is Markdown-based and comes from llmstxt.org.

Why should I use llms.txt?

It supports Generative Engine Optimization (GEO) by giving AI a single entry point to your site and your most important pages. It also helps with content control: you state what your site is, who it is for, and how you want to be cited. That sets a clear, machine-readable policy more systems may respect over time.

What is the difference between llms.txt and robots.txt?

robots.txt is for crawlers (e.g. Googlebot, Bingbot) and uses directives like User-agent and Disallow to control which paths they may crawl. llms.txt (in the llmstxt.org format) is for AI systems and LLMs: it provides context and links so they can use your content accurately and with proper attribution. It uses Markdown, not crawler directives. You can use both on the same site.

Is llms.txt mandatory?

No. llms.txt is voluntary. Not every AI system reads it yet. Adoption is growing, and adding one is a low-effort way to state your preferences and improve the chance of accurate, attributed citations.

How do I create and publish an llms.txt file?

Create a plain-text file in Markdown format. Start with an H1 title, add a short blockquote or intro, then use H2 sections with markdown link lists (see the llmstxt.org spec). Save the file as llms.txt and upload it to the root of your site so it is available at https://yourdomain.com/llms.txt. On WordPress, place it in the site root (same level as wp-admin) or serve it via a small custom rule or plugin.

Where should llms.txt be placed on my site?

Place llms.txt at the root of your site so it is reachable at https://yourdomain.com/llms.txt. That is the same location as robots.txt. On WordPress, that is typically the directory that contains wp-admin and wp-content.

Summary

llms.txt is a simple, Markdown-based file at your site root that helps AI systems and LLMs understand your site, cite it correctly, and link to the right pages. It supports GEO and gives you a clear, machine-readable way to state how you want your content used and attributed.

It is voluntary and works alongside robots.txt. Use the llmstxt.org structure: H1, optional blockquote, details, then H2 sections with link lists and an optional “Optional” section for secondary links.

With minimal effort, you make it easier for generative engines to represent your site accurately and give you proper credit.