Last updated November 17, 2026

What is llms.txt? Does it help AI search?

llms.txt is a proposed text file that lists a website's key pages in Markdown for AI tools. As of November 2026, Google says Search ignores it, and an Ahrefs study found 97% of these files got no requests in May 2026.

thesocialtheory.com/llms.txt
# The Social Theory

> Nathan Swift is an AEO and local SEO specialist who builds AI-ready websites that help small businesses get found on Google and in AI answers.

## Start here
- [AI Search Optimization for Small Business](https://thesocialtheory.com/): how to get named in ChatGPT, Google and AI answers, a free check for your site and real results
- [AI Search Guides for Small Business](https://thesocialtheory.com/blog/): every guide, grouped by the question it answers
- [Answer Engine Optimization Services for Small Businesses](https://thesocialtheory.com/answer-engine-optimization-services/): what Nathan does for small businesses, what AEO usually costs, real results and how to start

## AI search guides
...

The first lines of our own llms.txt, rewritten in November 2026.

Question 1

What is llms.txt?

Short answer

llms.txt is a Markdown file at a website's root that lists its most useful pages for AI tools. Jeremy Howard proposed it in September 2024.

The format, from the llmstxt.org proposal, shown with our own file.

“We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.”

Jeremy Howard, who proposed llms.txt, published September 3, 2024: The /llms.txt file
llms.txtrobots.txt
What it doesLists your pages, with notes, for AI tools to readTells crawlers which pages they may fetch
Who reads itGoogle says Search ignores it. OpenAI, Anthropic and Perplexity don't say their crawlers read itGoogle, OpenAI, Anthropic and Perplexity all say their crawlers follow it
Blocks anything?No. It controls nothing and blocks nothingYes, for every crawler that follows it

Robots.txt rules come from each company's crawler page. "Controls nothing and blocks nothing" is Ahrefs' description of llms.txt.

What this means for you

llms.txt is a table of contents written for AI tools. It sits at yoursite.com/llms.txt, next to robots.txt, but it does a different job: it suggests what to read, and it can't hide or open up any page.

The proposal says the file is used "on demand, when an agent needs information about a topic while assisting a user." So it helps only when an AI tool goes looking for it.

Question 2

Does llms.txt help you show up in AI search?

Short answer

No, llms.txt doesn't help you show up in Google Search or its AI answers. A 137,000-site study also found AI search bots almost never request the file.

“You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.”

Google Search Central, documentation, read November 17, 2026: Optimizing your website for generative AI features
97%

of llms.txt files got zero requests in May 2026.

Ahrefs checked 137,210 sites. About 38,000 had a valid llms.txt.

“If your goal is showing up in ChatGPT, Perplexity, or AI Overviews, an llms.txt file is largely decoration.”

Louise Linehan, Ahrefs, June 15, 2026: We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read

What this means for you

Two different checks point the same way. Google says its Search and AI answers ignore the file. Ahrefs looked at what bots request and found almost nothing asks for it.

Google also says keeping one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." So a file you already have is fine. Building one won't move your rankings.

One limit: Ahrefs' sites skew technical, and it calls every number a ceiling, because a request doesn't prove anything read the file.

Question 3

Who uses llms.txt files today?

Short answer

Mostly non-AI bots and tools fetch llms.txt files, and AI agents are the biggest AI group. In Ahrefs' May 2026 study, AI search bots made 1.1% of requests.

Who requested llms.txt filesShare of requestsDetail
Bots and tools that aren't AI77%Ahrefs: "77% of the bots fetching llms.txt aren't AI tools at all"
AI agents10.5%Includes Claude Code, Anthropic's coding agent
AI training crawlers5.3%GPTBot 4.51%, ClaudeBot 0.8%
People4%Visits from people, not bots
AI assistants2.5%
AI search bots1.1%OAI-SearchBot, ChatGPT's search crawler, 0.74%

Share of all llms.txt requests in May 2026, from the 3% of files that got any. Ahrefs, June 15, 2026.

What this means for you

llms.txt was built for AI agents that read documentation, like coding assistants. That is who comes closest to using it. In Ahrefs' words, "Slackbot alone fetched llms.txt files more often than PerplexityBot did."

OpenAI and Perplexity both link an llms.txt from their own developer docs, as an index for coding tools. Their crawler pages control access with robots.txt and say nothing about reading the llms.txt on your site (see the OpenAI and Perplexity crawler pages).

Question 4

Should a small business have an llms.txt file?

Short answer

A small business doesn't need an llms.txt file to show up in Google or AI answers. If your website platform makes one for free, keep it accurate.

Your platform makes one for free

Keep it. Check a few times a year that it describes your business the way you do today.

Costs you nothing, can't hurt your Google rankings.

You'd have to build it by hand

Skip it for now. Put that hour into your pages and your Google Business Profile.

That is where people and AI tools find you.

What happened to ours

Before: checked November 17, 2026

Our llms.txt still described the business we used to run.

  • "helps small businesses build their brand and learn to use AI"
  • Linked "Learn" and "AI News", menu labels we had already retired
  • Sent "Contact" to /#contact/, a typo that skips the contact form
After: rewritten the same week

One line that matches the rest of the site, and every guide listed by its current title.

  • "Nathan Swift is an AEO and local SEO specialist who builds AI-ready websites..."
  • Grouped the way the guides page is grouped
  • Older case studies moved to the Optional section

What this means for you

A file nobody reads goes stale without anyone noticing. Ours did. If you keep one, it should say the same thing your website, your Google Business Profile and your structured data say.

Some audit tools still score it. The free AEO Site Checker gave our site 5 of its 100 points for having an llms.txt. Treat those points as tidiness, not visibility.

Question 5

How do you create an llms.txt file?

Short answer

To create an llms.txt file, list your key pages in a short Markdown file. Start with your business name and a one-line summary, then upload it to yoursite.com/llms.txt.

Template: llms.txt for a small businessSwap in your pages, save it as llms.txt, upload it to your site's root.
# Your Business Name

> One sentence: what you do, for whom, and where.

## Main pages
- [Services](https://yoursite.com/services/): what you offer and what it costs
- [About](https://yoursite.com/about/): who you are, licenses and service area
- [Contact](https://yoursite.com/contact/): phone, hours and how to book

## Guides
- [Your most useful guide](https://yoursite.com/your-guide/): the question it answers

## Optional
- [Blog](https://yoursite.com/blog/): older posts and news

AWrite it in Markdown

Start with your business name as a heading, then one sentence in a quote block. List your key pages under short headings, each with a few words on what it covers.

BUpload it to your site's root

It has to open at yoursite.com/llms.txt as plain text. Some website platforms make one for you (Ahrefs names Wix). Otherwise, ask yours where to upload it.

CKeep it true

Update it when you add, rename or retire a page. A stale file tells AI tools the wrong thing about you.

Question 6

What helps you show up in AI search instead?

Short answer

Pages that AI tools can crawl, index and quote help you show up in AI search. Lead with direct answers, add real numbers and sources, and run a free check.

  • Google can crawl and index the page: robots.txt allows it and it loads without a login
  • The page opens with a direct answer to the question, in 25 to 50 words
  • Headings are the questions your customers ask
  • Real numbers, named sources and first-hand detail
  • Your name, phone and hours match on your site, your Google Business Profile and your structured data
94
Run it freeFree AI visibility checkSee all 27 checks on a real site (Forg3d scored 94), and run yours in about 10 seconds.

What this means for you

These are what Google's guide and the GEO study point to. Our AI Overview optimization guide lists the tricks Google says to skip, llms.txt among them. For ChatGPT, how to rank in ChatGPT covers what its search reads.

The GEO study found that adding quotes, statistics and sources made pages show up more in AI answers. That is time better spent than a text file.

Question 7

Can someone set up my website for AI search?

Short answer

Yes, an AEO expert can set up your website for AI search. I build answer-first pages on Halo, quoted for your business, or you can run Halo yourself.

What this means for you

Halo is the AI website you run by text that I build on. Every Halo site ships with sitemaps, structured data and an AEO readiness score, so the technical checks on this page are handled. You text Halo the questions your customers ask, and it builds the pages.

Get it built by an AEO expert

I'm an AEO and local SEO specialist. I find the questions your customers ask and build answer-first pages for them on Halo.

Quoted for your business.Work with me

Do it yourself on Halo

Halo Pro is $99 a month for two websites on your own domains, and Halo Start is free for one page.

Checked on huzihalo.com, November 16, 2026. Prices can change.See Halo pricing
The short version

What is llms.txt: every answer in one place

  1. What is llms.txt?

    llms.txt is a Markdown file at a website's root that lists its most useful pages for AI tools. Jeremy Howard proposed it in September 2024.

  2. Does llms.txt help you show up in AI search?

    No, llms.txt doesn't help you show up in Google Search or its AI answers. A 137,000-site study also found AI search bots almost never request the file.

  3. Who uses llms.txt files today?

    Mostly non-AI bots and tools fetch llms.txt files, and AI agents are the biggest AI group. In Ahrefs' May 2026 study, AI search bots made 1.1% of requests.

  4. Should a small business have an llms.txt file?

    A small business doesn't need an llms.txt file to show up in Google or AI answers. If your website platform makes one for free, keep it accurate.

  5. How do you create an llms.txt file?

    To create an llms.txt file, list your key pages in a short Markdown file. Start with your business name and a one-line summary, then upload it to yoursite.com/llms.txt.

  6. What helps you show up in AI search instead?

    Pages that AI tools can crawl, index and quote help you show up in AI search. Lead with direct answers, add real numbers and sources, and run a free check.

  7. Can someone set up my website for AI search?

    Yes, an AEO expert can set up your website for AI search. I build answer-first pages on Halo, quoted for your business, or you can run Halo yourself.

Start with a free AI visibility check or read what AEO is next.

Sources

Where these facts come from

Skip the file. Fix the pages.

Have an AEO expert build pages AI tools quote, or see how AI tools read your site today.

Want an AEO expert to set it up?

I'm an AEO and local SEO specialist. Get in touch and tell me which questions you want AI tools to answer with your business. Or read what Halo costs and my Huzi Halo review first.