llms.txt Guide: What Google Says and How to Check It

llms.txt is a Markdown document placed at the top-level address of your site (/llms.txt). It describes your site and lists key pages in a form that is easy for AI to read. In its AI optimization guide, Google states that Google Search ignores this file. It also explains that creating one does not help or hurt your search visibility or rankings. So if your goal is to look better to Google, there is no reason to make one. Create it only if you want a site guide for other AI services or agents.

What is llms.txt, and do you need one?

Many people read posts saying "llms.txt is essential in the AI era" and wonder if their site needs one. llms.txt is like a store directory handed to a new visitor. In a big store, the directory shortens the walk. But if visitors already find the store through signs and road markers (links and sitemaps), they get in without it. Google Search also understands your site through indexed pages and links, with no need for this file. This analogy does leave out one thing, though: whether each AI service actually reads the file varies.

The proposal comes from Jeremy Howard, and llmstxt.org publishes the format as a convention. On June 15, 2026, Google noted in the Search Central update log that it "clarified the guidance on llms.txt files." The original guide says "creating and maintaining an llms.txt file is perfectly fine" and also says "Google Search ignores it, so it does not help or hurt your visibility or ranking in Google Search."

This judgment applies only to Google Search. Google's documents do not say whether other services, such as ChatGPT or Claude, read this file. So before you create one, decide who you want it to be read by.

Situation	Should you create it?
You want to raise your Google Search ranking	No effect. Spend your time on indexing and content improvements
Your site is developer or API documentation that AI agents read often	Worth making. There are public examples, such as the Cloudflare developer docs
A blog or intro site with only a few posts	Optional. If you make one, you must update it whenever your posts change

llms.txt vs robots.txt: the difference

The two files have similar names but do different jobs. robots.txt is an access rule that decides which addresses a crawler may enter. llms.txt is a guide that introduces pages worth reading. So even if you write "do not collect" in llms.txt, it cannot block AI crawlers.

Item	robots.txt	llms.txt
Role	Allow or block crawler access	Introduce pages to read
Format	Rules such as User-agent and Disallow	Title, summary, link list (Markdown)
Used by Google Search	Used to control crawling	Not used
Can it block AI collection?	Yes, per crawler	No

According to Google's AI features guide, you must allow crawling in robots.txt and your hosting settings. To limit Google's use of your content for AI training or as grounding material, use Google-Extended. If your goal is to block AI crawlers, you need to edit the AI crawler allow and block settings in robots.txt, not llms.txt. A blocking example is covered in how to block Meta's crawler.

How to create llms.txt

In the llmstxt.org specification, the only required element is a single top-level title (H1). After that, you place a one-line summary (blockquote), an optional description paragraph, and link lists under subheadings (H2), in that order. You can add a colon after a link to describe what the page is. Supplementary material that can be skipped goes under an H2 called "Optional".

For a blog, it looks like the example below. Change the addresses and post titles to match your own site.

# 내 블로그 이름

> 마케팅 실무자를 위한 광고·검색 가이드 블로그입니다. 글은 한국어로 쓰고 기준일을 밝힙니다.

## 대표 가이드

- [서치콘솔 등록 방법](https://example.com/ko/blog/search-console): 소유권 확인부터 색인 요청까지
- [UTM 만드는 법](https://example.com/ko/blog/utm): 파라미터 규칙과 GA4 확인 위치

## Optional

- [전체 글 목록](https://example.com/ko/blog)

1. Write the structure above in a plain editor such as Notepad and save it as llms.txt. Use UTF-8 encoding.
2. Upload it to the top-level folder of your site so it opens at https://yourdomain/llms.txt. On WordPress, upload it through your hosting file manager, or use a plugin that generates the file, such as the llms.txt generation feature in the All in One SEO plugin.
3. Include only pages that open right now. If deleted posts remain, the guide gives wrong information instead.

How to check that llms.txt is live

The job is not done once the address opens. Check these three things in order.

1. Open the /llms.txt address in a browser. If you see the file content as is, it worked. If you see an error page styled like your site, or a 404 message, the file was not uploaded. In a terminal, run curl -I with the address and check that the first line shows 200.
2. Check that the text is not garbled. It is fine if Content-Type in the response header includes charset=utf-8 and the text on screen looks normal.
3. Look at the agentic browsing item in Lighthouse in Chrome DevTools. According to the Chrome Lighthouse guide, this item checks whether a server error occurs when fetching llms.txt. If the server returns 404, providing the file is optional, so it shows as not applicable (N/A) rather than a failure.

Here is an example of checking in a terminal. If the first response line and the Content-Type line match the ones below, it passes.

curl -sI https://내도메인/llms.txt

HTTP/2 200
content-type: text/plain; charset=utf-8

Response status	Lighthouse result	What to do
200, content visible	No server error	Click through the links once to confirm they open
404 (no file)	Not applicable (N/A)	Leave it as is if you do not need a guide file
Server error (5xx)	Fail	Fix the hosting settings or the plugin generation error

So Lighthouse shows a failure not when the file is missing, but when the address returns a server error. This check does not judge whether the content is good. It only checks whether the file can be fetched.

Common mistakes

• Creating it to raise rankings: Google Search ignores this file. Do not expect any effect, and check your Google visibility with Search Console indexing requests and the performance report.
• Using it as a blocker: llms.txt is not an access rule, so it cannot stop collection. To block, use robots.txt.
• Creating it once and forgetting it: if deleted or moved posts stay in the list, the guide is wrong. Update it whenever you change your post structure.
• Uploading only PDFs and pointing to them: a PDF whose text cannot be selected may not be readable, so start with how to check PDF indexing.

Frequently asked questions

What is the difference between llms.txt and llms-full.txt?

llms.txt is a short guide centered on a link list. llms-full.txt is a variant that puts the full text of your documents in a single file. Unless you run a long documentation site, llms.txt alone is enough.

Where do I put the llms.txt file?

The default is the top-level address of your site (for example, https://yourdomain/llms.txt). The llmstxt.org specification explains that it can also be placed in a subpath, and that such a file covers only addresses under its own path.

What should I do to appear in Google's AI features?

According to Google's AI features guide, you do not need a separate AI-specific file or markup. Your page just needs to be indexed and eligible to appear in Search with a snippet.