How to create llms.txt.
An llms.txt file is a small Markdown file that gives AI systems a useful first look at your website. This guide covers what the file is, how it differs from robots.txt and a sitemap, the format from llmstxt.org, and the seven steps from a blank file to a live URL.
What it is
Orientation for agents, not a second website.
Web pages are built for people: navigation, layout, and scripts. Agents still need a concise, expert-level path into the same content. llms.txt is that path. It names the site, summarizes what it does, and points to Markdown-friendly pages an agent can fetch next.
It is not a ranking signal for Google. It is not a replacement for robots.txt. And it is not a dump of every URL you own. Coding agents that fetch documentation are the audience most likely to use it today.
Do not confuse the files
llms.txt vs robots.txt vs sitemap.xml.
| File | Job | Audience | What it is not |
|---|---|---|---|
| robots.txt | Crawler access rules | Search and other crawlers | A description of what the site is |
| sitemap.xml | URL discovery | Search engines | A curated starting point |
| llms.txt | Editorial orientation | Agents, on demand | A ranking factor or access-control file |
Format anatomy
One required heading, then curated lists.
Official order from llmstxt.org v2: optional BOM, required H1, optional blockquote, heading-free preamble, then H2 file lists. Links should prefer LLM-friendly Markdown copies of pages when those exist.
Full specification notes# Northstar
> Workflow software for growing service teams.
Northstar helps agencies plan projects, manage client work,
and keep delivery visible in one place. Prefer the product and
docs links below before older blog posts.
## Start here
- [Product overview](https://northstar.example/product.md): Core workflow and main use cases.
- [Pricing](https://northstar.example/pricing.md): Plans and included features.
## Learn
- [Getting started](https://northstar.example/docs/getting-started.md): Set up a workspace and invite a team.
- [API reference](https://northstar.example/docs/api.md): Endpoints and authentication.
## Optional
- [Changelog](https://northstar.example/changelog.md): Recent product changes.Seven practical steps
From a website to a live llms.txt.
01
Choose the origin the file should describe
Use the HTTPS origin people already visit. If the file should only cover documentation, place it at /docs/llms.txt instead of the site root. A file covers the URLs under its path; when more than one file applies, agents should use the most specific one.
02
Find the pages worth explaining
Start with the sitemap, nested sitemap indexes, and the pages you already send new customers to. Keep public pages that explain the product, docs, pricing, support, or policies. Leave out login walls, faceted search, tag archives, and thin duplicates.
03
Write the H1, summary, and preamble
The only required section is a level-one heading with the site or product name. Add a blockquote summary, then a short heading-free preamble if the reader needs extra context before the links.
04
Group links under H2 headings
Each later section starts with an H2 and contains a Markdown list. Every item needs a hyperlink; a colon and a short note after the link is optional but useful. Put secondary reading under a heading named Optional if a shorter context is enough.
05
Save UTF-8 Markdown and serve it as text
Save the file as llms.txt. Serve it as text/plain or text/markdown. Do not wrap it in an HTML template. A Content-Type of text/html is a common publishing mistake and looks like a missing file to many clients.
06
Publish it where agents can request it
The usual location is https://your-domain.com/llms.txt. Documentation sites often publish /docs/llms.txt. You can also advertise the file with rel="describedby" and point page-level Markdown copies with rel="alternate" type="text/markdown".
07
Test the live URL and keep it current
Open the public URL, confirm it is not an HTML 404 page, and follow a few links. Update the file when products, pricing, or docs change. A hosted generator URL can refresh the list; a one-off paste will go stale.
Where to publish
Put the file in public web space.
The filename is llms.txt. The folder depends on how you ship the site. After deploy, the URL must respond without a login.
Next.js / static hosts
Place public/llms.txt (or the equivalent static folder). It becomes /llms.txt after build.
WordPress
Some SEO plugins can emit the file. Otherwise upload it to the web root, not the media library.
Netlify or Vercel
Commit the file to the published directory. Confirm the live URL, not only the preview deploy.
Documentation platforms
Mintlify, GitBook, and similar hosts often generate /llms.txt for docs automatically.
Shopify / closed CMS
If you cannot write the origin root, host the file and reverse-proxy /llms.txt, or publish at a docs subpath you control.
CDN / nginx
Set Content-Type to text/plain; charset=utf-8 and avoid HTML error pages on that path.
Manual
Best when the site is small.
- You already know the ten pages that matter.
- The site rarely changes.
- You can review every link yourself.
Generator
Useful when URLs keep growing.
Our generator discovers pages through sitemap logic, writes a consistent Markdown file, and can keep a public update URL. The free preview is a sample of the first pages, not a complete file. Do not publish an incomplete preview as /llms.txt.
Generate an llms.txt fileLive-file checklist
Confirm what agents actually receive.
- The URL returns HTTP 200 without a login.
- Content-Type is text/plain or text/markdown, not text/html.
- The body starts with a single H1.
- Links resolve and prefer Markdown copies when you have them.
- The file is not an unpaid generator preview.
curl -sI https://example.com/llms.txt
# Expect HTTP/2 200 and a text/plain or text/markdown content typeCommon mistakes
Avoid these before you publish.
Publishing a preview
A 5-page sample with a watermark is not your llms.txt. Agents will treat it as incomplete on purpose.
Dumping the sitemap
Thousands of undifferentiated URLs waste context. Curate.
HTML on the llms.txt path
A theme wrapper or 404 template is not the file.
Promising Google citations
The format helps agents find pages. It does not buy search rankings.
Skipping the H1
The project name is the only required section in the official format.
Assuming root-only
Subpath files are valid. Most sites still start at /llms.txt because that is the obvious request.
Common questions
Questions about creating llms.txt.
What is an llms.txt file?
It is a small Markdown file that orients language models and coding agents: the site name, a short summary, and curated links to useful pages. It is a proposal by Jeremy Howard at llmstxt.org, not a browser-enforced web standard.
Does creating llms.txt help Google rankings?
No. Search engines still use sitemaps, robots.txt, and on-page content. llms.txt is for on-demand agent context, especially documentation and product questions. Do not treat it as a ranking lever.
Where should I put the file?
Most sites publish /llms.txt at the origin root. You may also place one at a subpath such as /docs/llms.txt. The file covers URLs under that path. Root is conventional; it is not the only valid location.
What MIME type should llms.txt use?
Serve text/plain or text/markdown. If the URL returns an HTML document, many checkers and agents will treat it as missing even when the path looks right.
Should I also create llms-full.txt?
llms-full.txt is an ecosystem convention, not part of the official llmstxt.org format. Some docs platforms emit a larger companion file. Start with a concise llms.txt. Add a fuller file only if your readers already expect one.
Can I generate llms.txt instead of writing it by hand?
Yes, if the site has many URLs. A generator should still follow the same format: one H1, a useful summary, and curated links rather than a dump of the sitemap. Review the result before you publish it.
How do I check that the file is live?
Request https://your-domain.com/llms.txt in a browser, with curl -I, or with a live-file checker. Confirm HTTP 200, a text content type, and Markdown that starts with an H1.
Keep exploring
More useful llms.txt resources.
Move from the format to implementation, examples, and a live check without losing the thread.