📝 llms.txt Spec

Proposed community-driven standard providing Large Language Models (LLMs) and AI agents with a clean, structured, and curated overview of a website’s content.

---

🚀 Quick Start / Deployment

• Location: Host the files at the root of the target domain:
  • https://example.com/llms.txt — Streamlined navigation and high-level table of contents.
  • https://example.com/llms-full.txt — Optional full text compilation of all target documents.
• Static Serving: Add raw text files directly to public or content directories of static site generators (Docusaurus, VitePress, Quartz).

---

🔧 Configuration / Commands

File Syntax Specification

A compliant llms.txt file is structured in clean Markdown and must follow this precise order:

# Site/Project Name
 
> Blockquote containing a short, descriptive context summary.
 
Detailed project descriptions and usage guidelines go here.
 
## Navigation Group Heading
 
- [Link Name](https://example.com/target): Optional description text separated by a colon.
- [Additional Link](/local-path): Description details.

• H1 Header (Required): Represents the project name on the first line.
• Blockquote (Required): One-paragraph summary of the website’s context.
• H2 Navigation Lists: Group links logically into sections with H2 headers.
• Links: Written as standard Markdown links [Name](URL) followed optionally by : Description.

---

📦 Dependencies / Modules

• Automation Tools:
  • WordPress: Integrate via plugins (e.g., Yoast SEO, AIOSEO).
  • CI/CD: Generate dynamically during builds using project crawlers or custom scripts.

---

💡 Best Practices

• Jargon-Free Language: Keep descriptions simple and direct for optimal LLM context parsing.
• Clean Markdown: Avoid complex HTML tags or styles within files to maintain scraper compatibility.
• Relative Paths: Use absolute URLs or root-relative paths for unambiguous link resolution.

---

🚨 Gotchas / Traps / Warnings

• Not an Official Protocol: Major AI platforms do not officially commit to parsing llms.txt automatically.
• Complementary to Robots: llms.txt guides attention but does not replace permission controls in robots.txt.
• Token Optimization: Keep llms.txt high-level and offload full texts to llms-full.txt to minimize token overhead.