Bruno Sarlo
da9207ded3
agents.md protocol for AI agent web discovery. Key features: - Two formats: Pure Markdown (simple) or YAML frontmatter (structured) - MCP gateway integration for tool access - Discovery via /.well-known/agents.md - Security: origin trust, endpoint validation, auth guidance - Backward compatible with robots.txt and llms.txt Design based on 3-iteration process: 1. Gap analysis and planning 2. Multi-model consensus on format decisions 3. Code review for completeness and clarity Philosophy: robots.txt says what agents CANNOT do, agents.md says what they CAN do. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
agents.md
Tell AI agents what they can do on your website.
The Problem
| File | What it tells agents |
|---|---|
| robots.txt | What you cannot access |
| llms.txt | What content matters |
| ??? | What you can do |
The Solution
Create /.well-known/agents.md:
# My Site
An online bookstore.
## Can
- Search catalog
- Read book details
- Check availability
## Cannot
- Place orders (requires human)
## Contact
hello@mysite.com
That's it. Plain text. Human readable. Machine parseable.
With MCP Gateway
Point agents to your MCP server for structured tool access:
---
version: "1.0"
mcp:
endpoint: https://mysite.com/.well-known/mcp
transport: streamable-http
auth: oauth2
---
# My Site
An online bookstore.
## Can
- Search and browse
- Check prices
- Place orders (authenticated)
## Contact
hello@mysite.com
How It Works
Agent requests /.well-known/agents.md
│
├─► Basic: reads text, understands capabilities
│
└─► Advanced: connects to MCP gateway for tools
Documentation
- Specification - Full protocol spec (v1.0.0-draft)
- Examples - Real-world examples
- FAQ - Common questions
- Changelog - Version history
Quick Comparison
| Aspect | robots.txt | llms.txt | agents.md |
|---|---|---|---|
| Purpose | Crawl control | Content summary | Capabilities |
| Format | Custom | Markdown | Markdown + YAML |
| MCP | No | No | Yes |
Status
v1.0.0-draft - First public draft release
Feedback welcome via issues.
Related Standards
- robots.txt - RFC 9309 (1994)
- llms.txt - Jeremy Howard (2024)
- AGENTS.md - OpenAI/Sourcegraph (2025)
- MCP - Anthropic (2024)
License
CC0 1.0 Universal - Public Domain