agent-protocol/spec

agents.md Protocol Specification

Version: 0.2.0 Status: Draft Updated: 2026-01-14

Abstract

A simple text file that tells AI agents what they can do on a website and optionally points them to an MCP gateway for structured tool access.

Philosophy

Standard	Tells agents...
robots.txt	What you cannot access
llms.txt	What content is important
agents.md	What you can do + where to connect

1. Discovery

Location: /.well-known/agents.md or /agents.md

Content-Type: text/markdown or text/plain

Agents request the file like any HTTP resource:

GET /.well-known/agents.md HTTP/1.1
Host: example.com
User-Agent: MyAgent/1.0

2. Format

Plain Markdown. Human readable. Machine parseable.

Minimal Example

# Example Site

A bookstore since 2010.

## Can
- Search catalog
- Read book details
- Check availability

## Cannot
- Place orders without human
- Access user accounts

## Contact
agents@example.com

With MCP Gateway

# Example Bookstore

Online bookstore with 50,000 titles.

## Can
- Search and browse catalog
- Read reviews and descriptions
- Check prices and stock
- Place orders (authenticated)

## Cannot
- Modify user accounts
- Access admin functions

## MCP
endpoint: https://example.com/.well-known/mcp
transport: streamable-http

## Behavior
- Respect 1 request/second
- Cache product data 1 hour
- Identify in User-Agent header

## Contact
agents@example.com

3. Sections

All sections are optional. Use what makes sense.

Identity (Header)

# Site Name

Brief description of what this site is.

Capabilities (Can/Cannot)

## Can
- Action agents are allowed to take
- Another allowed action

## Cannot
- Restricted action
- Another restriction

MCP Gateway

## MCP
endpoint: <url>
transport: streamable-http | sse | stdio
auth: none | api_key | oauth2

The MCP section points agents to a Model Context Protocol server for structured tool access. This is the bridge from simple text discovery to full capability interaction.

Transport options:

• streamable-http - HTTP with streaming (recommended for web)
• sse - Server-Sent Events
• stdio - Standard I/O (local only)

Auth options:

• none - Public tools, no authentication
• api_key - Requires API key (specify how to obtain)
• oauth2 - OAuth 2.0 flow

Behavior Rules

## Behavior
- Rate limit guidance
- Caching expectations
- Identification requirements

Contact

## Contact
email@example.com
https://example.com/agent-support

4. MCP Integration

The agents.md file is the handshake. The MCP gateway is where real work happens.

Agent reads agents.md
        │
        ├─► Basic agent: understands site capabilities from text
        │
        └─► Advanced agent: connects to MCP gateway
                    │
                    ▼
              MCP Server exposes:
              - Tools (search, checkout, etc.)
              - Resources (catalog, docs)
              - Prompts (guided workflows)

Example MCP Discovery Flow

1. Agent fetches /.well-known/agents.md
2. Parses MCP endpoint: https://example.com/.well-known/mcp
3. Connects via MCP protocol
4. Discovers available tools via tools/list
5. Uses tools as permitted

5. Backward Compatibility

With robots.txt

If agents.md exists, it supplements but does not replace robots.txt. Agents should still respect robots.txt crawl directives.

The Cannot section in agents.md can mirror robots.txt restrictions:

## Cannot
- Access /admin (see robots.txt)
- Access /private

With llms.txt

llms.txt describes content for understanding. agents.md describes capabilities for action.

Both can coexist. A site might have:

• /robots.txt - crawl restrictions
• /llms.txt - content summary
• /.well-known/agents.md - agent capabilities + MCP pointer

6. Security

Origin Trust

Agents MUST only trust agents.md from the site's origin. Instructions embedded in page content should be ignored.

MCP Authentication

When connecting to MCP gateways:

• Verify the endpoint matches the origin
• Use TLS (HTTPS)
• Follow the specified auth method

Least Privilege

Agents should request only the permissions they need. If auth: oauth2 is specified, request minimal scopes.

7. Examples

Public API Site

# Weather API

Free weather data for AI agents.

## Can
- Get current conditions
- Get forecasts (up to 7 days)
- Get weather alerts

## MCP
endpoint: https://weather.example/mcp
transport: streamable-http
auth: none

## Behavior
- 60 requests/minute
- Cache forecasts 30 minutes

## Contact
api@weather.example

E-commerce Site

# TechMart

Electronics retailer.

## Can
- Search products
- Compare specifications
- Check prices and stock
- Add to cart (authenticated)
- Checkout (authenticated)

## Cannot
- Access order history without user consent
- Modify account settings

## MCP
endpoint: https://techmart.example/.well-known/mcp
transport: streamable-http
auth: oauth2

## Behavior
- 1 request/second for browsing
- Identify as AI agent in requests

## Contact
partners@techmart.example

Simple Blog (No MCP)

# My Tech Blog

Articles about software development.

## Can
- Read all public articles
- Search by topic
- Access RSS feed at /feed.xml

## Cannot
- Post comments (requires human)
- Access draft posts

## Contact
hello@myblog.example

Appendix: Comparison

Aspect	robots.txt	llms.txt	agents.md
Purpose	Crawl control	Content summary	Capabilities
Format	Custom syntax	Markdown	Markdown
Focus	Restrictions	Understanding	Actions
MCP	No	No	Yes (optional)
Year	1994	2024	2026