> ## Documentation Index
> Fetch the complete documentation index at: https://wtd2026.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Discovering: AI agents reading your docs

> Half of all content reads on Mintlify documentation sites come from AI agents.

**How AI agents discover and consume your documentation.**

Agents can read documentation directly on behalf of users. Agent traffic now rivals human traffic and is on pace to become the majority of all traffic to documentation sites.

The discovering layer is about the machines that consume your content based on humans' prompts and queries.

Data comes from across 17,788 active documentation sites.

## Seven day traffic

| Measure                                                                                          | Value      |
| ------------------------------------------------------------------------------------------------ | ---------- |
| AI machine reads                                                                                 | 14,434,895 |
| Verified human views                                                                             | 26,659,653 |
| Inconclusive traffic                                                                             | 9,278,858  |
| **Raw AI share** (AI / (AI + human views + inconclusive traffic))                                | **28.7%**  |
| **Verified AI share** (AI / (AI + human views))                                                  | **35.1%**  |
| **Upper bound AI share** (AI + inconclusive traffic) / (AI + human views + inconclusive traffic) | **47.1%**  |

## Twenty four hour traffic

| Traffic type             | Events         |
| ------------------------ | -------------- |
| Human page views         | 5,592,165      |
| AI machine reads         | 2,257,565      |
| Suspected scraper events | 3,896,380      |
| Other interactions       | 1,109,339      |
| **Total**                | **12,855,449** |

<Tip>
  46% of all active documentation sites were touched by suspected scrapers in a single day.
</Tip>

## The scraper fingerprint

Scrapers are identified by behavioral patterns. High event volume from a user agent and IP pair that otherwise looks like a normal browser:

| Metric                | Value |
| --------------------- | ----- |
| UA + IP pairs flagged | 6,810 |
| Distinct user agents  | 219   |
| Share of all traffic  | 41%   |

## Who's reading: named AI agents

These are AI clients that explicitly identify themselves in their HTTP user agent string. Named agents are a small fraction of total AI reads, the majority of which arrive via unidentified scrapers.

| Agent                  | Events (24h) |
| ---------------------- | ------------ |
| **Claude**             | **591,349**  |
| Bytedance              | 38,348       |
| ChatGPT                | 26,780       |
| Claude Code            | 11,008       |
| Other AI               | 1,026        |
| Gemini                 | 604          |
| Firecrawl              | 322          |
| Cursor                 | 252          |
| Perplexity             | 247          |
| Common Crawl           | 32           |
| Copilot                | 26           |
| **Total named agents** | **669,996**  |

Claude leads by a factor of 15 times over the next closest named agent (Bytedance at 38K).

<Tip>
  "Claude" here is a group of distinct user agents: ClaudeBot (Anthropic's crawler), Claude-User (real-time responses to user queries), and Claude Code (the coding agent fetching docs programmatically). They all show up in the user agent string and we group them.
</Tip>

## Where the traffic comes from: cloud providers

| Provider     | Events (24h) | Share | Unique IPs |
| ------------ | ------------ | ----- | ---------- |
| Cloudflare   | 9,458,107    | 73.6% | 47,318     |
| Other        | 2,730,788    | 21.2% | 92,542     |
| AWS          | 609,282      | 4.7%  | 3,372      |
| DigitalOcean | 24,405       | 0.2%  | 198        |
| GCP          | 21,324       | 0.2%  | 532        |
| Azure        | 9,102        | 0.1%  | 1,818      |

<Note>
  Most traffic arrives via Cloudflare edge nodes, so the recorded IP is usually a CDN proxy—not the originating client. "Other" has the highest IP diversity (92K unique IPs), largely residential. The cloud provider table reflects where requests exit the network, not where they originate.
</Note>

## AI-native endpoints

These are the endpoints that are classified as high certainty AI reads:

| Endpoint         | What it is                                                                            |
| ---------------- | ------------------------------------------------------------------------------------- |
| `/llms.txt`      | Lightweight manifest of site structure and key pages, designed for AI context windows |
| `/llms-full.txt` | Full text of all documentation content                                                |
| `/*.md`          | Markdown version of any page                                                          |
| `mcp.*` events   | An agent calling an MCP tool directly                                                 |

These endpoints were purpose-built for machine consumption.

## Benchmark

**AI readership: 35%**

This is a conservative estimate of agent readership based on the higher confidence signals of agent views.

## What this means for the craft

You are no longer writing for just humans.

* **Structure matters as much as ever.** Agents need well structured content to parse and process. People do too.
* **`llms.txt` is the entry point for agent discovery.** Agents need to know where to start.
* **Self-contained sections matter.** An agent might retrieve just one heading's worth of content. Write introductions and section openers that stand alone.