> ## Documentation Index > Fetch the complete documentation index at: https://wtd2026.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Methodology > How the traffic data was collected and classified. What the thresholds mean, why the range exists, and when to use each time window. ## Discovering: classifying AI versus human traffic ### Data source Mintlify's internal analytics dashboard, filtered down to only include sites with more than 100 views per month for 17,784 total sites. Two time windows: a seven day aggregate and a 24-hour snapshot, both from the week of April 2026. ### Three signals, in order of confidence If a request targeted `/llms.txt`, `/llms-full.txt`, `/*.md` (Markdown variant of a page), or was an MCP tool call, it's high confidence that it's a machine. These endpoints exist only to serve AI. This signal is the source of the 2.26M machine reads (24 hours) / 14.4M (7 days). Every HTTP request includes a user agent string. ClaudeBot says ClaudeBot. GPTBot says GPTBot. Mintlify matches against \~60 known AI agent patterns to produce the named agent breakdown. Limitation: not every AI client identifies itself accurately. Self-identification is the floor, not the ceiling. If a user agent plus IP pair looks like a normal browser generates more than 500 events in a single day across many sites, it's probably an agent. We can't know with full certainty, but it's not typical human behavior. We flag these events as "inconclusive" and show a range to reflect the uncertainty. ### Why there are three AI share percentages | Percentage | How it's calculated | What it means | | ------------------- | ------------------------------------ | ------------------------------------------------------------------ | | 28.7% (raw) | AI ÷ (AI + all human views) | Suspicious traffic stays in the denominator — understates AI share | | 35.1% (verified) | AI ÷ (AI + clean human views) | Suspicious pairs removed from both sides — this is the floor | | 47.1% (upper bound) | (AI + suspicious) ÷ (AI + all human) | All suspicious traffic attributed to AI — this is the ceiling | ### Suspicious traffic thresholds "Suspicious" is defined as a user agent plus IP pair exceeding a threshold within a time bucket: | Time window | Threshold | | ----------- | --------------------- | | Hourly | 10 events per hour | | Daily | 200 events per day | | Weekly | 1,400 events per week | Detection is per-bucket: a pair is only flagged in a bucket where it crossed the threshold. A single spike doesn't contaminate the entire window. ### Two time windows The data comes from two different views with fixed windows. | Use case | Window to use | | ------------------------------ | ------------- | | AI share percentages | 7 days | | Trend claims, headline figures | 7 days | | Agent breakdown | 24-hour | | Scraper pair counts | 24-hour | | Cloud provider table | 24-hour | ## Read and Write: Helicone inference logs ### Data source Helicone API, filtered to Mintlify's production inference products. Date: April 21, 2026 for a single representative weekday (Tuesday). ### Products included | Product | Category | | --------------- | -------- | | Assistant chats | Read | | Workflows | Write | | Atlas | Write | | Agent sessions | Write | ### Deduplication The raw dataset contains one row per LLM API request. Multi-step agentic sessions produce multiple rows sharing the same session ID. For session-level analysis (session counts, sentiment arcs, conversation length distributions), records are deduplicated to one row per session, keeping the row with the longest non-empty assistant response as canonical. Structural counts (total requests, requests by product) use the full non-deduplicated dataset. ### Anonymization All text analysis operates on anonymized data: Customer names, product names, model names, and subdomain tokens are stripped from message text before NLP analysis. No site or customer is identifiable in any reported figure. ### What's reliable and what's directional | Analysis type | Reliability | Source | | ------------------------------------------- | ----------- | -------------------------------------- | | Request counts, session counts, site counts | High | Full dataset | | Latency distributions | High | Full dataset | | Conversation length distributions | High | Full dataset | | Message length distributions | High | Full dataset | | Success rates | High | Full dataset | | Intent classification | High | Full dataset | | Sentiment analysis | Directional | \~3,300 sampled records, keyword-based | | Topic clustering | Directional | 10K record NLP samples | ### Turn counting Turn count is the number of user messages in the conversation history at the time of the request. The assistant's turn count was inferred from the messages array. This accurately reflects conversation depth but can't distinguish a fresh multi-turn session from a continued one if session IDs weren't tracked. ## Tool chain | Step | Tool | | ---------------------------------------- | ------------------------------------------------------------------- | | Data export | Python | | Preprocessing | Python — flattens JSONL, extracts messages, computes derived fields | | Structural analysis | DuckDB — counts, distributions, latency | | Sample export | DuckDB — random samples for NLP analysis | | Visualization | matplotlib | | NLP analysis (sentiment, intent, topics) | Claude Desktop and Claude Code, on sampled subsets | The NLP figures in the talk (sentiment, topic clustering, intent classification) were produced by loading sampled JSONL exports into Claude Desktop and Claude Code. This is why they're labeled directional. The sample sizes are reasonable but the method is not reproducible in the same way as a deterministic classifier.