AI crawling surged 33% in 30 days, and Grok appeared almost overnight

Between 24 August and 22 September 2026, total core AI requests across the fixed client panel increased from 167,120 to 172,878.

Split the same requests into the two behaviours and the picture changes:

For every observable user-driven AI fetch, websites received more than twice as much automated crawling as in the prior window. The median site recorded a 24.5% rise in crawling, so one enormous website is not carrying the average.

AI systems are consuming more of the web while returning less observable user activity to it.

AI Traffic Index #2 found the opposite pattern. Across a fixed panel of 173 websites, citation-driven AI activity rose 65.1%, while crawler activity increased 19.8%.

Thirty-day windows are volatile, platforms change behaviour quickly, and user-triggered fetches are not confirmed browser clicks. The change is still too large, and too widely distributed across the panel, to dismiss as background noise.

Issue 1 is AI crawlers out-hitting Googlebot. This month's warning is that the bargain between crawl volume and returned activity remains unstable.

Within the fixed panel, xAI/Grok activity moved from 17 requests in the prior window to 14,680 crawler requests in the latest 30 days. EZY first detected GrokBot-classified crawler activity on 25 August. Across the wider reporting dataset, Grok-related crawler signatures were detected on more than 190 domains.

That does not prove xAI launched a new crawler on that exact date. It may reflect a rollout, a user-agent change, or the point at which the activity became identifiable. What the logs establish is that a material new crawling footprint appeared extremely quickly, without an equivalent stream of measurable visitors.

Perplexity crawler activity rose from 10,541 to 17,029. User-triggered activity fell from 14,322 to 3,930.

GPT showed a similar, less extreme split: crawling rose 3.5%, while user-triggered activity fell 48.9%, from 48,677 to 24,851.

Claude was the exception. User-triggered activity increased 57.1%, from 10,170 to 15,982, alongside the 23.1% rise in crawling.

Cloudflare CEO Matthew Prince has argued that the old search bargain is breaking: search engines crawled publishers and also returned visitors. AI systems can consume far more content while returning far less traffic. In Cloudflare's analysis, a referral from OpenAI was estimated at roughly 750 times harder than Google's historic crawl-to-referral relationship, and roughly 30,000 times harder for Anthropic.

EZY's panel is a different measurement and is not a reproduction of those ratios. The direction matches. In this fixed panel, the crawler-to-user-fetch ratio doubled in 30 days.

When AI Traffic Index #1 was published, the panel was 125 websites. The cumulative dataset is now 41,206,881 requests from 319 domains that have reported data at some point. 175 websites reported during the latest complete 30-day window. That is growth in the measurement base, not a claim that web traffic doubled. Human requests still outnumbered core AI requests by roughly 19 to 1 in the latest window.

The answer is not to block every AI system, or to allow every crawler blindly. Measure the exchange. For each platform, a site owner should be able to answer four questions: