Publishers are negotiating with AI search the wrong way

Tipsheet's lead item exposes a blind spot in how publishers treat AI crawlers. An Axios analysis of 102 top US publisher domains found 42% block AI training crawlers outright, but only 25% block AI search crawlers, with another 39% running mixed rules that let some bots in. In other words, publishers are far more willing to let AI companies use their content for discovery than for training, a split John Ebbert reads as commercially sensible but strategically sloppy. His point: robots.txt is a blunt instrument that communicates crawling preferences, not the terms of discovery, attribution or compensation. Separately, The Verge is reorganising its audience team around newsletters, community and registered users rather than SEO and pageviews, a move Tipsheet flags for ignoring the AI-search layer almost entirely. "I'm still waiting for publishers to quantify the business value of AI search beyond referral traffic," Ebbert writes.

The crux

Publishers have a clearer stance on AI training than on AI search, and that asymmetry means they are ceding the discovery layer, where future advertising impressions and subscribers will be won, without setting terms.

Sell-side read

A publisher that blocks training but opens search is signalling it will supply the AI discovery economy without a pricing model, which is the same mistake the industry made with Google a decade ago in reverse. The publishers that formalise terms, compensation and attribution for AI search now will be the ones paid for discovery later.

Watch

Whether any major publisher publishes explicit AI-search terms or a pay-per-value arrangement, and how The Verge's post-SEO team performs.