Publishers weigh agentic trading and AI crawler access at Digiday's London town hall

At Digiday's Media Strategies London town hall on Tuesday, publisher executives laid out a three-way bind that has no proven playbook yet: how to measure audience value when a growing share of "readers" are machines, whether to block AI crawlers and trade visibility for scarcity, and whether agentic media trading can win back the programmatic value lost to DSPs and SSPs. The event ran under the Chatham House Rule, so the quotes are unattributed, but the direction of travel is clear. On measurement, one exec framed the shift bluntly: "Should we instead focus on visitors? You have human visitors and non-human visitors: agents, LLMs, bots, and so on." On crawlers, another said the question is whether there is a real market for licensed content yet: "restricting access to information could also create scarcity and help establish a market for that content. The question is how big that market is today." On agentic trading, one publisher said it is "starting building a selling agent" and sees a chance to "win back some of the value publishers have lost to the middlemen between them and advertisers, particularly DSPs and SSPs." The concrete data point came from Fiona Spooner, managing director of consumer revenue at the Financial Times, who said one trial account tried to pull "about a million different articles within about eight hours from 700,000 different IP addresses." Separately, the Standards for Publisher Usage Rights coalition, SPUR, added The New York Times, The Atlantic, The Economist and Agence France-Presse this week, taking it past 80 publishers across 20 countries. The NYT joining matters because its long-running copyright suit against OpenAI is, in the words of one town hall participant, what "could help establish the value that needs to be exchanged for this content."

The crux

Publishers are trying to build a market, and a measurement standard, for AI's use of their content before the economics get set for them, but the leverage is notional: chatbot referral traffic is still near zero, licensing talks have slowed, and the only hard counter is the New York Times's lawsuit against OpenAI.

Sell-side read

The scraping numbers are the story to internalise: a single test account pulling a million articles in eight hours is a publisher's content being monetised by someone else with no reciprocal value. Any publisher still treating AI crawlers as a neutral traffic question is pricing a market that is already being taken.

Watch

Whether SPUR's telemetry standard, which tracks content retrieval through citation, gains adoption from the AI firms themselves, and what the NYT-OpenAI ruling sets as the de facto price for licensed content.