The transatlantic push to make stealth bots identify themselves

Publishers are moving the fight against AI scraping out of the server room and into parliaments on both sides of the Atlantic. Digiday's Media Briefing reports that scraping accounted for 70.9% of malicious bot traffic analysed by DataDome in its September report, up 185.2% over the study's 12-month window, and that crawlers now routinely impersonate legitimate bots. Centinel found that on a typical site one in seven requests claiming to be Googlebot failed verification, 84% of requests claiming to be GPTBot were fake, and 99% claiming to be Perplexity-User were fake. The US News/Media Alliance marshalled more than 300 publishing executives to lobby Congress in September for the federal Stealth Bot Prohibition Act, and on 15 October a UK coalition including The Times, the Financial Times and Condé Nast urged the UK government to back an equivalent bill. "They come in with a named crawler; we block it. They pretend to be human; we block it... then they come through people's home proxies. And this is where governments start getting involved," People Inc. chief innovation officer Jon Roberts told Digiday's Media Strategies event. The scale is staggering: Reach CEO Piers North said the group has seen over 100 million successful scrapes in six months, and the Financial Times' Fiona Spooner described one account trying to access about a million articles in eight hours from roughly 700,000 IP addresses.

The crux

Publishers have exhausted technical bot-blocking as a defence, so the fight has become a legislative one for disclosure: forcing crawlers to identify themselves and their purpose is the precondition for any leverage over who gets to train on publisher content, and over the AI licensing deals that depend on knowing who is taking what.

Sell-side read

Invalid and scraper traffic now masquerades as legitimate demand, which corrupts the viewability and audience metrics your CPMs are priced on; Reuters' Phil Andraos conceded on stage that block-by-default is "not always 100% foolproof", so treat bots as a yield-leakage problem, not just a legal one.

Watch

Whether the UK bill and the US Stealth Bot Prohibition Act clear committee before year-end, and whether disclosure requirements give publishers a legal hook for licensing negotiations.