Cloud services giant Fastly has released a report claiming AI crawlers are putting a heavy load on the open web, slurping up sites at a rate that accounts for 80 percent of all AI bot traffic, with the remaining 20 percent used by AI fetchers. Bots and fetchers can hit websites hard, demanding data from a single site in thousands of requests per minute.

I can only see one thing causing this to stop: the AI bubble popping

According to the report [PDF], Facebook owner Meta's AI division accounts for more than half of those crawlers, while OpenAI accounts for the overwhelming majority of on-demand fetch requests.

Cloudflare creates AI crawler tollbooth to pay publishers

"AI bots are reshaping how the internet is accessed and experienced, introducing new complexities for digital platforms,

See Full Page