Cloudflare Monetization Gateway: AI Agents Pay Up

Cloudflare Monetization Gateway: AI Agents Pay Up

The numbers behind this story are staggering. Cloudflare, which sits in front of more than a fifth of the web, says daily requests from AI agents on its network have grown by more than 1,700 percent in a year, and its network now handles about 115 million requests a second. For the first time, more than half of all internet traffic is not human. And in some of the most heavily crawled sectors, including retail, software, and financial services, human traffic has fallen by as much as 40 percent in under a year. Every one of those bot visits still costs a site bandwidth and computing power, and most of them bring in no revenue.

That is the broken bargain Cloudflare is trying to fix. For thirty years the web ran on a simple deal. Sites let search engines crawl their pages, the engines sent visitors back, and those visitors saw ads or bought subscriptions. AI answer engines broke the deal by reading the page and summarising it, so the reader never visits. Cloudflare’s answer, announced this month, is to make the machines pay.

The new service is called Monetization Gateway, and it is now in closed beta for US-based sellers and buyers. The mechanics are elegant:

1. The seller sets a price per request. A site can charge for access to a webpage, an API, a dataset, or an MCP tool, with no payment infrastructure to build. A sports stats site could charge a fraction of a cent each time an agent asks who leads the league in assists.
2. The payment rides on HTTP 402. The Gateway uses the Payment Required status code, so price and payment instructions return as part of the HTTP request flow itself. The x402 payment protocol handles settlement, reportedly in USDC stablecoin, without redirecting the agent to a checkout page.
3. Subscriptions stay out of it. The model assumes agents make one-off purchases per task rather than maintaining accounts across hundreds of services, which matches how agents actually work.

Alongside the Gateway, Cloudflare is testing Pay Per Use, which pays publishers when an AI company actually uses their content in answers rather than for each visit. Buyers are verified and report how the content was used, so publishers can see what earned the money. An independent comparison of the eleven schemes claiming to pay websites for AI use noted that, for now, none of the schemes names an actual AI company paying through it. Both Cloudflare products are explicitly being launched as tests.

Cloudflare has also split its single “block AI bots” control into separate switches for search crawling, agent fetching, and model training, so site owners can pick which machines they let in for free and which ones they charge. For context, the share of crawler traffic collecting data to train AI models jumped from 22 percent in spring 2025 to 52 percent by June this year.

Failon’s POV: I have watched publishers swing between blocking every bot on sight and giving everything away for free for two years, and both are losing strategies. Blocking everything costs you citation visibility in the answers, and giving everything away costs you money while the value flows to the answer engines. Cloudflare just built the third option, which is pricing the machine traffic by value. My honest read is that the revenue will be small at first, since none of the payment schemes can yet name a buyer, and the closed beta is US only. But the strategic move is not about this quarter’s payout. It is about getting off the free-crawl default before the industry standard settles without you. If you run on Cloudflare, the bot controls and the beta waitlist are worth your attention this week.

Here is what I would do about it:

1. Segment your bot traffic by intent. Use Cloudflare’s split controls to separate search crawlers, agent fetchers, and training crawlers. Each category deserves its own policy, because search brings you visitors and training data brings you nothing.
2. Join the waitlists, but budget zero revenue from them. Register interest in Monetization Gateway and Pay Per Use, then plan as if they do not exist. Anything they pay out is a bonus until real buyers show up.
3. Audit what agents are actually taking. Check which pages and endpoints draw the most agent traffic, and price the ones with proprietary or expensive-to-produce data first. Public commodity content is not what anyone will pay for.
4. Do not block agent traffic reflexively. If an agent fetches your page to answer a user’s question, that citation is brand visibility in the new discovery layer. Charge the bulk training crawls, keep the citing agents close.
5. Watch for copycat pricing models. The nohacks comparison counts eleven schemes already, from TollBit to the RSL Collective. When a market has eleven toll booths, standards are still forming, and the terms will move. Review your bot policy quarterly, not once.

This is the early infrastructure of the agent economy, and early infrastructure always looks overbuilt. The 30-year free-crawl bargain is not coming back, so the question is not whether sites charge machines but how, and at what price. For publishers and SEO teams, the answer starts with knowing exactly which bots are on your site and why. A technical audit can map that traffic before you decide which parts of it to charge.