Cloudflare launches an open registry to authenticate web bots
Cloudflare has unveiled an ambitious proposal that could change how web platforms identify and manage automated traffic.
The American company — a cornerstone of global Internet infrastructure — aims to introduce an open registry format to authenticate web bots and agents, a decentralized approach based on verifiable cryptographic keys, rather than the traditional IP address identification systems.
Why is this change necessary?
Historically, servers identified bots based on their IP addresses, an effective mechanism during the era of fixed networks. However, today, intelligent agents and AI bots — search engines, assistants, commercial crawlers — use shared infrastructures, proxies, or dynamic IP addresses.
The result:
- False positives, where legitimate bots are blocked,
- and false negatives, where malicious bots slip through the cracks.
Cloudflare believes it is time to adopt a more reliable, open, and verifiable system.
How does the new format work?
The proposed open registry by Cloudflare will allow bot creators to publish verifiable public keys, in a standardized manner. These keys would be accessible via simple text files, hosted on public repositories (like GitHub) or distributed via email, based on web standards.
For example: a ChatGPT bot could list its key on https://chatgpt.com/.well-known/agent-registry-keys. A web server could then query this URL to confirm that the traffic indeed originates from the declared bot, without going through a central authority.
In simple terms: instead of relying on an agent’s IP address, we verify its cryptographic identity.
A decentralized and interoperable model
This approach marks a break from the proprietary lists and closed filters used until now. Anyone — from independent developers to large companies — could maintain their own registry, interoperable with others.
The benefits are numerous:
- E-commerce: clearly distinguish legitimate purchase bots from malicious scrapers.
- Research and AI: avoid mistakenly blocking academic crawlers.
- Interoperability: enable platforms to automatically validate external agents.
Cloudflare is already collaborating with Visa and Mastercard to apply this principle to the security of autonomous commercial agents.
An extension of the work on Web Bot Auth
This project builds on Cloudflare’s previous work regarding Web Bot Auth, a cryptographic authentication system for bots launched earlier. The difference: here, the registry aims to standardize and open the process for the entire Internet ecosystem, beyond just Cloudflare services.
Cloudflare presents this innovation as a step toward a universal trust infrastructure, founded not on the reputation of an IP, but on a verified digital identity. This could reduce abusive blocking, streamline interactions between sites and bots, and enhance overall web security.
“The future of automated traffic must rely on cryptography, not presumption,” summarizes a Cloudflare engineer quoted in the report.
Upcoming challenges
Experts praise this initiative, but note several challenges:
- Community adoption: the format must become a recognized standard, not just a Cloudflare tool.
- Governance: who will arbitrate in case of conflicts between keys or abuse?
- Interoperability: convincing major web players (Google, Amazon, Meta, etc.) to adopt the same protocol will be necessary.
Remaining true to its open-source DNA, Cloudflare will publish the format on GitHub (github.com/cloudflare/agents), hoping to rally a community of developers and researchers.
By launching this registry for verifiable agents, Cloudflare is not just implementing a technical update: it is proposing a new paradigm of trust on the Internet, where every bot, every agent, and every AI assistant can transparently and universally prove its identity.
A small cryptographic revolution for a safer, more open, and collaborative web.


