Verified AI Agents with Web Bot Auth
The bot problem changed. Some automation is your customers' AI agents trying to buy from you. Security now means telling declared automation apart from anonymous automation — and treating each accordingly.
"Block all bots" is now a business decision
More than half of web traffic is automated, and a growing share of it is legitimate — AI agents shopping, booking, and researching on behalf of real people. An agent filling your checkout with a customer's card is revenue; a scraper farming your catalogue is not. To a naive bot filter, they look identical: both score low on any honest humanity test, because both are automation.
The distinction that matters is no longer human versus bot. It's automation that declares who it is versus automation that hides. Edge Shield is built around exactly that line.
How Shield identifies agents
The primary mechanism is Web Bot Auth — the
emerging IETF standard (RFC 9421 HTTP Message Signatures with the
web-bot-auth tag)
backed by OpenAI, Google, Amazon, and Cloudflare. The agent signs each request with an
Ed25519 key and points to its published key directory. Shield fetches the directory,
checks the signature, and confirms exactly which agent is calling — identity that
can't be spoofed by copying a user-agent string.
GET /products HTTP/1.1
Host: example.com
Signature-Agent: "https://agent.bot.goog"
Signature-Input: sig1=("@authority" "signature-agent");created=1753100000;\
keyid="poqkLGiy...";alg="ed25519";expires=1753100300;nonce="e8N7...";tag="web-bot-auth"
Signature: sig1=:jdq0SqOwHdyHr9+r5jw3iYZH6aNGKijYp/EstF4RQTQdi5N5YYKrD+mCT1HA1nZDsi6nJKuHxUi/5Syp3rLWBA==:Established crawlers that predate the standard — Googlebot, Bingbot, Applebot — are confirmed the way their operators document: reverse-DNS lookup of the calling IP, forward-confirmed back to the same address. A client merely claiming to be Googlebot from an unrelated IP gets no credit, and the lie counts against its humanity score.
Verified identity is bound into the Shield response token and surfaced in the
siteverify response as
the agent field.
Set a per-widget agent policy
Each Shield widget chooses how verified agents are treated, in its settings:
| Policy | Behaviour |
|---|---|
allow default | Verified agents pass without interactive escalation. The token carries their identity — you decide what they can do. |
challenge | No identity credit. Agents are scored like any anonymous client, and clear automation signals cap their score. |
block | Verified agents are rejected outright with the agent-blocked error code. |
Whatever the policy, anonymous automation is always treated aggressively — clear automation signals with no declared identity hard-cap the humanity score. Declaring identity is the fast path; that's the incentive for agents to sign their traffic.
Route on identity, not just score
The score answers "is this a human?"; the agent field answers "then who is it?". Use both in your server-side check:
const result = await siteverify(token)
if (!result.success) reject()
else if (result.agent?.verified) {
// Declared automation — route it, don't fight it
routeToApi(result.agent.domain)
} else if (result.score >= 40) {
allow() // human traffic
} else {
block() // automation that refused to identify itself
}
Note that agents don't only arrive through forms. When a signed request hits your
API or a content page directly, forward its signature headers to Shield's
agentverify endpoint
and get the same verified identity back — no widget, no JavaScript, no challenge.
See Protecting APIs from Bot Abuse for
the pattern.
One caution: Web Bot Auth adoption is early and uneven — Google signs its AI-browsing agent but not Googlebot indexing, and rollouts are gradual. Treat a valid signature as strong positive identity; treat its absence as neutral, not negative.
The other direction: your agents on Edge
If you also run AI agents — deploying infrastructure, managing DNS — the same principle applies in reverse: give automation scoped, revocable identity rather than full credentials. On Edge that means Agent Access Codes: per-agent credentials with product restrictions, budget caps, and expiry dates, so a leaked agent context never exposes your whole account.