The code is right. The sentence is what charges twice.
We read eight agent-payment toolkits published by card processors and billing platforms, for one question: when a payment succeeds but the reply is lost, does a retry charge the customer again? Five of the eight can. In three of those, the idempotency mechanism is implemented correctly — and the one line of documentation handed to the AI model instructs it to defeat that mechanism.
The number
Every one of the five carries an open finding filed with the mechanism, the file and line, and the fix. None of them is named here. A company is never put next to a bug it has not fixed — that rule is enforced by a gate in our tooling, not by our discretion. When a fix ships, the row goes up on the Index with credit and time-to-fix as the headline. The three that were clean are named immediately, because that costs them nothing.
Three ways the same guarantee fails
The failure is always the same shape — an outcome that could not be determined becomes a second payment. What differs is where the break sits.
| Variant | What the code does | Why it fails anyway |
|---|---|---|
| The key is described as "unique" 2 toolkits |
Idempotency key is required from the caller and never generated server-side. Correct, and better than most. | The description says the value must be "unique for every request." A human reads "request" as the purchase. A model reads it as every call I make — so on a retry it mints a fresh key, the platform correctly sees a new payment, and the customer is charged twice. In one case the platform even ships the correct recovery endpoint for an unknown outcome, whose precondition is holding the original key — the same key the description told the caller to change. |
| The key is per-client, not per-call 1 toolkit |
Sets the platform's idempotency header from a configuration field on the client object. | There is no per-call path, and nothing in the toolkit ever assigns the field. Left at its default it is absent, so nothing is protected. Set as the configuration API invites, the same value rides every request for the client's lifetime, so distinct legitimate purchases collide on one key and the second silently returns the first one's result. Two options, neither correct. |
| No key, and an automatic retry 2 toolkits |
Sends no idempotency identity on money-moving POSTs. | One re-sends the request automatically after an auth failure, with the same body and no shared identity between attempts. Another loads create-money tools inside a profile described as non-destructive. In both, a transport failure leaves the caller unable to distinguish "not applied" from "unknown", and the only available action is to send it again. |
The three that were clean
Named immediately and with credit, because a clean read costs them nothing and the ones who get it right deserve to be the visible ones.
Adyen — adyen-mcp
Annotations are honest. Create operations declare idempotent: false, which is exactly right: creating twice makes two things and they say so rather than implying coverage they do not have. No retry wrapper overrides the hints. Read 4 Sep 2026, by code read.
Paddle — paddle-agent-skills
The best work we read. Their webhook skill teaches idempotency to the model explicitly — "handlers may run multiple times for the same event" — with upsert keyed on the platform's own identifier and retry semantics spelled out. This is what writing for a model caller looks like when it is done deliberately. Read 4 Sep 2026, by code read.
Chargebee — agentkit
Exposes no money-moving tools at all: documentation search and code planning. Clean by scope rather than by implementation — there is no create-money write path for a retry to duplicate. Note, added after publication: the repository is archived and read-only, last pushed Nov 2025, so this is a verdict on a project that is no longer maintained. We record that rather than let a clean row imply active support. Read 4 Sep 2026, by code read.
One further toolkit — the largest by adoption — could not be assessed at all: it serves its tool descriptions from a hosted endpoint that requires credentials, and we do not authenticate into anyone's production system to grade it. It is recorded as unassessable, which is not the same as safe and is not counted in either column.
Method, and its limits
What we actually did
Read the money path in each toolkit: where a payment is signed, sent, settled or retried, and what identity travels with it. Then read the parameter descriptions and tool annotations as the model receives them, because for an MCP server that text is the interface.
- Every verdict is a code read, labelled as such.
- Findings were filed with the target before anything was written here.
- No system was tested without permission.
What we did not do, and it matters
We did not run an agent against these toolkits and count duplicate charges. That experiment would turn a mechanism into a rate, and until it is run this is a claim about what the instruction says, not about how often a given model obeys it.
- Model behaviour varies; the instruction pointing the wrong way does not.
- Where a platform deduplicates server-side, a finding reduces to a documentation gap — we asked each one directly and said we would publish the correction.
- Eight is not a random sample. It is the set of toolkits published by payment platforms that we could read.
Get your own read — free
Submit any client, facilitator, SDK or toolkit that moves money — yours or someone else's. It gets read and graded on the public Index at no cost. If a finding is confirmed you are counted, never named, until you ship a fix; when you do, the row goes up with credit and your time-to-fix.
Submit for grading — free See the Index