A card hold for every prompt

In risk and treasury, a payment you cannot size in advance is an old problem. A hotel or a petrol pump does not know the final bill when you hand over your card, so it places a hold first and takes the real amount later. X by Nexow has the same problem each time you describe a widget: the model writes until it is done, and only then does the provider report how many tokens it used. So X does what a card network does.

Authorise

Before a request goes to the model, the server asks the database to hold credits for it: an estimate of the input it is about to send, plus the most output the call may write. The database grants that estimate or what is left on your balance, whichever is smaller. If what is left is below the minimum a call needs to start, the request stops there, with HTTP 402, and the model is never called. If the grant is smaller than the estimate, the call’s output limit is lowered to what the hold can pay for.

Capture

When the call ends, the provider’s usage report sets the real charge, priced per model and per kind of token: input, output, cache reads and cache writes. Settling takes the smaller of that charge and the hold, and gives the rest back. This is the line in the database function that does it:

v_charge := least(coalesce(v_hold.pending_actual, 0), v_hold.amount);   -- clamp to reserved

The clamp is on purpose. If the estimate was short, the difference is ours to absorb, and a comment in the code says so: a low estimate only ever risks the operator eating the overflow, never a user overspending. A call that used nothing releases its whole hold. A stream cut before the usage report arrived is charged a cautious floor, worked out from what was sent and what had already come back, so tokens the provider has served are not lost either.

You are never charged more than was held. If our estimate was short, we pay the difference.

The database says no

Balances live in Postgres, and so do the rules, as constraints rather than application code:

balance  bigint not null default 0 check (balance >= 0),
held     bigint not null default 0 check (held >= 0),
…
constraint held_le_balance check (held <= balance)

A balance cannot fall below zero, and credits on hold cannot exceed it, whatever a bug further up tries to do. Every change goes through a database function that first locks the account’s row, so two builds started at the same moment wait for each other instead of both spending the same credits. Only the server can call those functions, and a user can only read their own rows.

Credits come in lots, each with its source and, for some, an expiry. A charge burns them in a fixed order: by priority, so subscription credits go before purchased ones, then the soonest to expire, then the oldest. Every movement is written to a ledger, with the balance after it.

One event, one grant

Credits are bought through Stripe, and Stripe may deliver the same event more than once. Each event’s id goes into a table where it is the primary key, before anything is granted, so a repeat is acknowledged and ignored. If handling fails, the row is removed and Stripe is told to try again. The credits granted carry a unique key of their own, such as the invoice id, so even a duplicate that slipped past the first check could not grant twice.

Holds do not outlive their call. A sweep runs every five minutes: a hold past its five-minute life is settled if a charge was recorded on it, and released if not. A generation times out after two minutes, so the sweep never touches a call that is still running. And if you bring your own model key, none of this applies: your calls never spend credits.