# Design a Payment System: Ledger, Idempotency, Reconciliation

> How to design a payment system that never loses or double-charges money: a payment state machine, an append-only double-entry ledger in integer cents, idempotency keys, webhooks and daily reconciliation.

URL: https://www.matthewswong.com/en/blog/design-payment-system-ledger-reconciliation/

- Author: Matthews Wong (https://www.matthewswong.com/en/about/)
- Published: 2026-10-10
- Updated: 2026-10-10
- Category: Backend

> **Key takeaway:** A payment system that never loses or double-charges money combines four things: a payment state machine with an explicit unknown state, an append-only double-entry ledger in integer minor units, idempotency keys at the API boundary, and a reconciliation job comparing your ledger with the processor's settlement reports. Refunds are new entries, never edits.

The scariest payment bug is not a crash. It is a request that times out after the processor already took the money, because your code cannot tell whether to retry or give up, and either guess can cost a customer or the business real cash.

This post is the whole-system design: the pieces and the order they have to be built in. I work on ERP and POS systems on Postgres and NestJS, so the examples use Postgres and TypeScript. Every number below is derived in front of you or cited from documentation, and nothing here is a benchmark.

## What does it mean to never lose or double-charge money?

Two failures define the problem, and they pull in opposite directions. A lost payment happens when the customer was charged and your system has no record. A double charge happens when one intent is executed twice. Retrying harder fixes the first and causes the second, so the design has to give you three guarantees at once.

- Every attempt to move money is recorded before it is sent, so nothing can be charged without a trace.
- Every external call and every incoming event is safe to repeat, so retries never multiply a charge.
- Your books are checked against the processor's books on a schedule, so any disagreement is found by you and not by the customer.

Notice what is absent: a promise that nothing ever fails. Networks time out, processors send events twice and sometimes never. The goal is a system where every failure ends in a known state or a ticket, never a silent mismatch.

## How should a payment state machine work?

Model each payment as a state machine with a fixed table of allowed transitions, and write that table as data. The one unusual state is unknown: the request was sent and the outcome is not. Only reconciliation or a confirmed webhook may leave it. The transition below is a compare-and-set update plus an audit row in the same transaction.

The compare-and-set is what stops two workers, or a webhook and a retry, from both moving the same payment. The loser matches zero rows and treats it as a no-op. Failed is terminal on purpose: a customer retrying is a new payment with a new idempotency key, never a resurrected old one.

> Keep the state machine and the ledger in the same database transaction. If the state says captured, the capture entries exist, and if the transaction rolls back, neither does. Two systems that must agree will eventually disagree.

## Why use an append-only double-entry ledger with integer cents?

Store money as integers in the currency's minor unit, never as floats. Binary floating point cannot represent most decimal fractions, and the error shows up exactly when you multiply or sum. Minor units differ by currency, and zero-decimal currencies exist, so store the currency next to every amount and let rounding rules be explicit code.

On top of integers, use double-entry bookkeeping: every movement is a set of debit and credit entries that sum to zero, so money is never created or destroyed, only moved between accounts. Entries are only ever inserted. The schema below enforces both rules in the database instead of trusting application code.

A CHECK constraint sees a single row, so it cannot express that a transaction balances. A deferred constraint trigger runs at commit, after all the entries of the transaction exist, and raises if any currency does not sum to zero. The unique pair of kind and external reference means the same processor event cannot be posted twice, even if your code tries.

> Revoke UPDATE, DELETE and TRUNCATE from the application role as well as adding the trigger. A trigger protects you from bugs, and permissions protect you from the migration someone runs at 2 am. Balances are always derived by summing entries, never stored and edited.

## How do idempotency keys prevent double charges at the API boundary?

The client generates a unique key per intent and sends it with the request. The server inserts the key into a table whose primary key is the guarantee, before doing any work. Stripe documents the same idea: keys up to 255 characters, and for its v1 API they may be removed automatically once they are at least 24 hours old, so choose your own retention deliberately. I cover the mechanics in a separate post, so here is only the shape that matters for payments.

Store a hash of the request with the key. Replaying the same key with a different body is a client bug, and silently returning the first result would hide it. Return a conflict instead. Also keep the key row and the payment row in one transaction, so a crash cannot leave a key that points at nothing.

## What do you do when a payment call times out?

A timeout is not a failure, it is an unknown. What you saw does not tell you what the processor did, so the table below maps each observation to the safe action. The rule underneath is that you may resend the same idempotency key as often as you like, but you must never create a new key to retry an unknown outcome.

What you observed

What may be true at the processor

Safe action

Ledger effect

Connection refused or DNS failure before sending

The request never arrived

Retry with the same idempotency key

None

Read timeout after the request was sent

Captured, declined or still processing

Mark the payment unknown, retry the same key or look it up by your reference

None until the outcome is confirmed

HTTP 5xx from the processor

Ambiguous, same as a timeout

Treat it exactly like a read timeout

None until the outcome is confirmed

A clear decline response

Definitively refused

Move to failed, ask the customer to try again as a new payment

Nothing was posted

Unknown payments are resolved by one of two things arriving later: a webhook, or the reconciliation job asking the processor directly. Until then the customer sees pending, never failed and never paid. Showing failed for an unknown payment is how people get charged twice by paying again.

## How do webhooks and reconciliation catch what the API missed?

Webhooks are how the processor tells you what happened after your call returned. They are delivered at least once and can arrive out of order or not at all, so the handler stores the event id under a unique index and ignores repeats. Stripe's own guidance is to log processed event ids and skip already-logged events. Acknowledge quickly and do the work from the stored row.

Webhooks can be lost, so a webhook-only design has a hole. The backstop is a daily reconciliation job that imports the processor's settlement report and compares it with your ledger. A full outer join shows all three failure shapes in one result: in our ledger but not at the processor, at the processor but not in our ledger, and amounts that differ.

Reconciliation never edits the ledger to make numbers match. Each row becomes a ticket with a fix that is itself a new entry or a state transition, for example resolving an unknown payment to captured and posting its entries. The job is also your monitoring: the count of unmatched rows per day is the one number that tells you the system is healthy.

## How do refunds and corrections work in an append-only ledger?

A refund is a new transaction that moves money back, not an edit of the original capture. The worked example below traces USD 19.99 through capture, payout and a partial refund in cents. Notice that the payout nets an 88 cent fee from the settlement file, so 1999 minus 88 is 1911 reaching the bank, and that every transaction balances by construction.

Guard the refund by the arithmetic: refunds already posted plus the new amount must not exceed the captured amount. After the 500 refund, 1499 is still refundable, so a second request for 1600 fails because 500 plus 1600 is 2100, more than 1999. A wrong entry is corrected by a reversal that points at the original through its reverses column, followed by the right entry, which keeps both visible forever.

## What belongs in the audit trail and the pre-launch checklist?

The audit trail is what you already built: the payment events table records every transition with its source and reference, and the ledger records every movement. Together they let you answer who moved this money, when and why, without reading application logs. Before launch, check the design against this list.

1. Money is stored as integer minor units with a currency on every row, and no float touches an amount.
2. Ledger entries are append-only, enforced by a trigger and by revoked permissions, and every transaction is checked to balance at commit.
3. Every payment request starts with an idempotency key insert, and the key and payment rows commit together.
4. The payment state machine has an explicit unknown state, and the customer sees pending while a payment is in it.
5. Webhook handlers dedupe on the event id and the ledger dedupes on the processor reference.
6. A daily reconciliation job compares the ledger with the settlement report, and unmatched rows page someone.

For a small system on a single Postgres instance, all of this fits in a handful of tables and two background jobs. The cost is discipline, not infrastructure, and it is far cheaper than explaining to a customer why they were charged twice.

Treat unknown as a first-class state, make every write repeatable, record money as balanced append-only entries, and let a reconciliation job prove the books against the processor every day. Correctness here is a set of constraints in the database, not a property of careful code.

## Sources and further reading

- Stripe API docs: idempotent requests
- Stripe docs: receiving webhooks and handling duplicate events
- Stripe docs: supported currencies, including zero-decimal currencies
- PostgreSQL docs: CREATE TRIGGER and deferred constraint triggers
- Wikipedia: double-entry bookkeeping

## Source links

- https://docs.stripe.com/api/idempotent_requests
- https://docs.stripe.com/webhooks
- https://docs.stripe.com/currencies
- https://www.postgresql.org/docs/current/sql-createtrigger.html
- https://en.wikipedia.org/wiki/Double-entry_bookkeeping

## FAQ

### How do you design a payment system that never double-charges?

Require an idempotency key on every payment request and insert it into a table whose primary key is the key, before doing any work. A replay then returns the stored result instead of charging again. Back this with a payment state machine that has compare-and-set transitions, and a ledger that rejects a second posting of the same processor reference.

### Why store money as integers instead of floats?

Binary floating point cannot represent most decimal fractions exactly, so 0.1 plus 0.2 gives 0.30000000000000004 in JavaScript. Storing integers in the currency's minor unit, such as 1999 cents for USD 19.99, keeps sums exact. Store the currency beside every amount because minor units differ and some currencies have no decimals.

### What should you do when a payment API call times out?

Treat it as unknown, not failed. Mark the payment unknown, keep the customer on a pending screen, and retry with the same idempotency key or look the payment up by your own reference. Never create a new key for the retry, because that is how a customer ends up charged twice.

### What is payment reconciliation and why is it needed?

Reconciliation compares your ledger with the processor's settlement report, usually daily, and lists anything that exists on one side only or differs in amount. It is needed because webhooks can be lost or delayed and API calls can end in unknown outcomes. Each mismatch becomes a ticket fixed with a new entry, never by editing history.

### How do you handle refunds in a double-entry ledger?

Post a new balanced transaction that moves the money back, for example debit revenue and credit the processor clearing account, instead of editing the original capture. Check that refunds already posted plus the new amount do not exceed the captured amount. A wrong entry is corrected by a reversal that references the original, so both stay visible for audit.

---

- HTML version: https://www.matthewswong.com/en/blog/design-payment-system-ledger-reconciliation/
- Bahasa Indonesia: https://www.matthewswong.com/id/blog/design-payment-system-ledger-reconciliation/
- Site index for agents: https://www.matthewswong.com/llms.txt
- Sitemap: https://www.matthewswong.com/sitemap.xml
- API description (OpenAPI): https://www.matthewswong.com/openapi.json
