# Idempotency Explained: Safe Retries and Idempotency Keys

> Idempotency explained: safe vs idempotent HTTP methods, why retries need it, how idempotency keys work, idempotent consumers, upserts and re-runnable jobs.

URL: https://www.matthewswong.com/en/learn/system-design/idempotency-explained/

System design, step by step · Lesson 10 of 11 · Reviewed on 18 October 2026 · By Matthews Wong (https://www.matthewswong.com/en/about/)

> Idempotency means doing an operation twice has the same effect as doing it once. It matters because networks fail mid-request: after a timeout the client cannot tell whether the work happened, so it retries. GET, PUT and DELETE are idempotent by definition; POST is not, so APIs add an idempotency key that lets the server replay its first answer instead of repeating the work.

## You will be able to

- Tell safe methods from idempotent ones, and say which requests a client may retry on its own after a timeout.
- Design an idempotency key flow: who creates the key, what the server stores, for how long, and what a duplicate in flight gets.
- Make consumers and scripts safe to re-run with upserts, a processed-messages table and writes that set a value instead of adding to it.

## A lost response, a retry, and an idempotency key

A payment request succeeds but its response is lost; retried without a key it charges twice, while with an idempotency key the server stores its first answer and replays it, answers 409 to a duplicate still in flight, and rejects the key when it comes back with a different body.

1. **Meet the parts.** A checkout app sends POST /charges to a payments API, which writes each charge to a table. The reply travels back over the network. The key store is unused for now.
2. **The reply is lost.** The API charges the card and answers 201, but the network drops the reply. The client only sees a timeout, so it cannot tell whether the charge happened.

   *Predict:* The client sends the same POST again, with nothing added. What does the API do? (Returns the first answer / Charges the card again / Rejects it as a duplicate)
3. **The retry charges twice.** Without a key, the API cannot tell a retry from a new order, so it charges again. POST is not idempotent: two identical sends, two effects.
4. **Send a key, keep the result.** Fix: the client creates one idempotency key before the first attempt. The API claims the key, charges, then saves the result under it. This reply is lost too.

   ```
   Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324"
   ```
5. **The retry gets the stored answer.** The retry carries the same key. The API finds k1 and returns the stored 201 without charging again. Same request, same effect: the endpoint is now idempotent.

   ```
   Idempotent-Replayed: true
   ```

   *Predict:* A new order sends key k2, then a duplicate with k2 lands while the first is still running. What should the duplicate get? (A second charge / The stored answer for k1 / A conflict: try again later)
6. **A duplicate in flight.** The key is claimed as in progress before the work starts. A duplicate that lands mid-flight gets 409 Conflict and retries later, instead of racing the first request.

   ```
   HTTP/1.1 409 Conflict
   ```
7. **Keys have rules.** Reusing k1 with a different body is a client bug: the IETF draft answers 422, Resend 409. Keys also expire: Stripe keeps them at least 24 hours.

   ```
   HTTP/1.1 422 Unprocessable Content
   ```

## What is idempotency?

An operation is idempotent when doing it twice has the same effect as doing it once. Setting a light switch to off is idempotent; flipping it is not. RFC 9110, the HTTP specification, defines it for requests: a method is idempotent if the intended effect on the server of several identical requests is the same as the effect of a single one.

Two words in that definition carry the weight. Effect, not response: RFC 9110 notes that repeating an idempotent request has the same intended effect even if the first one succeeded, though the response might differ. A second DELETE of the same item finds nothing to delete, yet the item is gone either way. Intended: the server is still free to log every request separately or keep a revision history. Idempotency is about what the client asked for.

## Safe vs idempotent: which HTTP methods are which?

A safe method is read-only by definition: the client neither requests nor expects a state change on the server. RFC 9110 defines GET, HEAD, OPTIONS and TRACE as safe. Every safe method is also idempotent, and so are PUT and DELETE, which change state but land on the same state when repeated. POST is neither. RFC 5789 says PATCH is neither safe nor idempotent either, while noting that a PATCH can be written so that it is.

The difference between PUT and POST is intent. A PUT carries the complete new state of a resource the client names, so sending it twice leaves that same state. A POST hands its data to the resource to process by its own rules, such as creating a new order, so each send can create another. RFC 9110 draws the line itself: a service that picks the new resource's URI on the client's behalf should use POST rather than PUT.

## Why do retries make idempotency a correctness problem?

A timeout is ambiguous. The request may never have arrived, the server may have stopped halfway, or the work may have finished and only the reply was lost. Stripe's error guide says it plainly: restoring connectivity does not tell you whether the original request succeeded. That leaves the client two bad choices, give up and maybe lose the order, or retry and maybe do it twice.

RFC 9110 sets the rules that follow. A client can repeat an idempotent request automatically after a connection failure. It should not automatically retry a non-idempotent one unless it knows the request is idempotent anyway, or that the original was never applied, and a proxy must never retry a non-idempotent request. The reliability lesson (lesson 5) covered when and how often to retry; this one makes the retry safe, so that at-least-once attempts produce the effect exactly once.

## How does an idempotency key work, and what does the server store?

The client generates a unique key for one operation, before the first attempt, and sends it on every retry in an Idempotency-Key header. A random UUID is the usual choice: the IETF draft recommends a UUID or a similar random identifier, Stripe suggests V4 UUIDs, and crypto.randomUUID() makes one in browsers and in Node. Make it per operation, never per attempt, because a new key looks like a new request. Stripe also suggests deriving the key from something the user already holds, such as a shopping cart's ID, which guards against a double submission.

The server keeps a small record per key: a fingerprint of the request body, whether the work is in progress or done, and the status code and body it answered. On the first request it claims the key, does the work and saves the result. On a retry it returns the saved result without doing the work again, and Stripe marks such a reply with Idempotent-Replayed: true. Stripe saves a result only once the endpoint has started executing, so a request rejected by validation can simply be retried, while a saved 500 is replayed as a 500.

Keys do not live forever, and the server should publish how long it keeps them. Stripe keeps them for at least 24 hours and may remove them after that, so a key reused later can run as a new request; Resend keeps them for 24 hours. Two edge cases need an answer. A duplicate that arrives while the first is still running gets 409 Conflict in the IETF draft, and Stripe's guide says to wait and retry it with the same key. A key reused with a different body gets 422 in the draft; Resend answers both cases with 409 and different error names.

The header name comes from an IETF HTTPAPI working group draft. On 2026-10-18 the IETF datatracker lists it as an expired Internet-Draft, last revised as version 07 on 2025-10-15, so it is not a standard. Stripe and Resend use the same header name anyway, which makes it a safe default for your own API.

## What are at-least-once delivery and idempotent consumers?

Message queues meet the same problem from the other side. RabbitMQ's reliability guide says acknowledgements guarantee at-least-once delivery, and without them only at-most-once is guaranteed. A consumer should acknowledge a message only after it has done the work. If its channel or connection closes first, every unacknowledged delivery is requeued and comes back with the redelivered flag set. A publisher whose confirm was lost may send a message twice as well.

So duplicates are normal, and RabbitMQ recommends designing consumers to be idempotent rather than explicitly deduplicating. The redelivered flag is only a hint: when it is false the message is new, when it is true the consumer may have seen it. The common recipe is the Idempotent Consumer pattern on microservices.io: inside the same database transaction as the business change, record the ID of every processed message. A redelivery finds its ID already recorded, and the consumer skips the work and acknowledges.

## What is natural idempotency, and where do you already use it?

The cheapest idempotency needs no key at all: write operations that set a state instead of changing it by an amount. Set the stock to 11 can run twice; subtract one cannot. PUT replaces a resource, so it repeats safely. In SQL, an upsert does the same for inserts: INSERT with ON CONFLICT DO NOTHING, in PostgreSQL and in SQLite since 3.24.0, skips a row whose unique key already exists, and ON CONFLICT DO UPDATE turns the insert into an update. PostgreSQL guarantees an atomic insert-or-update outcome for DO UPDATE, even under high concurrency.

Declarative tools are idempotent by design, because they compare the state you declare with the state that exists. docker compose up recreates a service's container only when its configuration or image changed after the container was created, so a second up -d with nothing changed leaves the running container alone. Shell has the same split: chmod 600 sets the mode absolutely, mkdir -p does not fail when the directory exists, and > replaces a file's contents while >> appends, so re-running a script that uses >> doubles every line. This repo's scripts/setup-resend-templates.mjs says in its header that re-running it updates the templates: it finds them by a fixed alias instead of creating new ones.

## The real config behind this lesson

`app/api/contact/route.ts:148-154`

```typescript
  try {
    // Keyed by the payload, so a retry of the same message (a visitor
    // resubmitting after a lost reply) replays the first send instead of
    // delivering it twice. See lib/contact-idempotency.ts.
    const { error } = await getResend(apiKey).emails.send(payload, {
      idempotencyKey: contactIdempotencyKey(payload),
    });
```

Until 2026-10-18 this was the non-idempotent POST this lesson warns about. The send passed Resend no key, the SDK turns a lost connection into the same error value as a rejection, and the form asks the visitor to try again, so a lost reply meant a second email: against a local stand-in for Resend that drops its first reply, two submissions made two deliveries. Now the call carries a key, and the same test makes one delivery and one replay.

`lib/contact-idempotency.ts:27-32`

```typescript
export function contactIdempotencyKey(payload: object): string {
  const digest = createHash("sha256")
    .update(JSON.stringify(payload))
    .digest("hex");
  return `${CONTACT_KEY_PREFIX}${digest}`;
}
```

The key is the SHA-256 of the exact payload: sender, recipient, reply-to, template and every variable. An identical retry serialises to the same string, so it carries the same key with nothing for the form to remember, even across a page reload. An edited message hashes differently and is a new email. And a key can never meet a different payload, so Resend's 409 for that case cannot happen.

`app/[locale]/(pages)/contact/page.tsx:279-286`

```typescript
      setShowSuccessModal(true);
      resetWizard();
    } catch (error) {
      console.error(error);
      setShowErrorModal(true);
    } finally {
      setIsSubmitting(false);
    }
```

This is where the retry comes from. Any failure, a lost connection included, lands in catch and opens a dialog that says the message didn't send and asks the visitor to try again. Only success calls resetWizard, so the typed message is still there and one more press of Send posts the same message again. The route derives the same key from it, so that retry replays the first send. isSubmitting disables the button while a request is in flight, which stops a double click.

`.github/workflows/deploy.yml:226-244`

```yaml
            cat > .env <<'ENVEOF'
            GROQ_API_KEY=${{ secrets.GROQ_API_KEY }}
            RESEND_API_KEY=${{ secrets.RESEND_API_KEY }}
            CONTACT_FROM_EMAIL=${{ vars.CONTACT_FROM_EMAIL }}
            CONTACT_TO_EMAIL=${{ vars.CONTACT_TO_EMAIL }}
            IMAGE=${{ needs.build-and-push.outputs.image }}
            APP_PORT=${{ vars.APP_PORT || 3002 }}
            ENVEOF
            chmod 600 .env

            echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
            docker compose pull
            # Normal path updates the container in place. On the first cutover the
            # pre-existing (non-compose) container holds the name — remove it once,
            # then compose adopts the name going forward.
            docker compose up -d --remove-orphans || {
              docker rm -f matthewswong-portfolio 2>/dev/null || true
              docker compose up -d --remove-orphans
            }
```

Read this deploy step as a script that must be safe to run twice, which is what makes running the job again after a half-finished deploy harmless. cat > .env rewrites the whole file each time, where >> would add a second copy of every line. chmod 600 sets the mode absolutely. docker compose up -d recreates the container only when its configuration or image changed. Even the fallback is re-runnable: rm -f with || true succeeds whether or not the old container exists.

`scripts/merge-blog-staging.mjs:12-23`

```javascript
 * Idempotent: a slug already in BLOG_POSTS_STATIC is skipped.
 *
 *   node scripts/merge-blog-staging.mjs [--dry-run]
 *
 * Design note — why this is plan-then-write rather than validate-then-write:
 * the previous version validated the staged CONTENT completely, then began
 * writing, and resolved each insertion anchor at write time. When a constant
 * moved module (REAL_IMAGE_SLUGS left blog-posts-static.ts), the third write
 * threw after messages/*.json and the post registry had already been updated —
 * a half-merged repo that a re-run would not fix, because by then the slug
 * existed in en.json and got skipped. Anchors are now resolved during
 * validation, so either every edit is possible or nothing is touched.
```

Idempotent by skipping: a post that is already registered is not merged again. The design note records how that went wrong. One write failed after earlier writes had landed, so the done marker existed, the re-run skipped the post, and the repo stayed half-merged. The fix is the rule behind every re-runnable job: settle everything that can fail before the first write, so a job is either fully done or untouched.

## What breaks

- **A new key on every attempt.** Generating the key inside the retry loop makes every retry a new request, and the protection is gone. Create the key once per operation, store it with the pending operation, and reuse it across retries, process restarts and page reloads. Stripe's guide warns that a new key can start a second operation.
- **Check the key, then save it, in two steps.** Two duplicates that arrive together both read no record for the key, both do the work, and the customer is charged twice. Claim the key first, in one atomic step that a unique constraint decides, mark it in progress, and only the request that won the claim does the work. The other gets 409.
- **A done marker written before the work is done.** Skip-if-exists is only idempotent if the marker means the whole job finished. If a later step fails, a re-run skips a half-done job forever, as this repo's merge script once did. Write the marker in the same transaction as the work, or settle everything that can fail before the first write.
- **Treating a 500 as permission to try again fresh.** A 500 is indeterminate: the operation may have had side effects. Stripe's guide says not to retry with a new key until you have resolved the original outcome, and to stop retrying an unresolved operation after the 24-hour window and reconcile it instead.
- **A key made from the content merges identical messages.** This site's contact key is the hash of the message, so two genuinely identical messages sent within Resend's 24 hours arrive once. For a contact form that is the point. For payments it is wrong: two identical orders are two orders, so there the client mints a key once per intended operation, such as crypto.randomUUID() kept until success and cleared when the input changes.

### HTTP methods by RFC 9110 and RFC 5789: safe, idempotent, and what a client may do after a timeout

| Method | Safe | Idempotent | After a timeout |
| --- | --- | --- | --- |
| GET, HEAD, OPTIONS | Yes | Yes | Retry automatically |
| PUT | No | Yes | Retry: it sets the same state again |
| DELETE | No | Yes | Retry: the resource is gone either way |
| POST | No | No | Retry only with an idempotency key |
| PATCH | No | No, unless the patch is written to be | Retry only if idempotent or keyed |

## Try it yourself

You need Node 22 or later, curl, and the sqlite3 command line (preinstalled on macOS; on Linux, install it from your package manager). A toy payments server on 127.0.0.1:4242 takes 1.5 seconds per charge, and no real payment or email API is involved.

1. Save the server as server.mjs and start it with node server.mjs in its own terminal. It prints the number of charges after each one.

   ```bash
   // server.mjs: a toy payments API. Each charge takes 1.5 s. With an
   // Idempotency-Key header, the first result is stored and replayed.
   import { createHash } from "node:crypto";
   import http from "node:http";
   
   const charges = [];
   const keys = new Map(); // key -> { fingerprint, done, status, body }
   const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
   
   function reply(res, status, body, headers = {}) {
     res.writeHead(status, { "Content-Type": "application/json", ...headers });
     res.end(JSON.stringify(body) + "\n");
   }
   
   http
     .createServer(async (req, res) => {
       if (req.method === "GET") return reply(res, 200, { charges: charges.length });
       let raw = "";
       for await (const chunk of req) raw += chunk;
       const key = req.headers["idempotency-key"];
       const fingerprint = createHash("sha256").update(raw).digest("hex");
       if (key) {
         const seen = keys.get(key);
         if (seen && seen.fingerprint !== fingerprint)
           return reply(res, 422, { error: "key reused with a different body" });
         if (seen && !seen.done)
           return reply(res, 409, { error: "a request with this key is in progress" });
         if (seen) return reply(res, seen.status, seen.body, { "Idempotent-Replayed": "true" });
         keys.set(key, { fingerprint, done: false }); // claim the key BEFORE the work
       }
       await sleep(1500); // the slow part, like a call to a card network
       const charge = { id: `ch_${charges.length + 1}`, body: raw };
       charges.push(charge);
       if (key) keys.set(key, { fingerprint, done: true, status: 201, body: charge });
       console.log("charges so far:", charges.length);
       reply(res, 201, charge);
     })
     .listen(4242, "127.0.0.1", () => console.log("listening on 127.0.0.1:4242"));
   ```
2. Lose a reply on purpose. curl -m 1 gives up after one second, before the 1.5-second charge answers, so it prints a timeout (error 28) although the server charged. Retry the same POST and read the count: 2 charges for one order.

   ```bash
   curl -sS -m 1 -X POST http://127.0.0.1:4242/charges -d 'order=A-1'
   curl -sS -X POST http://127.0.0.1:4242/charges -d 'order=A-1'
   curl -sS http://127.0.0.1:4242/charges
   ```
3. Do it again with an idempotency key made once, before the first attempt. The first call times out again; the retry gets the stored 201 with Idempotent-Replayed: true, and the count goes up by one, not two.

   ```bash
   KEY=$(node -p "crypto.randomUUID()")
   curl -sS -m 1 -X POST http://127.0.0.1:4242/charges -H "Idempotency-Key: $KEY" -d 'order=B-1'
   sleep 1
   curl -sS -i -X POST http://127.0.0.1:4242/charges -H "Idempotency-Key: $KEY" -d 'order=B-1'
   curl -sS http://127.0.0.1:4242/charges
   ```
4. Send a duplicate while the first request is still running. The duplicate prints 409 at once; the original prints 201 when it finishes.

   ```bash
   KEY=$(node -p "crypto.randomUUID()")
   curl -sS -w '%{http_code}\n' -X POST http://127.0.0.1:4242/charges -H "Idempotency-Key: $KEY" -d 'order=C-1' &
   sleep 0.2
   curl -sS -w '%{http_code}\n' -X POST http://127.0.0.1:4242/charges -H "Idempotency-Key: $KEY" -d 'order=C-1'
   wait
   ```
5. In the same terminal, reuse that key with a different body. The server refuses with 422, as the IETF draft suggests, instead of replaying an answer to a different request.

   ```bash
   curl -sS -w '%{http_code}\n' -X POST http://127.0.0.1:4242/charges -H "Idempotency-Key: $KEY" -d 'order=C-2'
   ```
6. Now natural idempotency in SQLite. The upsert inserts once (changes 1) and does nothing the second time (changes 0). Subtracting twice leaves 10; setting 11 twice leaves 11.

   ```bash
   sqlite3 :memory: <<'SQL'
   CREATE TABLE processed (message_id TEXT PRIMARY KEY);
   CREATE TABLE stock (sku TEXT PRIMARY KEY, qty INTEGER);
   INSERT INTO stock VALUES ('mug', 12);
   -- a redelivered message, handled twice
   INSERT INTO processed VALUES ('msg-42') ON CONFLICT (message_id) DO NOTHING;
   SELECT changes();
   INSERT INTO processed VALUES ('msg-42') ON CONFLICT (message_id) DO NOTHING;
   SELECT changes();
   -- add-by-amount twice, then set-a-value twice
   UPDATE stock SET qty = qty - 1 WHERE sku = 'mug';
   UPDATE stock SET qty = qty - 1 WHERE sku = 'mug';
   SELECT qty FROM stock;
   UPDATE stock SET qty = 11 WHERE sku = 'mug';
   UPDATE stock SET qty = 11 WHERE sku = 'mug';
   SELECT qty FROM stock;
   SQL
   ```
7. Stop the server with Ctrl+C in its terminal. It keeps everything in memory, so nothing is left behind.

## Quick quiz

1. **Which HTTP method is idempotent but not safe?**
   A. PUT
   B. GET
   C. POST
   D. PATCH
   Answer: A. PUT changes state, so it is not safe, but repeating it leaves the same state, so it is idempotent. GET is safe and therefore idempotent; POST is neither, and PATCH is neither unless the patch is written to be idempotent.

2. **A POST with idempotency key k1 times out, and the client retries. When should it generate a new key?**
   A. On every retry, so the server sees a fresh request
   B. Only for a new operation, never for a retry of this one
   C. After a 409, because k1 is now locked
   D. After a timeout, because k1 may be used up
   Answer: B. The key names the operation, so every retry of it carries k1. A 409 means the first attempt is still running: wait and retry with the same key, because a new key can start a second operation.

3. **A RabbitMQ consumer writes an order row from a message, then crashes before it acknowledges. What happens next?**
   A. The message is gone, and the order is safe
   B. RabbitMQ removes the duplicate for you
   C. It is redelivered, so the consumer must skip an order it already wrote
   D. It waits until the broker restarts
   Answer: C. Unacknowledged deliveries are requeued when the consumer's channel or connection closes, so delivery is at least once. Recording the message ID in the same transaction as the order lets the redelivery find it and skip.

## Think it through

- **Why is DELETE idempotent even though a second call may answer differently?** Idempotency is about the intended effect on the server, not the response. After one DELETE or two, the resource is gone; RFC 9110 says the response to a repeat might differ.
- **What does a server store for each idempotency key, and when?** A fingerprint of the body, the state of the work, and the status and body it answered. It claims the key as in progress before doing the work, so a duplicate in flight gets 409, and saves the result when the work ends.
- **A deploy script appends to .env with >>. What happens when the deploy runs again, and what is the fix?** Every line appears twice, and the file grows on each run. Rewrite the whole file with >, as this site's deploy does with cat > .env, so each run leaves the same file.
- **When is natural idempotency enough, and when do you need a key?** When the request names the resource and carries its full state, as with PUT, a set or an upsert on a natural unique key, repeats converge. When each request creates something new, such as a charge or an email, only a key the client chose can tell a retry from a new request.

## Go deeper

- [Idempotency Keys: Designing APIs That Survive Retries](https://www.matthewswong.com/en/blog/idempotency-keys-api-design/)
- [The Idempotent Consumer Pattern for Message Queues](https://www.matthewswong.com/en/blog/idempotent-consumer-pattern-message-queue/)
- [Designing Reliable Webhooks: Retries, Signing, and Replay](https://www.matthewswong.com/en/blog/webhooks-reliable-delivery-design/)

## Sources

- [RFC 9110: HTTP Semantics (safe and idempotent methods, PUT vs POST)](https://www.rfc-editor.org/rfc/rfc9110.html#name-idempotent-methods)
- [RFC 5789: PATCH Method for HTTP](https://www.rfc-editor.org/rfc/rfc5789.html)
- [IETF datatracker: The Idempotency-Key HTTP Header Field (expired draft, revision 07)](https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/)
- [Stripe API reference: Idempotent requests](https://docs.stripe.com/api/idempotent_requests)
- [Stripe docs: Advanced error handling (network errors, 409, retry window)](https://docs.stripe.com/error-low-level)
- [Resend docs: Idempotency keys](https://resend.com/docs/dashboard/emails/idempotency-keys)
- [RabbitMQ docs: Reliability guide (at-least-once, idempotent consumers)](https://www.rabbitmq.com/docs/reliability)
- [RabbitMQ docs: Consumer acknowledgements and publisher confirms](https://www.rabbitmq.com/docs/confirms)
- [microservices.io: Idempotent Consumer pattern](https://microservices.io/patterns/communication-style/idempotent-consumer.html)
- [PostgreSQL docs: INSERT, ON CONFLICT](https://www.postgresql.org/docs/current/sql-insert.html)
- [SQLite docs: UPSERT](https://sqlite.org/lang_upsert.html)
- [Docker docs: docker compose up](https://docs.docker.com/reference/cli/docker/compose/up/)
- [POSIX: chmod (absolute modes)](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/chmod.html)
- [POSIX: mkdir -p](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/mkdir.html)
- [POSIX Shell Command Language: redirecting and appending output](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html)
- [MDN: Crypto.randomUUID()](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID)

---

- HTML version: https://www.matthewswong.com/en/learn/system-design/idempotency-explained/
- Bahasa Indonesia: https://www.matthewswong.com/id/learn/system-design/idempotency-explained/
- Site index for agents: https://www.matthewswong.com/llms.txt
- Sitemap: https://www.matthewswong.com/sitemap.xml
- API description (OpenAPI): https://www.matthewswong.com/openapi.json