Idempotent Webhook Handlers: Dedupe Keys, Retry Windows and Sends That Happen Once

Imagine a support agent that answers inbound mail. A customer asks to move a meeting, the agent drafts a reply, and the customer gets two identical answers a minute apart. Nothing in the agent's logic was wrong. The same message reached it twice, and it did its job both times.

Duplicates come from the delivery path, not from the sender. Mail servers retry when an acknowledgment is late. Webhook senders retry when your endpoint is slow. Then your own recovery code replays messages after an outage. Each repeat looks like new work unless something remembers what already happened.

The fix is idempotency, which means running the same job twice has the same effect as running it once. For an email agent, that takes three decisions, and the sections below make them in order with working code. First, the ID that marks a repeat. Then where to record it. Then the reply.

Why does an email agent see the same message twice?

An email agent sees the same message twice because every layer under it delivers at least once. Nothing promises exactly once. SMTP, the protocol mail servers use to hand messages to each other, has this built in. RFC 5321 tells a receiving server it must answer quickly "to avoid receiving duplicate messages as the result of timeouts."

RFC 1047 describes the gap behind that rule. The receiving server has accepted the message, the sending server hasn't heard back yet, and if the connection drops in between, the sender delivers again. That memo is from 1988, and the timeout rule in RFC 5321 exists for the same reason.

Webhooks repeat the pattern one layer up. AgentMail delivers each event at least once and retries any delivery that doesn't get a 2xx answer, meaning any success status, within 15 seconds. A handler that waits on a slow model call before answering can run past that window and set off its own retries. The broader case for why inbound is harder than sending is in inbound is the hard part.

Which ID should you deduplicate on?

You should deduplicate on the ID that stays the same across the kind of repeat you are guarding against. Email gives you four candidates, and each one is stable across a different set of repeats. Picking the wrong one is how a dedupe check passes in testing and fails in production.

The pair matters more than either half. Here's why. One email sent to three of your inboxes arrives as three separate messages, so message_id alone would merge work that belongs to three agents. The Message-ID header is weaker still, because RFC 5322 makes it a SHOULD rather than a MUST and leaves uniqueness to whichever host generated it.

So the design uses two keys. event_id stops repeat deliveries at the door. inbox_id plus message_id guards the reply, because a catch-up replay from the API has no event at all.

Record the event before the agent acts

The handler verifies the request, writes one row, and answers. That's all. The row is the job, so a worker picks it up later and the handler never waits on a model. A unique constraint does the deduplication, which means two copies of one event arriving at the same moment still produce one row.

These are the two tables. The first is the queue of inbound events, the second is the record of replies the agent owes or has sent.

-- One row per inbound event (the job queue) and one per reply the agent owesCREATE TABLE inbound_events (  event_id    text PRIMARY KEY,  inbox_id    text NOT NULL,  message_id  text NOT NULL,  status      text NOT NULL DEFAULT 'queued',  received_at timestamptz NOT NULL DEFAULT now());CREATE TABLE agent_replies (  inbox_id        text NOT NULL,  message_id      text NOT NULL,  idempotency_key text NOT NULL,  reply_text      text NOT NULL,  sent_message_id text,  created_at      timestamptz NOT NULL DEFAULT now(),  PRIMARY KEY (inbox_id, message_id));-- psql output: CREATE TABLE, CREATE TABLE

The handler verifies the Svix signature over the raw body first. Svix is the service AgentMail delivers webhooks through, and the signature check needs the exact bytes, so the route reads the body with express.raw. It was tested with express 4.21 and svix 1.99 against Postgres.

// Verify the signature, record each event once, answer inside the 15 second windowimport express from "express";import pg from "pg";import { Webhook } from "svix";const app = express();const db = new pg.Pool(); // reads PGHOST, PGUSER, PGDATABASE from the environmentconst wh = new Webhook(process.env.AGENTMAIL_WEBHOOK_SECRET as string);app.post("/webhooks", express.raw({ type: "application/json" }), async (req, res) => {  let event: any;  try {    event = wh.verify(req.body, req.headers as Record<string, string>);  } catch {    return res.status(400).send(); // bad signature: not from AgentMail  }  if (event.event_type === "message.received") {    const { inbox_id, message_id } = event.message;    try {      const result = await db.query(        `INSERT INTO inbound_events (event_id, inbox_id, message_id)         VALUES ($1, $2, $3)         ON CONFLICT (event_id) DO NOTHING         RETURNING event_id`,        [event.event_id, inbox_id, message_id],      );      console.log(event.event_id, result.rows.length === 1 ? "queued" : "duplicate");    } catch {      return res.status(500).send(); // nothing recorded, so let the retry come    }  }  res.status(204).send(); // duplicates get a 2xx too, or the retries keep coming});app.listen(3000);// The same delivery posted twice prints:// evt_7f3a queued// evt_7f3a duplicate

Two details carry the weight. First, a duplicate gets the same 2xx as a first delivery, because an error would only make AgentMail try again. Second, a database failure gets a 500 on purpose, since nothing was recorded and the retry is the recovery. The webhook verification docs cover the Svix setup in Python as well.

The OWASP Webhook Security Cheat Sheet reaches the same rule from the security side. Persist processed event IDs and keep them across the publisher's retry period. Return 200 for an authenticated duplicate.

Make the reply itself happen once

The reply is the side effect that can't be undone, so it gets its own key. AgentMail's send, reply, forward and draft-send endpoints accept an Idempotency-Key header. A retry with the same key returns the original message ID and thread ID, and no second email goes out.

The key has to be the same every time the worker tries to answer the same message. Deriving it from inbox_id and message_id does that without storing anything extra. Message IDs contain angle brackets and an @ sign, which keys don't allow, so the worker hashes the pair.

# Reply once per inbound message, across webhook retries, restarts and catch-up replaysimport hashlibfrom agentmail import AgentMailclient = AgentMail()  # reads AGENTMAIL_API_KEYdef reply_key(inbox_id: str, message_id: str) -> str:    # Keys allow only A-Z a-z 0-9 - . _ ~ and message IDs carry < @ >, so hash the pair    return "reply-" + hashlib.sha256(f"{inbox_id}\n{message_id}".encode()).hexdigest()def reply_once(db, inbox_id: str, message_id: str, write_reply) -> str:    cur = db.cursor()    cur.execute(        "SELECT idempotency_key, reply_text, sent_message_id FROM agent_replies"        " WHERE inbox_id = %s AND message_id = %s",        (inbox_id, message_id),    )    row = cur.fetchone()    if row and row[2]:        return row[2]  # answered already, maybe days ago    if row is None:        row = (reply_key(inbox_id, message_id), write_reply(), None)  # the model runs once        cur.execute(            "INSERT INTO agent_replies (inbox_id, message_id, idempotency_key, reply_text)"            " VALUES (%s, %s, %s, %s)",            (inbox_id, message_id, row[0], row[1]),        )        db.commit()    sent = client.inboxes.messages.reply(        inbox_id=inbox_id,        message_id=message_id,        text=row[1],        request_options={"additional_headers": {"Idempotency-Key": row[0]}},    )    cur.execute(        "UPDATE agent_replies SET sent_message_id = %s WHERE inbox_id = %s AND message_id = %s",        (sent.message_id, inbox_id, message_id),    )    db.commit()    return sent.message_id# First run, then a rerun after a crash before the UPDATE landed:# sent <a1b2c3@agentmail.to> posts=1# sent <a1b2c3@agentmail.to> posts=2 emails=1 model_calls=1

The order inside reply_once is the point. The reply text is written and stored before the send, so a rerun after a crash sends the same text with the same key. That matters because the same key on a different request returns a 409 conflict. A model asked twice rarely writes the same reply twice, and without the stored text every crash recovery would turn into an error.

The worker is written against agentmail 2.0.18 for Python, and the header option comes from the idempotency docs. The loop around it picks up queued rows from inbound_events, calls reply_once, and marks each row done.

One gap is left, and it's about time. A send key expires 24 hours after the send completes. Webhook retries run for about 27.5 hours, and your own replays can reach back days. So if a row has no sent_message_id and is older than 24 hours, fetch the thread before sending. Look for a message labeled sent whose in_reply_to is the message you are answering. Found one? Record it and stop.

Exactly once is not on offer, so what do you build instead?

You build at-least-once delivery with idempotent effects, which is the only version of exactly once that holds up. Every effect the agent has needs its own key, not only the email. A ticket, a CRM update or a refund triggered by one message needs the same treatment. Derive its key from inbox_id and message_id. Record it with the effect.

Order needs care too. Events are not guaranteed to arrive in order, so a correction can land before the message it corrects. Sort by the timestamp in the payload, not by when your handler saw it.

A gateway in front of your endpoint can thin out duplicates too. Hookdeck's deduplication drops repeats inside a window of one second to one hour, and its docs call the feature best effort. That window is far shorter than a 27.5-hour retry schedule, so the handler keeps its own check either way.

How do you catch up after an outage without double replies?

You catch up from the API and run every recovered message through the same reply_once. Redelivery won't do it for you. If every delivery to an endpoint keeps failing for 5 days, AgentMail disables it, and no notification goes out. The webhook's enabled field reads false, so your monitoring has to check it. Events that occurred while it was disabled are not redelivered after you set enabled back to true.

To recover, list messages with after set to the time of the last event you processed and ascending set to true. Each message that comes back goes to the worker keyed on inbox_id plus message_id. Messages the agent already answered return early from reply_once, and the rest get exactly one reply.

Agents that are offline for long stretches can skip the outage problem with a polling webhook, which queues events until the agent fetches them. Polling webhooks only hold events from after they were created and are US region only for now. Before trusting either path, replay duplicates and slow answers locally; testing inbound email webhooks shows how.

How long should dedupe records live?

Dedupe records should outlive both the retry window and your longest catch-up replay. For inbound_events, that means more than 27.5 hours, and a week covers the retry schedule several times over. Prune older rows on a schedule if the table grows.

Keep agent_replies much longer. A replay after a disabled endpoint can reach back past five days. The reply row is what stops it from answering a customer again. The rows are small, so keep them as long as you keep the conversation.

Still wiring the webhook up? Start with building real-time agents with AgentMail webhooks, and for how replies attach to the right conversation, read email threading for AI agents. More guides for developers building on the API are collected on the AgentMail build hub.

What AgentMail does not do

We deliver each event at least once and give you a stable event_id and send keys, but we don't know which events your handler finished. Dedupe of your own side effects, such as tickets, CRM rows, payments and model calls, is yours to build. Order isn't guaranteed, a disabled endpoint isn't announced and its missed events aren't redelivered, and a send key lasts 24 hours.