Your Message Queue is Lying to You: Why the Inbox Pattern is the Only Way to Stop Duplicate Processing

Your message queue is lying to you. It promised “at-least-once delivery” in the documentation, but what that actually means is: you’ll see the same message multiple times, and it’s your problem to deal with it. Networks are unreliable, consumers crash mid-processing, and brokers redeliver messages you’ve already handled. If you’re processing payments, sending receipts, or updating projections, duplicates aren’t just annoying, they’re billing errors, double-shipped orders, and customers getting two emails for one purchase.

The Inbox Pattern is the consumer-side answer to this mess. It’s a durable, transactional way to make duplicate messages harmless. And unlike the outbox pattern (which handles the producer side), the inbox is where the real battle is won.

Let’s dig into what it actually takes to implement this correctly, what the chaos tests reveal, and why most “idempotency solutions” are built on sand.

The Problem: Exactly-Once Delivery is a Myth

Here’s the uncomfortable truth: exactly-once delivery doesn’t exist in distributed systems. Every broker you’ve used, Kafka, RabbitMQ, SQS, Pub/Sub, delivers at-least-once by default. The Kafka design documentation is refreshingly honest about this: exactly-once delivery into external systems requires those systems’ cooperation. Your PostgreSQL database can’t join a Kafka transaction, so there’s always a window.

Consider this scenario from a recent chaos test walkthrough: a payment service processes 1,000 payments. The producer crashes after publishing but before committing its offset. Result? 201 duplicate messages on the topic. Every payment event appears twice, with the same event IDs.

The queue didn’t fail. The protocol worked exactly as designed. But your consumer now faces a choice: process the same payment twice, or figure out how to recognize and skip the duplicate.

The Inbox Pattern: A Durable “Already Seen” List

The Inbox Pattern solves this by storing every processed event ID in a dedicated table, the inbox, and checking it before executing any business logic. The key insight is that the deduplication check and the business logic must commit in the same transaction.

Here’s what the core of it looks like in PostgreSQL:

CREATE TABLE processed_events (
    event_id     uuid PRIMARY KEY,
    processed_at timestamptz NOT NULL DEFAULT now()
);

And the consumer logic, using INSERT ... ON CONFLICT DO NOTHING as the deduplication mechanism:

def handle(conn, event):
    with conn.transaction():
        claimed = conn.execute(
            "INSERT INTO processed_events (event_id) VALUES (%s) ON CONFLICT DO NOTHING",
            (event["event_id"],),
        ).rowcount
        if not claimed:
            return "duplicate, skipped"

        # Business logic: update projection, send email, whatever
        conn.execute(
            """INSERT INTO payment_status (payment_id, status) VALUES (%s, %s)
               ON CONFLICT (payment_id) DO UPDATE SET status = EXCLUDED.status""",
            (event["payment_id"], event["type"].removeprefix("payment.")),
        )

The rowcount check tells you whether the event was new. Zero means it’s a duplicate, skip the business logic. One means it’s fresh, process it. Both the deduplication claim and the business effects commit atomically. If the consumer crashes after the commit but before acknowledging the message, the redelivery hits the unique key and does nothing.

The Transaction is Non-Negotiable

Here’s where most implementations go wrong: they check the inbox table, find the event already processed, and skip the business logic, but they do it outside the transaction. That’s a race condition waiting to happen. Two consumers can check the inbox simultaneously, both see no record, and both process the event.

The transactional integrity of the Inbox Pattern is what makes it work. The ON CONFLICT DO NOTHING insert acts as a claim: it atomically checks existence and inserts if absent. The unique constraint on event_id enforces this at the database level, not just in application logic. Application-level checks are not enough when two workers or two HTTP requests run concurrently, the database constraint is the only thing you can trust.

The Chaos Test: Proof in the Numbers

The most compelling evidence comes from a chaos test running 1,000 payments through a deliberately broken pipeline. Two relays crash after producing on 10% of batches. The notifier crashes before committing its offset on 0.2% of events. Here’s what the verification script reported:

payments captured:          1000
outbox events:              2000 (0 unpublished)
messages on the topic:      2201 (201 duplicates)
events processed once:      2000
receipts sent:              1000 for 1000 payments
projection shows captured:  1000

Read those numbers carefully. 201 duplicate messages, yet exactly 1,000 receipts and exactly 1,000 payments captured. The inbox absorbed every single duplicate. Without it, that second receipt would have been a real email to a real customer.

The chaos test also revealed something subtle about ordering. With the crashes, 98 payments had their authorized event arrive after their captured event. The topic was out of order because crashed batches get replayed whole. Without the inbox, those payments would flip back to authorized in the projection until the replay finished.

The TTL Trap: Clean Up or Pay the Price

The inbox table grows forever if you don’t clean it. Every processed event ID is dead weight. But here’s the trap: if you clean up too aggressively, you break the deduplication guarantee.

The guidance from practitioners who’ve been burned is clear: set a TTL on inbox records, but make sure it’s longer than the queue’s retention period. If events are retained for 7 days, cleaning up inbox records after 3 days means a replayed event from day 5 will process again, and your idempotency check becomes useless.

The standard approach is an expires_at column with opportunistic cleanup:

-- Delete published events older than 7 days, 10,000 rows per run.
DELETE FROM outbox
WHERE id IN (
    SELECT id FROM outbox
    WHERE published_at < now() - interval '7 days'
    ORDER BY id
    LIMIT 10000
);

Deleting in capped batches keeps each cleanup run short. The 7-day window should be sized to how long you might need to replay or inspect events. If you’re processing payments, err on the side of longer retention.

What the Inbox Pattern Can’t Fix

Time for some brutal honesty. The Inbox Pattern handles duplicate message processing, but it has limits:

The “Just Use a Shared Database” Critique

There’s a legitimate criticism of the entire outbox/inbox approach: if you’re going to slow your communications down to the speed of two ACIDic transactional database commits, why not just share a database table and skip the queue entirely?

The answer comes down to fan-out and decoupling. A queue lets multiple consumers subscribe to the same event independently. A shared table couples all consumers to the same schema, the same transaction load, and the same database. Handling duplicates and ordering issues in event-driven systems requires the abstraction boundary that a queue provides, even with the added complexity of deduplication on the consumer side.

Choosing the Right Idempotency Pattern

The Inbox Pattern isn’t the only way to achieve idempotency. There are alternative idempotency patterns for message-driven systems worth evaluating:

• Natural idempotency: If your processing operation is naturally idempotent (setting a status, updating a timestamp), you might not need a dedup table at all.
• Deduplication keys: Some brokers support built-in deduplication (SQS FIFO’s MessageDeduplicationId, for example). These only prevent producer-side duplicates, not redeliveries.
• State-based idempotency: If your business logic checks the current state before acting (e.g., only transition from authorized to captured), you get idempotency for free, but you need a WHERE clause that fails on invalid transitions.

The Inbox Pattern is the most general-purpose solution because it works regardless of the broker and the business logic. But it requires the transaction discipline we’ve discussed.

The Bottom Line

The Inbox Pattern costs you a table, a unique constraint, and a discipline: never check-then-act across a transaction boundary. In return, it gives you the one guarantee that matters in distributed systems, your business effects happen exactly once, even when messages arrive twice.

The chaos test numbers are the proof: 1,000 payments, 201 duplicates on the topic, exactly 1,000 receipts sent. No reconciliation. No manual cleanup. No “sorry, you got two charges.”

Your message queue is going to deliver duplicates. The question isn’t whether it happens, it’s whether your system survives it. With the Inbox Pattern, it does.