The Transactional Outbox and Exactly-Once Delivery

The Black Box

The logistics platform needs to publish a PackageReadyForDispatch event to Kafka whenever a package status changes to READY_FOR_DISPATCH. The application writes to PostgreSQL and publishes to Kafka. If the Kafka publish fails after the PostgreSQL commit, the event is lost. If the application crashes between the PostgreSQL commit and the Kafka publish, the event is lost. The dual write problem from Chapter 7 appears again.

The transactional outbox solves this: write the event to a table in the same PostgreSQL transaction as the data change. Debezium reads the outbox table via CDC and publishes to Kafka. No dual write. No lost events.

The Mechanism

Outbox Table Design

-- Concept: outbox table for the logistics platform
CREATE TABLE outbox (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    aggregate_type VARCHAR(100) NOT NULL,    -- "Package", "Route", "Inventory"
    aggregate_id VARCHAR(100) NOT NULL,      -- "PKG-40291"
    event_type VARCHAR(100) NOT NULL,        -- "PackageReadyForDispatch"
    payload JSONB NOT NULL,                  -- Event data
    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

-- No indexes beyond the primary key. This table is append-only and read by CDC.
-- Debezium reads it via the WAL, not via SQL queries.

The application writes to the outbox in the same transaction as the data change:

// Concept: atomic data change + event publication via outbox
void markReadyForDispatch(Connection conn, String packageId) throws SQLException {
    conn.setAutoCommit(false);

    // Data change
    try (var stmt = conn.prepareStatement(
            "UPDATE packages SET status = 'READY_FOR_DISPATCH' WHERE package_id = ?")) {
        stmt.setString(1, packageId);
        stmt.executeUpdate();
    }

    // Event (same transaction)
    try (var stmt = conn.prepareStatement(
            "INSERT INTO outbox (aggregate_type, aggregate_id, event_type, payload) " +
            "VALUES (?, ?, ?, ?::jsonb)")) {
        stmt.setString(1, "Package");
        stmt.setString(2, packageId);
        stmt.setString(3, "PackageReadyForDispatch");
        stmt.setString(4, String.format(
            "{\"packageId\":\"%s\",\"warehouse\":\"WH-042\",\"timestamp\":\"%s\"}",
            packageId, Instant.now()));
        stmt.executeUpdate();
    }

    conn.commit();  // Both writes committed atomically
}

Debezium Outbox Event Router

Debezium provides a built-in transformation that reads outbox table inserts and routes them to Kafka topics based on the aggregate_type and event_type fields:

// Concept: Debezium outbox event router configuration
{
    "name": "logistics-postgres-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "table.include.list": "public.outbox",
        "transforms": "outbox",
        "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
        "transforms.outbox.route.topic.replacement": "events.${routedByValue}",
        "transforms.outbox.table.field.event.id": "id",
        "transforms.outbox.table.field.event.key": "aggregate_id",
        "transforms.outbox.table.field.event.type": "event_type",
        "transforms.outbox.table.field.event.payload": "payload",
        "transforms.outbox.route.by.field": "aggregate_type"
    }
}

// This configuration:
// 1. Captures inserts to the outbox table
// 2. Uses aggregate_type to route to a topic: events.Package, events.Route, etc.
// 3. Uses aggregate_id as the Kafka message key (for partition ordering)
// 4. Uses payload as the Kafka message value
// 5. Uses id as a unique event identifier (for deduplication)

The Observable Consequence

Exactly-Once Delivery Does Not Exist

Debezium provides at-least-once delivery. If the connector restarts, it may re-read WAL records that were already published to Kafka. The outbox table insert appears in Kafka twice. The downstream consumer must handle duplicates.

// Concept: idempotent consumer using a processed events table
// Track which event IDs have been processed. Skip duplicates.

void processEvent(ConsumerRecord<String, String> record) throws SQLException {
    JsonNode event = objectMapper.readTree(record.value());
    String eventId = event.path("id").asText();

    // Check if already processed
    try (var check = conn.prepareStatement(
            "SELECT 1 FROM processed_events WHERE event_id = ?")) {
        check.setString(1, eventId);
        if (check.executeQuery().next()) {
            return; // Already processed. Skip.
        }
    }

    // Process the event
    String eventType = event.path("event_type").asText();
    if ("PackageReadyForDispatch".equals(eventType)) {
        dispatchPackage(event.path("payload"));
    }

    // Record as processed (same transaction as the processing if possible)
    try (var insert = conn.prepareStatement(
            "INSERT INTO processed_events (event_id, processed_at) VALUES (?, now())")) {
        insert.setString(1, eventId);
        insert.executeUpdate();
    }
}

The alternative to deduplication is idempotent processing: design the consumer so that processing the same event twice produces the same result. Setting a package status to READY_FOR_DISPATCH is idempotent. Incrementing a counter is not.

Outbox Table Cleanup

The outbox table grows as events are written. Since Debezium reads from the WAL, the outbox rows are needed only until they are captured. After that, they can be deleted.

-- Concept: outbox table cleanup
-- Delete outbox rows older than 24 hours.
-- Debezium has already read them from the WAL.
-- The 24-hour retention provides a manual replay window.

DELETE FROM outbox WHERE created_at < now() - interval '24 hours';

-- Schedule this as a cron job or a pg_cron task:
SELECT cron.schedule('clean-outbox', '0 * * * *',
    $$DELETE FROM outbox WHERE created_at < now() - interval '24 hours'$$);

The Decision Rule

Use the transactional outbox when the application needs to publish domain events with the same atomicity guarantees as the data change. The outbox table is the coordination mechanism that avoids dual writes.

Use direct CDC (without the outbox) when you need to replicate database row changes as-is. CDC on the packages table directly captures every row change. The outbox pattern is only necessary when the event payload differs from the row content or when you need to control the event structure independently of the table schema.

Design consumers to be idempotent. At-least-once delivery is the realistic guarantee. Exactly-once requires end-to-end transactional guarantees across Kafka and the consumer’s data store, which is architecturally expensive and rarely justified outside of financial systems.