Zero-Loss Webhook Delivery with Java 21 Virtual Threads

I built a zero-loss webhook delivery engine with Java 21 Virtual Threads

WireVault is a specialized delivery engine designed to eliminate data loss from external providers like Stripe and PayPal during server downtime. The system captures every incoming webhook and persists it to a dedicated PostgreSQL instance before processing. It leverages Java 21 Project Loom to handle high-concurrency delivery tasks using lightweight virtual threads.

Why This Matters

In modern distributed architectures, the reliability of webhook ingestion is often the weakest link. When a target server experiences downtime or high latency, synchronous webhook endpoints fail, leading to permanent data loss and out-of-sync states between payment processors and application databases. This necessitates a buffer layer that prioritizes persistence over immediate processing.

By moving from a blocking I/O model to Java 21 Virtual Threads, developers can maintain thousands of concurrent connections for retries without exhausting system resources. This shift from ‘ideal’ synchronous processing to a resilient, store-and-forward model ensures that every event is eventually delivered, regardless of transient failures in the underlying infrastructure.

Key Insights

• Java 21 Virtual Threads (Project Loom) enable the system to manage high-volume webhook retries without the overhead of traditional OS threads.
• PostgreSQL 16 serves as the persistence layer, utilizing GIN indexes on JSONB columns to provide instant payload search capabilities.
• The engine implements exponential backoff strategies to prevent overwhelming downstream services during recovery phases.
• Quarkus integration facilitates rapid deployment, allowing the entire stack to be initialized via Docker Compose in approximately 30 seconds.
• A WebSocket-based live dashboard provides real-time observability into delivery status, mimicking ‘tail -f’ functionality for webhooks.
• The architecture supports on-demand replay of any stored webhook, allowing for easy recovery from application-level logic errors.

Working Examples

Command to deploy the WireVault stack including Java 21 service and PostgreSQL 16.

docker-compose up -d

Practical Applications

• Use Case: E-commerce platforms processing Stripe or PayPal payments can use WireVault to ensure payment confirmations are stored during maintenance windows.
• Pitfall: Processing webhooks synchronously without a persistence buffer leads to lost events when the application server is down or under heavy load.
• Use Case: Systems requiring high auditability can leverage the GIN-indexed JSONB storage to instantly query historical webhook payloads for debugging purposes.
• Pitfall: Implementing retry logic without exponential backoff can create a thundering herd problem that crashes a recovering server.

References:

• https://github.com/Ameliob18/wirevault
• https://dev.to/ameliob18/i-built-a-zero-loss-webhook-delivery-engine-with-java-21-virtual-threads-2can