Time Limiter
Time Limiter
Chapter 2 established HTTP client timeouts as mandatory. The Resilience4J TimeLimiter adds a second layer: a timeout that wraps the entire call including any local processing, not just the HTTP request.
Why Two Timeout Layers
The HTTP client timeout (read timeout, connection timeout) operates at the socket level. It measures time waiting for bytes from the network. It does not measure:
- Time spent serializing the request body
- Time spent in a retry loop (each retry has its own timeout)
- Time spent waiting for a circuit breaker permit
- Time spent waiting for a bulkhead permit
- Time spent in fallback logic
The TimeLimiter wraps the entire decorated call chain. If the call is decorated with Retry, CircuitBreaker, Bulkhead, and the HTTP client, the TimeLimiter caps the total time across all of them.
// The call chain without TimeLimiter:
// Retry -> CircuitBreaker -> Bulkhead -> HTTP Client (with socket timeout)
// Time spent: retry_backoff + circuit_breaker_wait + bulkhead_wait + http_timeout
// With 3 retries, 200ms backoff, 100ms bulkhead wait, 500ms HTTP timeout:
// Worst case: 3 * (100ms + 500ms) + 200ms + 400ms = 2400ms
// With TimeLimiter:
// TimeLimiter(2s) -> Retry -> CircuitBreaker -> Bulkhead -> HTTP Client
// Total time capped at 2 seconds regardless of retry/backoff behaviorThe Internals: From Scratch
The TimeLimiter is the TimeoutWrapper from Chapter 2, but with awareness of CompletableFuture and cancellation:
// FROM SCRATCH - TimeLimiter that cancels the underlying future
public class TimeLimiter {
private final Duration timeoutDuration;
private final boolean cancelRunningFuture;
public TimeLimiter(Duration timeoutDuration, boolean cancelRunningFuture) {
this.timeoutDuration = timeoutDuration;
this.cancelRunningFuture = cancelRunningFuture;
}
public <T> T executeFuture(Supplier<CompletableFuture<T>> futureSupplier)
throws Exception {
CompletableFuture<T> future = futureSupplier.get();
try {
return future.get(timeoutDuration.toMillis(), TimeUnit.MILLISECONDS);
} catch (java.util.concurrent.TimeoutException e) {
if (cancelRunningFuture) {
// mayInterruptIfRunning=true: sets the thread's interrupt flag
// The HTTP client may or may not respect this
future.cancel(true);
}
throw new TimeLimiterTimeoutException(
"TimeLimiter timed out after " + timeoutDuration.toMillis() + "ms");
} catch (ExecutionException e) {
throw (Exception) e.getCause();
}
}
/**
* For synchronous calls: wraps the supplier in a CompletableFuture.
* This requires a separate thread for execution.
*/
public <T> T execute(Supplier<T> supplier, ExecutorService executor)
throws Exception {
return executeFuture(() ->
CompletableFuture.supplyAsync(supplier, executor));
}
}What the Scratch Implementation Reveals
cancelRunningFuture is a lie for blocking I/O. When future.cancel(true) is called, it sets the interrupt flag on the executing thread. If that thread is in a Thread.sleep() or Object.wait(), the InterruptedException is thrown and the thread is freed. If the thread is blocked on a socket read (InputStream.read()), the interrupt flag is set but the thread remains blocked. The thread is only freed when the socket timeout fires. This means cancelRunningFuture only works reliably with non-blocking I/O or when the HTTP client respects interrupts.
For the transaction platform using Spring RestClient with Apache HttpClient 5, the HTTP client does check the interrupt flag during I/O operations. Setting cancelRunningFuture to true with Apache HttpClient is effective. With the JDK’s default HttpURLConnection, it is not.
TimeLimiter needs its own thread for synchronous calls. The execute method that wraps a synchronous Supplier runs it on a separate thread via CompletableFuture.supplyAsync. This means the TimeLimiter consumes a thread from the provided executor while the calling thread waits. Two threads are used per call. This is the same tradeoff as the thread pool bulkhead: you get true timeout enforcement at the cost of an additional thread.
The Production Implementation
# PRODUCTION - application.yml
resilience4j:
timelimiter:
instances:
fraudDetection:
timeout-duration: 2s
# Total time budget for the fraud detection call,
# including retries, bulkhead wait, and HTTP timeout.
# Must be >= the sum of worst-case retry delays + HTTP timeout.
# With 500ms HTTP timeout and no retries: 2s is generous.
# This catches edge cases where local processing or GC pauses
# add unexpected latency.
cancel-running-future: true
# Cancel the underlying HTTP call when the timeout fires.
# With Apache HttpClient 5, this interrupts the socket read.
balanceCheck:
timeout-duration: 1500ms
cancel-running-future: true
paymentGateway:
timeout-duration: 8s
# Payment gateway has the longest normal latency (p99: 800ms)
# and 3 retries with backoff. Total budget:
# 3 attempts * 5s HTTP timeout + 200ms + 400ms backoff = ~16s worst case
# Set to 8s to fail before the absolute worst case,
# accepting that some retry-assisted recoveries may be cut short.
cancel-running-future: trueThe TimeLimiter timeout should be set based on the total time budget for the entire call chain, not based on any single layer’s timeout. When the TimeLimiter fires, it means the entire budget is exhausted and no further retries or waits should be attempted.
Interaction with Circuit Breaker
When the TimeLimiter fires, it produces a TimeoutException. This exception is recorded by the circuit breaker as a failure. If the TimeLimiter fires frequently enough to exceed the circuit breaker’s failure rate threshold, the circuit breaker opens. This is correct: if the dependency consistently cannot respond within the total time budget, the circuit breaker should stop attempts.
The configuration chain: TimeLimiter -> Retry -> CircuitBreaker -> Bulkhead -> HTTP Client. The TimeLimiter is outermost (except for retry in some configurations), ensuring that the total time is bounded regardless of what happens inside the chain.