Execution Isolation with Virtual Threads

Java 21 introduces Virtual Threads as a permanent feature, designed to solve the scalability bottleneck where OS threads are exhausted before CPU or Network resources. Virtual threads are lightweight, user-mode threads managed by the Java Virtual Machine (JVM) rather than the operating system. They are optimized for high-throughput blocking tasks and offer significant improvements over traditional platform threads in memory footprint and context switching efficiency.

Key Characteristics of Virtual Threads

Virtual threads have several key characteristics that make them suitable for high-throughput concurrency. They utilize dynamic heap-based stacks starting at bytes to kilobytes, compared to the approximately 1MB allocated for each platform thread. The creation cost of virtual threads is very low, as they are JVM objects, whereas platform threads require an OS syscall. The JVM scheduler, specifically a Work-Stealing ForkJoinPool, is used for virtual threads, allowing for more efficient scheduling and better utilization of available processors.

Comparison with Platform Threads

The following table summarizes the key differences between platform threads and virtual threads:

Handling High-Cardinality Concurrent Intents

To handle high-cardinality concurrent intents, it is recommended to use an executor service that starts a new virtual thread for every submitted task. This can be achieved using Executors.newVirtualThreadPerTaskExecutor(). To limit resource usage, a Semaphore can be used to throttle the number of concurrent executions. The following code example demonstrates how to implement this:

import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;

public class IntentExecutor {
    // Limit resource-intensive intents (e.g., to 100 concurrent DB connections)
    private static final Semaphore RESOURCE_LIMITER = new Semaphore(100);

    public void handleIntents(List<Intent> intents) {
        try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
            for (Intent intent : intents) {
                executor.submit(() -> {
                    try {
                        RESOURCE_LIMITER.acquire();
                        processIntent(intent);
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                    } finally {
                        RESOURCE_LIMITER.release();
                    }
                });
            }
        }
    }

    private void processIntent(Intent intent) {
        // Blocking I/O work here... 
    }
}

## Best Practices for Virtual Threads

When using virtual threads, it is essential to follow best practices to ensure efficient and scalable execution. These include never pooling virtual threads, using a Semaphore to limit concurrency, and avoiding blocking I/O within synchronized blocks to prevent carrier thread pinning. Additionally, using `ReentrantLock` instead of `synchronized` blocks can help prevent thread pinning and ensure efficient unmounting.

## Sources

[1] Java 21 Documentation: Virtual Threads
[2] JEP 444: Virtual Threads