I still remember the panic of a Tuesday afternoon deployment back in 2022. Our dashboard turned a festive shade of red, and the logs were screaming one specific error over and over: RejectedExecutionException. We weren’t out of memory. We weren’t out of CPU. We were out of threads.
For years, my approach to Java Backend development involved a delicate dance of tuning thread pools. Set the core pool size too low, and throughput suffers. Set it too high, and the context switching overhead kills the CPU. It felt less like engineering and more like numerology. I spent hours tweaking ThreadPoolExecutor parameters, trying to find that “Goldilocks” zone for our Spring Boot services.
Fast forward to late 2025, and I rarely touch those settings anymore. The introduction and stabilization of Virtual Threads in recent Java LTS releases completely altered how I approach scalability. If you are still manually calculating optimal thread counts for I/O-bound applications, you might be doing unnecessary work.
The Heavy Cost of Platform Threads
To understand why I changed my workflow, we have to look at what we were actually doing before. In the traditional model (pre-Java 21), a java.lang.Thread was a thin wrapper around an operating system (OS) thread. This is a “Platform Thread.”
OS threads are expensive. They reserve a large chunk of memory for the stack (often 1MB or more), and the OS kernel has to schedule them. If you have a web server handling 10,000 concurrent requests, you can’t just spawn 10,000 platform threads. The server would crash. So, we used thread pools to cap the number of threads, forcing tasks to wait in a queue.
Here is the classic pattern I used to write everywhere:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class LegacyPoolExample {
public void processRequests() {
// Limiting to 200 threads to prevent crashing the OS
try (ExecutorService executor = Executors.newFixedThreadPool(200)) {
for (int i = 0; i < 10_000; i++) {
int requestId = i;
executor.submit(() -> {
handleRequest(requestId);
});
}
} // Executor auto-closes here
}
private void handleRequest(int id) {
// Simulate IO operation (DB call, API request)
try {
Thread.sleep(100);
System.out.println("Processed " + id + " on " + Thread.currentThread());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}The problem here is blocking. When handleRequest calls a database or an external API, that expensive OS thread sits there doing absolutely nothing but waiting for a response. It’s tied up. It can’t process other requests. This is why high-concurrency Java applications used to require complex reactive frameworks (like WebFlux or RxJava) to handle scale—they unblocked the thread but made the code significantly harder to debug and read.
Virtual Threads: The New Standard
Virtual threads changed the math. Unlike platform threads, virtual threads are managed by the JVM, not the OS. They are just Java objects stored in the heap. They are incredibly cheap to create—you can have millions of them.
When a virtual thread performs a blocking I/O operation (like reading from a socket), the JVM automatically unmounts it from the carrier thread (the actual OS thread). The OS thread is then free to execute other virtual threads. When the I/O completes, the JVM schedules the virtual thread back onto a carrier.
This means I can write simple, synchronous, blocking code—the kind that is easy to read and debug—and still get the scalability of asynchronous code. I don’t need to worry about thread pool exhaustion for I/O tasks anymore.
Here is how I write that same logic now:
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadFactory;
public class VirtualThreadExample {
public void processRequests() {
// No fixed pool size needed. New thread per task.
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
for (int i = 0; i < 10_000; i++) {
int requestId = i;
executor.submit(() -> {
handleRequest(requestId);
});
}
}
}
private void handleRequest(int id) {
// This blocking call now unmounts the virtual thread
// freeing the underlying OS thread for other work
try {
Thread.sleep(100);
System.out.println("Processed " + id + " on " + Thread.currentThread());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}Notice the switch to Executors.newVirtualThreadPerTaskExecutor(). I’m not pooling threads. I’m creating a new one for every single task. It feels wrong if you’ve been doing Java Development for a decade, but it is exactly how this feature is designed to be used.
Structured Concurrency
One of the biggest headaches with CompletableFuture or raw threads was error handling. If a sub-task failed, how did you ensure the main task knew about it? How did you cancel the other running sub-tasks to save resources? It often resulted in “thread leaks” where orphan threads kept running in the background.
With the adoption of structured concurrency patterns, I treat multiple concurrent tasks as a single unit of work. If one fails, the scope handles the cleanup. It keeps the code clean and the logic tight.
Here is a practical example using StructuredTaskScope. This pattern is invaluable when I need to fetch data from multiple microservices in parallel and combine the results.
import java.util.concurrent.StructuredTaskScope;
import java.util.concurrent.ExecutionException;
import java.util.function.Supplier;
public class UserDashboardService {
public UserDashboard loadDashboard(String userId) {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
// Fork tasks in parallel
Supplier userTask = scope.fork(() -> fetchUserProfile(userId));
Supplier> ordersTask = scope.fork(() -> fetchRecentOrders(userId));
Supplier> notificationsTask = scope.fork(() -> fetchNotifications(userId));
// Wait for all to finish, or fail if one fails
scope.join();
scope.throwIfFailed();
// Combine results
return new UserDashboard(
userTask.get(),
ordersTask.get(),
notificationsTask.get()
);
} catch (ExecutionException | InterruptedException e) {
throw new RuntimeException("Failed to load dashboard", e);
}
}
// Mock methods for demonstration
private UserProfile fetchUserProfile(String id) throws InterruptedException {
Thread.sleep(50); // Simulate DB
return new UserProfile(id);
}
private List fetchRecentOrders(String id) throws InterruptedException {
Thread.sleep(80); // Simulate Service call
return List.of();
}
private List fetchNotifications(String id) throws InterruptedException {
Thread.sleep(30); // Simulate Cache
return List.of();
}
// DTOs
record UserProfile(String id) {}
record Order() {}
record Notification() {}
record UserDashboard(UserProfile profile, List orders, List notes) {}
} In this example, ShutdownOnFailure() is the key. If fetchUserProfile throws an exception, the scope automatically cancels the other two tasks (interrupting their threads) and propagates the error. I don’t have to write complex cancellation logic manually. This aligns perfectly with Java Best Practices for robust error handling.
The “Pinning” Problem
It’s not all sunshine and rainbows. I ran into a specific performance killer when I first migrated a legacy library: Pinning.
A virtual thread gets “pinned” to its carrier OS thread if it executes code inside a synchronized block or calls a native method. While pinned, if it performs a blocking operation, it blocks the actual OS thread, defeating the purpose of virtual threads.
I learned to audit my code for synchronized blocks. If you are protecting a critical section that involves I/O, you should replace synchronized with ReentrantLock.
Here is the fix I applied to a custom caching component:
import java.util.concurrent.locks.ReentrantLock;
public class CacheComponent {
// BAD: Pins the thread
public synchronized void updateCacheLegacy(String key, String data) {
writeToDisk(key, data); // Blocking I/O while pinned!
}
private final ReentrantLock lock = new ReentrantLock();
// GOOD: Does not pin
public void updateCacheModern(String key, String data) {
lock.lock();
try {
writeToDisk(key, data); // Virtual thread can unmount here
} finally {
lock.unlock();
}
}
private void writeToDisk(String key, String data) {
// Simulate IO
try { Thread.sleep(10); } catch (InterruptedException e) {}
}
}Most modern Java libraries and frameworks (like Spring Boot 3.2+ and Hibernate 6.x) have already updated their internals to avoid pinning, but it is something I watch out for in older dependencies or in-house utility classes.
When Not to Use Virtual Threads
I don’t use virtual threads for everything. They are designed for throughput, not latency. Specifically, they shine for I/O-bound tasks (waiting for databases, networks, files). They are not a magic bullet for CPU-bound tasks.
If I have a task that crunches numbers, processes images, or does heavy cryptographic hashing for 5 seconds straight, a virtual thread won’t help. In fact, it might be slightly slower due to the overhead of the scheduler. For those heavy computational tasks, I still stick to a traditional ForkJoinPool or a fixed thread pool limited to the number of available processor cores.
Final Thoughts
The shift to virtual threads simplified my mental model of Java Concurrency. I no longer worry about “running out of threads” for web requests. I write synchronous, linear code that is easy to read, and the JVM handles the massive scale under the hood. It removed a whole category of infrastructure tuning from my daily workload.
If you haven’t audited your ExecutorService usage recently, take a look. You might find that replacing your complex thread pools with a virtual thread executor is the single best performance optimization you can make this year.