Getty Images


A primer on Java 21 virtual threads with examples

Virtual threads let developers build applications that handle potentially millions of users and ensure consistent service delivery. Here's how Java 21 virtual threads work.

On the day before Thanksgiving 2020, the Amazon Kinesis data streaming service in AWS' main region US-East-1 went down for several hours. The company explained the outage in its subsequent failure report.

We were able to confirm a root cause, and it turned out this wasn't driven by memory pressure. Rather, the new capacity had caused all of the servers in the fleet to exceed the maximum number of threads allowed by an operating system configuration. As this limit was being exceeded, cache construction was failing to complete and front-end servers were ending up with useless shard-maps that left them unable to route requests to back-end clusters.

Simply put, that Amazon Kinesis system ran out of operating system threads. A machine can support only so many threads, and it must accommodate that limit. The system failed to make the accommodation and, as a result, stopped working. Because of service dependencies, the Kinesis outage affected other AWS services too, including internal ones. The result was major disruption among many popular third-party websites and businesses as well as Amazon itself just hours before Black Friday, one of the busiest shopping days of the entire year.

Accommodating thread limit has been part of a developer's work since the Linux kernel added threading in 1996. It's a laborious, detailed undertaking. Fortunately, new technologies have evolved to make working with threads easier and safer. One of the new technologies is virtual threads.

This article explains what virtual threads are and how they work, the important problem they solve, and an example of using virtual threads under Java.

Understanding the expense of threads

A thread is the smallest unit of a program's execution that the operating system's kernel manages. As a unit of execution within a process, a thread shares the same memory space and resources as other threads in the same process.

Multiple threads within a single process can run concurrently. Support for concurrency makes threads a critical component in systems that support many consumers simultaneously. A consumer in this context can be a system call in a variety of formats, such as an HTTP request on a web server or a remote procedure call under gRPC. It also can be tasks that must execute within a process, such as activities within a workflow process.

Programming using threads has become commonplace, particularly with the proliferation of large-scale systems that support millions of users. You'd be hard pressed to make a viable software system without them.

However, threads come with an expense that requires particular attention. Systems cannot make an unlimited number of threads.

The number of threads a system can support at any moment in time is constrained by two factors: CPU capacity and memory. If too many threads take up too much CPU capacity, processes can grind to a halt. Exhausting CPU capacity is possible in situations that are computationally intensive.

The more common constraint on thread capacity is memory. A Linux kernel typically requires around 2MB of memory to create a thread, according to José Paumard, a member of the Java Developer Relations team at Oracle. Large-scale websites that dedicate each HTTP request to a distinct thread can easily create a million threads, which would require 2TB of memory. This is a significant burden on system resources. Sometimes that amount of memory is available. Sometimes it's not, particularly when an application runs in a Linux container that has limited memory allocation.

There are frameworks for certain programming languages that aim to manage operating system threads. Examples include the .NET Task Parallel Library and the Erlang actor model, which has a special type of lightweight process that can be invoked on the order of millions of calls without incurring excessive memory consumption.

Java has the java.util.concurrent library, which uses thread pooling to optimize thread creation. In addition, as of version 21, Java supports virtual threads, which significantly simplifies the asynchronous programming typically used with threads.

How virtual threads work

Java 21 virtual threads are much lighter and more efficient than traditional operating system threads. Virtual threads are managed by the Java Runtime Environment (JRE).

A virtual thread is not a "thread" in the sense of operating system thread or the Java thread that's derived from a given operating system thread. Rather, a virtual thread is created using a special, private object named Continuation, which then creates the virtual thread against a Java thread.

How Java 21's virtual threading with the Continuation object works.
The virtual thread framework's Continuation object controls a virtual thread's lifetime in terms of blocking code.

When a virtual thread runs blocking code, which is code that forces threads to wait until an operation completes -- for example, executing an HTTP request to a web server -- the Continuation object unmounts the virtual thread from the Java thread it's using and moves the virtual thread's context into memory. The JRE keeps track of the call associated with the blocking code -- in this case, the HTTP request. Once the HTTP request returns a response, the virtual thread framework within the JRE creates a new virtual thread using the context stored in memory.

This notification pattern is similar to that of Node.js, which relegates blocking code to an event loop that notifies the Node.js runtime when code completes.

To create virtual threads, use the Thread.virtualThread() or the Executors.newVirtualThreadPerTaskExecutor() factory methods.

Virtual threads work like any other thread. Developers write code to start a thread, stop a thread and wait for a thread to finish, just as they would for regular threads. Virtual threads also support all the same APIs as do operating system threads, such as ThreadLocal, Lock and Semaphore.

Using virtual threads

To illustrate virtual threads in Java 21, we created a set of demonstration applications in a GitHub repository named SimpleVirtualThreads. The purpose is to compare the result of creating and running a million threads under Java 11 and Java 21.

The Java 11 code attempts to create a million threads constructing new threads.

Thread thread = new Thread(aTask));

The Java 21 code tries to create a million virtual threads using the static Thread.virtualThread() method.

The following code shows an excerpt of Java 11 code that creates one million threads of a task that is represented by a class named BlockedThread, which encapsulates blocking code.

final int numberOfThreads = 1_000_000;


for (int i = 0; i < numberOfThreads; i++) {

 Thread thread = new Thread(new BlockedThread(i));


 String str = String.format("Java 11 thread number %s is running.", i);



When the Java 11 code runs in a Docker container using default memory settings, the code generates errors due to memory issues as shown in Figure 2. There simply isn't enough memory to support the number of intended threads.

Memory errors result when running one million virtual threads under Java 11.
Running one million virtual threads under Java 11 creates memory errors.

The following excerpt of Java 21 code runs the same thread creation logic from within a Docker container that uses the default settings as the Java 11 example. However, instead of using Thread thread = new Thread(aTask)) to create a thread, this time the code uses the Thread.ofVirtual() static factory method to create a virtual thread.

final int numberOfThreads = 1_000_000;


for (int i = 0; i < numberOfThreads; i++) {

 Thread virtualThread = Thread.ofVirtual().unstarted(new BlockedThread(i));


 String str = String.format("Java 21 virtual thread number %s is running.", i);



As you can see in Figure 3, the code executes a million virtual threads under Java 21 without incident.

Running one million virtual threads under Java 21 completes with no memory errors.
You can run one million virtual threads under Java 21 without any memory errors.

The Java 11 and Java 21 codes are available on the SimpleVirtualThreads repository with instructions to run the comparison in both Java 11 and Java 21 containers.

Putting it all together: Virtual threads in Java 21

Thread programming is a necessary part of modern software development. A commercially viable application must support a million, if not millions, of users. Before virtual threads came along, threading at this scale required special programming effort. Virtual threads reduce that effort and simplify programming.

In addition to making large-scale programming easier for the developer, virtual threads also provide an essential business benefit. They avoid failures that prevent delivery of service to customers when they need it. Virtual threads in general, and under Java 21 in particular, play an important role in the creation of large-scale applications that run efficiently and safely.

Bob Reselman is a software developer, system architect and writer. His expertise ranges from software development technologies to techniques and culture.

Dig Deeper on Core Java APIs and programming techniques

App Architecture
Software Quality
Cloud Computing