A primer on Java 21 virtual threads with examples

Understanding the expense of threads

A thread is the smallest unit of a program's execution that the operating system's kernel manages. As a unit of execution within a process, a thread shares the same memory space and resources as other threads in the same process.

Multiple threads within a single process can run concurrently. Support for concurrency makes threads a critical component in systems that support many consumers simultaneously. A consumer in this context can be a system call in a variety of formats, such as an HTTP request on a web server or a remote procedure call under gRPC. It also can be tasks that must execute within a process, such as activities within a workflow process.

Programming using threads has become commonplace, particularly with the proliferation of large-scale systems that support millions of users. You'd be hard pressed to make a viable software system without them.

However, threads come with an expense that requires particular attention. Systems cannot make an unlimited number of threads.

The number of threads a system can support at any moment in time is constrained by two factors: CPU capacity and memory. If too many threads take up too much CPU capacity, processes can grind to a halt. Exhausting CPU capacity is possible in situations that are computationally intensive.

The more common constraint on thread capacity is memory. A Linux kernel typically requires around 2MB of memory to create a thread, according to José Paumard, a member of the Java Developer Relations team at Oracle. Large-scale websites that dedicate each HTTP request to a distinct thread can easily create a million threads, which would require 2TB of memory. This is a significant burden on system resources. Sometimes that amount of memory is available. Sometimes it's not, particularly when an application runs in a Linux container that has limited memory allocation.

There are frameworks for certain programming languages that aim to manage operating system threads. Examples include the .NET Task Parallel Library and the Erlang actor model, which has a special type of lightweight process that can be invoked on the order of millions of calls without incurring excessive memory consumption.

Java has the java.util.concurrent library, which uses thread pooling to optimize thread creation. In addition, as of version 21, Java supports virtual threads, which significantly simplifies the asynchronous programming typically used with threads.

How virtual threads work

Java 21 virtual threads are much lighter and more efficient than traditional operating system threads. Virtual threads are managed by the Java Runtime Environment (JRE).

A virtual thread is not a "thread" in the sense of operating system thread or the Java thread that's derived from a given operating system thread. Rather, a virtual thread is created using a special, private object named Continuation, which then creates the virtual thread against a Java thread.

When a virtual thread runs blocking code, which is code that forces threads to wait until an operation completes -- for example, executing an HTTP request to a web server -- the Continuation object unmounts the virtual thread from the Java thread it's using and moves the virtual thread's context into memory. The JRE keeps track of the call associated with the blocking code -- in this case, the HTTP request. Once the HTTP request returns a response, the virtual thread framework within the JRE creates a new virtual thread using the context stored in memory.

This notification pattern is similar to that of Node.js, which relegates blocking code to an event loop that notifies the Node.js runtime when code completes.

To create virtual threads, use the Thread.virtualThread() or the Executors.newVirtualThreadPerTaskExecutor() factory methods.

Virtual threads work like any other thread. Developers write code to start a thread, stop a thread and wait for a thread to finish, just as they would for regular threads. Virtual threads also support all the same APIs as do operating system threads, such as ThreadLocal, Lock and Semaphore.

Using virtual threads

To illustrate virtual threads in Java 21, we created a set of demonstration applications in a GitHub repository named SimpleVirtualThreads. The purpose is to compare the result of creating and running a million threads under Java 11 and Java 21.

The Java 11 code attempts to create a million threads constructing new threads.

Thread thread = new Thread(aTask));

The Java 21 code tries to create a million virtual threads using the static Thread.virtualThread() method.

The following code shows an excerpt of Java 11 code that creates one million threads of a task that is represented by a class named BlockedThread, which encapsulates blocking code.

final int numberOfThreads = 1_000_000;

 

for (int i = 0; i < numberOfThreads; i++) {

 Thread thread = new Thread(new BlockedThread(i));

 thread.start();

 String str = String.format("Java 11 thread number %s is running.", i);

 System.out.println(str);

}

When the Java 11 code runs in a Docker container using default memory settings, the code generates errors due to memory issues as shown in Figure 2. There simply isn't enough memory to support the number of intended threads.

The following excerpt of Java 21 code runs the same thread creation logic from within a Docker container that uses the default settings as the Java 11 example. However, instead of using Thread thread = new Thread(aTask)) to create a thread, this time the code uses the Thread.ofVirtual() static factory method to create a virtual thread.

final int numberOfThreads = 1_000_000;

 

for (int i = 0; i < numberOfThreads; i++) {

 Thread virtualThread = Thread.ofVirtual().unstarted(new BlockedThread(i));

 virtualThread.start();

 String str = String.format("Java 21 virtual thread number %s is running.", i);

 System.out.println(str);

}

As you can see in Figure 3, the code executes a million virtual threads under Java 21 without incident.

The Java 11 and Java 21 codes are available on the SimpleVirtualThreads repository with instructions to run the comparison in both Java 11 and Java 21 containers.

Putting it all together: Virtual threads in Java 21

Thread programming is a necessary part of modern software development. A commercially viable application must support a million, if not millions, of users. Before virtual threads came along, threading at this scale required special programming effort. Virtual threads reduce that effort and simplify programming.

In addition to making large-scale programming easier for the developer, virtual threads also provide an essential business benefit. They avoid failures that prevent delivery of service to customers when they need it. Virtual threads in general, and under Java 21 in particular, play an important role in the creation of large-scale applications that run efficiently and safely.

Bob Reselman is a software developer, system architect and writer. His expertise ranges from software development technologies to techniques and culture.