Java 21 💣💥

Nicolai Parlog

nipafx.dev / @nipafx

JUG Karlsruhe

@jugka@ijug.social

Developer Advocate

Java Team at Oracle

Accento 2023

Karlsruhe
October 24th, 25th
50% discount: AccJUGKA23

I have two free tickets!

Let’s get started!

this talk covers Java 21
this is a showcase, not a tutorial
slides at slides.nipafx.dev/java-x
(hit "?" to get navigation help)

Lots to talk about!

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Why upgrade?

Costs of running on old versions:

support contract for Java
waning support in libraries / frameworks

Why upgrade?

Costs of not running on new versions:

lower productivity
less observability and performance
(more on that later)
less access to talent
bigger upgrade costs

Why upgrade?

Resistence is futile.

How to upgrade?

Preparations:

stick to supported APIs
stick to standardized behavior
stick to well-maintained projects
keep dependencies and tools up to date
stay ahead of removals (jdeprscan)
build on many JDK versions

How to upgrade?

Prepare by building on multiple JDK versions:

your baseline version
every supported version since then
latest version
EA build of next version

How to upgrade?

It’s not necessary to build …

… each commit on all versions
… the whole project on all versions

Build as much as feasible.

What about LTS?

Within OpenJDK, there is no LTS.

⇝ has no impact on features, reliability, etc.

It’s a vendor-centric concept
to offer continuous fixes
(usually for money).

You’re paying not to get new features.

What about LTS?

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

A simple app

Features:

scrapes GitHub projects
creates Page instances:
- GitHubIssuePage
- GitHubPrPage
- ExternalPage
- ErrorPage
further processes pages

A simple app

Features:

display as interactive graph
compute graph properties
categorize pages by topic
analyze mood of interactions
process payment for analyses
etc.

A simple architecture?

How to implement features?

methods on Page 😧
visitor pattern 😫
pattern matching 🥳

Pattern Matching

Ingredients:

switch expressions ⑭ (JEP 361)
type pattern matching ⑯ (JEP 394)
sealed types ⑰ (JEP 409)
patterns in switch ㉑ (JEP 441)

Approach:

make Page sealed
implement features as methods outside of Page
accept Page parameters and switch over it

Sealed Page

Sealed types limit inheritance,
by only allowing specific subtypes.

public sealed interface Page
	permits GitHubIssuePage, GitHubPrPage,
			ExternalPage, ErrorPage {
	// ...
}

Switch over Page

public void categorize(Page page) {
	switch (page) {
		case GitHubIssuePage issue
			-> categorizeIssue(issue);
		case GitHubPrPage pr
			-> categorizePr(pr);
		case ExternalPage external
			-> categorizeExternal(external);
		case ErrorPage error
			-> categorizeError(error);
	}
}

Maintainability

Unlike an if-else-if-chain,
a pattern switch needs to be exhaustive.

Fulfilled by:

switching over a sealed types
a case per subtype
avoiding the default branch

⇝ Adding a new subtype causes compile error!

Dynamic dispatch

Dynamic dispatch selects the invoked method by type.

As language feature:

via inheritance
makes method part of API

What if methods shouldn’t be part of the API?

Dynamic dispatch

Without methods becoming part of the API.

Via visitor pattern:

makes "visitation" part of API
cumbersome and indirect

Via pattern matching (new):

makes "sealed" part of type
straight-forward

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Data-Oriented Programming

Use Java’s strong typing to model data as data:

use types to model data, particularly:
- data as data with records
- alternatives with sealed types
use (static) methods to model behavior, particularly:
- exhaustive switch without default
- pattern matching to destructure polymorphic data

Data-Oriented Programming…

… isn’t funtional programming

but it’s similar (data + functions)
first priority is data, not functions

… doesn’t kill object-oriented programming

use OOP to modularize large systems
use DOP to model small, data-focused (sub)systems

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

A simple web request

Imagine a hypothetical HTTP request:

interpret request
query database (blocks)
process data for response

Resource utilization:

good for 1. and 3.
really bad for 2.

How to implement that request?

Synchronous

Align application’s unit of concurrency (request)
with Java’s unit of concurrency (thread):

use thread per request
simple to write, debug, profile
blocks threads on certain calls
limited number of platform threads
⇝ bad resource utilization
⇝ low throughput

Asynchronous

Only use threads for actual computations:

use non-blocking APIs (futures / reactive streams)
harder to write, challenging to debug/profile
incompatible with synchronous code
shares platform threads
⇝ great resource utilization
⇝ high throughput

Conflict!

There’s a conflict between:

simplicity
throughput

Nota Bene

There are other conflicts:

design vs performance (⇝ Valhalla)
explicitness vs succinctness (⇝ Amber)
flexibility vs safety (⇝ Panama)
optimization vs specification (⇝ Leyden)

Enter virtual threads!

A virtual thread:

is a regular Thread
low memory footprint ([k]bytes)
small switching cost
scheduled by the Java runtime
requires no OS thread when waiting

Virtual things

Virtual memory:

maps large virtual address space
to limited physical memory
gives illusion of plentiful memory

Virtual threads:

map large number of virtual threads
to a small number of OS threads
give the illusion of plentiful threads

Virtual things

Programs rarely care about virtual vs physical memory.

Programs need rarely care about virtual vs platform thread.

Instead:

write straightforward (blocking) code
runtime shares available OS threads
reduces the cost of blocking to near zero

Example

try (var executor = Executors
		.newVirtualThreadPerTaskExecutor()) {
	IntStream
		.range(0, 1_000_000)
		.forEach(number -> {
			executor.submit(() -> {
				Thread.sleep(Duration.ofSeconds(1));
				return number;
			});
		});
} // executor.close() is called implicitly, and waits

Effects

Virtual threads:

remove "number of threads" as bottleneck
match app’s unit of concurrency to Java’s

⇝ simplicity && throughput

Performance

Virtual threads aren’t "faster threads":

same number of CPU cycles
each task takes the same time (same latency)

So why bother?

Parallelism vs concurrency

	Parallelism	Concurrency
Task origin	solution	problem
Control	developer	environment
Resource use	coordinated	competitive
Metric	latency	throughput
Abstraction	CPU cores	tasks
# of threads	# of cores	# of tasks

Parallelism

Concurrency

Task origin

solution

problem

Control

developer

environment

Resource use

coordinated

competitive

Metric

latency

throughput

Abstraction

CPU cores

tasks

# of threads

# of cores

# of tasks

Performance

When workload is not CPU-bound:

start waiting as early as possible
for as many tasks as possible

⇝ Virtual threads increase throughput:

when workload is not CPU-bound
when number of concurrent tasks is high

Server How-To

For servers:

request handling threads are started by web framework
frameworks will offer (easy) configuration options

We’re not there yet.

Spring Boot

Replace executors:

@Bean
public TomcatProtocolHandlerCustomizer<?>
		createExecutorForSyncCalls() {
	return handler -> handler.setExecutor(
			Executors.newVirtualThreadPerTaskExecutor());
}

@Bean
public AsyncTaskExecutor
		createExecutorForAsyncCalls() {
	return new TaskExecutorAdapter(
			Executors.newVirtualThreadPerTaskExecutor());
}

Quarkus

Annotate request handling method:

@GET
@Path("api")
@RunOnVirtualThread
public String handle() {
	// ...
}

(Requires --add-opens java.base/java.lang=ALL-UNNAMED.)

📝 JEP 444
📝 On Parallelism and Concurrency
🎥 Java 21 new feature: Virtual Threads (Sep 2023)

Virtual Threads

Go forth and multiply (your threads)

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Preparing your code

Virtual threads:

always work correctly
may not scale perfectly

Code changes can improve scalability
(and maintainability, debuggability, observability).

Avoid thread pools

Only pool expensive resources
but virtual threads are cheap.

⇝ Replace thread pools (for concurrency),
with virtual threads plus, e.g., semaphores.

With thread pools

// limits concurrent queries but pools 👎🏾
private static final ExecutorService DB_POOL =
	Executors.newFixedThreadPool(16);

public <T> Future<T> queryDatabase(Callable<T> query) {
	return DB_POOL.submit(query);
}

With semaphore

// limits concurrent queries without pool 👍🏾
private static final Semaphore DB_SEMAPHORE =
	new Semaphore(16);

public <T> T queryDatabase(Callable<T> query)
		throws Exception {
	DB_SEMAPHORE.acquire();
	try {
		return query.call();
	} finally {
		DB_SEMAPHORE.release();
	}
}

Where are the virtual threads? ⇝ Later.

Caveats

To understand virtual thread caveats
we need to understand how they work.

(Also, it’s very interesting.)

Under the hood

The Java runtime manages virtual threads:

runs them on a pool of carrier threads
on blocking call:
- internally calls non-blocking operation
- unmounts from carrier thread!
when call returns:
- mounts to (other) carrier thread
- continues

Stack chunks

A virtual thread stack:

when waiting, is stored on heap (stack chunk objects)
when continuing, is lazily streamed to stack

This keeps switching cheap.

The simple web request

Remember the hypothetical request:

interpret request
query database (blocks)
process data for response

In a virtual thread:

runtime submits task to carrier thread pool
when 2. blocks, virtual thread unmounts
runtime hands carrier thread back to pool
when 2. unblocks, runtime resubmits task
virtual thread mounts and continues with 3.

Compatibility

Virtual threads work correctly with everything:

all blocking operations
synchronized
Thread, currentThread, etc.
thread interruption
thread-locals
native code

But not all scale perfectly.

Caveat #1: capture

Despite lots of internal rework (e.g. JEPs 353, 373)
not all blocking operations unmount.

Some capture platform thread:

Object::wait
file I/O (⇝ io_uring)

⇝ Compensated by temporarily growing carrier pool.

⚠️ Problematic when capturing operations dominate.

Caveat #2: pinning

Some operations pin (operations don’t unmount):

native method call (JNI)
foreign function call (FFM)
synchronized block (for now)

⇝ No compensation

⚠️ Problematic when:

pinning is frequent
contains blocking operations

Avoid long-running pins

If possible:

avoid pinning operations
remove blocking operations
from pinning code sections.

With synchronization

// guarantees sequential access, but pins (for now) 👎🏾
public synchronized String accessResource() {
	return access();
}

With lock

// guarantees sequential access without pinning 👍🏾
private static final ReentrantLock LOCK =
	new ReentrantLock();

public String accessResource() {
	// lock guarantees sequential access
	LOCK.lock();
	try {
		return access();
	} finally {
		LOCK.unlock();
	}
}

Caveat #3: thread locals

Thread-locals can hinder scalability:

can be inherited
to keep them thread-local,
values are copied
can occupy lots of memory

(There are also API shortcomings.)

⇝ Refactor to scoped values (JEP 446).

With thread-local

// copies value for each inheriting thread 👎🏾
static final ThreadLocal<Principal> PRINCIPAL =
	new ThreadLocal<>();

public void serve(Request request, Response response) {
	var level = request.isAdmin() ? ADMIN : GUEST;
	var principal = new Principal(level);
	PRINCIPAL.set(principal);
	Application.handle(request, response);
}

With scoped value

// immutable, so no copies needed 👍🏾
static final ScopedValue<Principal> PRINCIPAL =
	new ScopedValue<>();

public void serve(Request request, Response response) {
	var level = request.isAdmin() ? ADMIN : GUEST;
	var principal = new Principal(level);
	ScopedValue
		.where(PRINCIPAL, principal)
		.run(() -> Application
			.handle(request, response));
}

Preparing your code:

Most importantly:

replace thread pools with semaphores

Also helpful:

remove long-running I/O from pinned sections
replace thread-locals with scoped values
replace synchronized with locks

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

String composition

Composing strings in Java is cumbersome:

String property = "last_name";
String value = "Doe";

// concatenation
String query =
	"SELECT * FROM Person p WHERE p."
		+ property + " = '" + value + "'";
// formatting
String query =
	"SELECT * FROM Person p WHERE p.%s = '%s'"
		.formatted(property, value);

Comes with free SQL injection! 😳

String interpolation

Why not?

// (fictional syntax!)
String query =
	"SELECT * FROM Person p "
		+ "WHERE p.\{property} = '\{value}'";

Also comes with free SQL injection! 😳

String interpolation

SQL injections aren’t the only concern.

These also need validation and sanitization:

HTML/XML
JSON
YAML
…

All follow format-specific rules.

String templates

[Preview in Java 21 — JEP 430]

String query = STR."""
	SELECT * FROM Person p
	WHERE p.\{property} = '\{value}'
	""";

Template expression ingredients:

template with embedded expressions
~> StringTemplate
template processor (e.g. STR):
transforms StringTemplate into String*

Template procesor STR

String form = STR."""
	Desc     Unit   Qty   Amount
	\{desc} $\{price} \{qty} $\{price * qty}

	Subtotal  $\{price * qty}
	Tax       $\{price * qty * tax}
	Total     $\{price * qty * (1.0 + tax)}
	""";

Desc     Unit   Qty   Amount
hammer   $7.88  3     $23.64

Subtotal  $23.64
Tax       $3.546
Total     $27.186

Template processor FMT

String form = FMT."""
	Desc        Unit      Qty   Amount
	%-10s\{desc} $%5.2f\{price} %5d\{qty} $%5.2f\{price * qty}

	Subtotal  $%5.2f\{price * qty}
	Tax       $%5.2f\{price * qty * tax}
	Total     $%5.2f\{price * qty * (1.0 + tax)}
	""";

Desc        Unit      Qty   Amount
hammer      $ 7.88      3   $23.64

Subtotal  $23.64
Tax       $ 3.55
Total     $27.19

Why strings?

Often, strings are just exchange format, e.g.:

start with: String + values
validate / sanitize (i.e. parse)
dumb down to: String 🤔
parse to: JSONObject, Statement, …

Why the detour?

Custom templating

STR is a singleton instance of
a Processor implementation:

public interface Processor<RESULT, EX> {
	RESULT process(StringTemplate s) throws EX;
}

RESULT can be of any type!

Custom templating

// prevents SQL injections
Statement query = SQL."""
	SELECT * FROM Person p
	WHERE p.\{property} = '\{value}'
	""";

// validates & escapes JSON
JSONObject doc = JSON."""
	{
		"name": "\{name}",
		"year": "\{bday.getYear()}"
	}
	""";

Summary

String templates:

simplify string concatenation
enable domain-specific processing
incentivize the "right way"

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Order

Collections with order and indexed access:

List

Collections with order without indexed access:

SortedSet (sort order)
Deque (insertion order)
LinkedHashSet (insertion order)
and more

Sequence

New interfaces capture the concept of order:

SequencedCollection
SequencedSet
SequencedMap

Use as parameter or return type
and enjoy new methods.

Get First

Getting the first element:

list.get(0)
sortedSet.first()
deque.getFirst()
linkedHashSet.iterator().next()

Now for all:

sequencedCollection.getFirst()

Get Last

Getting the last element:

list.get(list.size() - 1)
sortedSet.last()
deque.getLast()
linkedHashSet.🤷🏾‍♂️()

Now for all:

sequencedCollection.getLast()

Reversing Order

list.listIterator() ⇝ ListIterator
navigableSet.descendingSet() ⇝ NavigableSet (view)
deque.descendingIterator() ⇝ Iterator
linkedHashSet.🤷🏾‍♂️()

Now for all:

sequencedCollection.reversed() ⇝ SequencedCollection (view)

📝 JEP 431
🎥 Java 21’s New (Sequenced) Collections (Mar 2023)

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Starting (With) Java

Java 21 makes life easier
for new (Java) developers.

Why Do We Care?

We all know Java, IDEs, build tools, etc.

do we all?
what about your kids?
what about students?
what about the frontend dev?
what about ML/AI folks?

Java needs to be approachable!
Java needs an on-ramp for new (Java) developers!

Running Java

To write and run a simple Java program, you need:

a JDK
an editor (IDE?)
javac (build tool? IDE?)
java (IDE?)
some Java code

Writing Java

Minimal Java code:

public class Main {
	public static void main(String[] args) {
		System.out.println("Hello, World!");
	}
}

visibility
classes & methods
static vs instance
returns & parameters
statements & arguments

Approachability

That’s a lot of tools and concepts!

Java is great for large-scale development:

detailed toolchain
refined programming model

This make it less approachable.

Let’s change that!

jshell

Java 9 added jshell:

all you need:
- tools: JDK, jshell
- concepts: statements & arguments
but:
- not great for beginners (IMO)
- no progression

More is needed.

Single-file Execution

Java 11 added single-file execution (JEP 330):

java Prog.java

removed: javac
but: no progression

Much better for beginners,
but just a section of an on-ramp.

On-Ramp

Expand single-file execution in two directions:

simplify code: reduce required Java concepts
ease progression: run multiple files with java

Simpler Code

Remove requirement of:

String[] args parameter
main being static
main being public
the class itself

// all the code in Prog.java
void main() {
	System.out.println("Hello, World!");
}

[Preview in Java 21 — JEP 445]

Running Multiple Files

Say you have a folder:

MyFirstJava
 ├─ Prog.java
 ├─ Helper.java
 └─ Lib
     └─ library.jar

Run with:

java -cp 'Lib/*' Prog.java

(JEP draft)

Progression

Natural progression:

start with main()
need arguments? ⇝ add String[] args
need to organize code? ⇝ add methods
need shared state? ⇝ add fields
need more functionality? ⇝ explore JDK APIs
even more? ⇝ explore simple libraries
need more structure? ⇝ split into multiple files
even more ⇝ use visibility & packages

Doesn’t even have to be that order!

Summary

Java’s strengths for large-scale development
make it less approachable:

detailed toolchain
refined programming model

There are new features that:

make it easier to start
allow gradual progression
entice the future dev generation

Java 21 💣💥

Pattern Matching

Data-Oriented Programming

Virtual Threads

Preparing For Virtual Threads

String Templates

Sequenced Collections

On-Ramp

GenZGC

Generational ZGC

Compared to other GCs, ZGC:

optimizes ultra-low pause times
can have higher memory footprint or higher CPU usage

In Java 21, ZGC becomes generational.

Generational Hypothesis

most objects die young
those who don’t, grow (very) old

GCs can make use of this by tracking
young and old generations.

ZGC didn’t do this, but can do it now with:
-XX:+UseZGC -XX:+ZGenerational

Some Benchmarks

A Cassandra 4 benchmark of ZGC vs GenZGC showed:

4x throughput with a fixed heap or
1/4x heap size with stable throughput

(Probably not representative but very promising.)

So long…

37% off with
code fccparlog

bit.ly/the-jms

Slides at slides.nipafx.dev
⇜ Get my book!

Follow Nicolai

nipafx.dev
/nipafx

Follow Java

inside.java // dev.java
/java // /openjdk

Image Credits

modern colors: Pawel Czerwinski, Unsplash license
random generator API diagrams:
Nicolai Parlog (CC-BY-NC 4.0)
question-mark: Milos Milosevic, CC-BY 2.0