Why Don’t They Just…​ ?!

Nicolai Parlog

nipafx.dev / @nipafx

Java

youtube.com/java / @java

Developer Advocate

Java Team at Oracle

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

Slides at slides.nipafx.dev/just

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

What We Want

Being able to add fields to records:

record Customer(String name) {

	// no accessor
	private final String id;

	// [...]

}

Why Do We Want That?

To benefit from encapsulation and
boilerplate reduction?

The thing is:
Records are about neither.

Record Semantics

Core semantics (source):

[Records] are classes that act as transparent carriers for immutable data.

Transparency inspires this motto for records (source):

The API for a record models the state, the whole state, and nothing but the state.

Restrictions

Transparency implies restrictions:

  • an accessor for each component
    (or the API doesn’t model the state)

  • an accessible constructor with
    one parameter per component
    (or the API doesn’t model the state)

  • no additional fields
    (or the API doesn’t model the whole state)

  • no class inheritance
    (or the API doesn’t model the whole state)

Ok, But Why?

Records were designed to be nominal tuples:

  • nominal means they have names
    (as all types in Java do)

  • tuples are a mathematical concept
    (pairs, triples, etc. are tuples)

For a type to be a tuple,
it must conform to the restrictions.

(A bit more on the math.)

Ok, But Why?!

If records are tuples,
their deconstruction becomes:

  • easy

  • lossless

⇝ We can do a few cool things!

Destructuring

JEP 405 proposes record patterns
that deconstruct records:

if (range instanceof Range(int low, int high)
		&& high < low)
	return new Range(high, low);

Transparency ⇝ no hidden state was lost.

With Blocks

Future Java may (!) introduce with blocks:

Range range = new Range(5, 10);
// SYNTAX IS MADE UP!
Range newRange = range with {
	low = 0;
};
// range: [5; 10]
// newRange: [0; 10]

Transparency ⇝ no hidden state was lost.

Serialization

Transparency makes record (de)serialization:

  • easier to implement and maintain in JDK

  • easier to use and maintain in your code

  • safer

  • faster

(More in Inside Java Podcast, episode 14.)

Summary

  • records want to be (nominal) tuples

  • that requires transparency

  • transparency requires no additional fields

  • transparency affords additional features
    like destructuring, with blocks, better serialization

Higher-Level Summary

It makes sense to introduce someting that has restricions
if those restrictions enable other features.

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

What We Want

Streams that work well with checked exceptions:

int compute(String word) throws IOException;
int supercharge(int val) throws InterruptedException;

try {
	List<Integer> supercharged = Stream
		.of("foo", "bar")
		.map(this::compute)
		.map(this::supercharge)
		.toList();
} catch (IOException | InterruptedException ex) {
	// ...
}

Let’s think this through

We need:

  • an exception throwing Function

  • changes to Stream methods,
    so they throw exceptions

Attempt #1

Let map throw Exception:

interface Function<IN, OUT> {
	OUT apply(IN input) throws Exception;
}

interface Stream<E> {
	<OUT> Stream<OUT> map(Function<E, OUT> f)
	throws Exception;

	List<E> toList();
}

Attempt #1

That’s not correct:

  • streams are lazy

  • map does not apply the function

  • the terminal operation does

⇝ Terminal ops have to declare throws.

Attempt #2

Let terminal operation throw Exception:

interface Function<IN, OUT> {
	OUT apply(IN input) throws Exception;
}

interface Stream<E> {
	<OUT> Stream<OUT> map(Function<E, OUT> f)

	List<E> toList() throws Exception;
}

Attempt #2

try {
	List<Integer> supercharged = Stream
		.of("foo", "bar")
		.map(this::compute)
		.map(this::supercharge)
		.toList();
} catch (Exception ex) {
	// which exceptions?
}

List<Integer> supercharged = Stream
	.of("foo", "bar")
	.map(word -> word)
	// compile error: unhandled exception
	.toList();

Attempt #2

That’s awful:

  • compiler doesn’t know exception type

  • we have to catch Exception and
    figure the rest out ourselves

  • we always have to catch Exception

⇝ Need more specific type than Exception.

Attempt #3

Capture exception in generic parameter,
and let terminal operation throw that:

interface Function<IN, OUT, EX extends Exception> {
	OUT apply(IN input) throws EX;
}

interface Stream<E, EX extends Exception> {
	<OUT, F_EX extends Exception>
	Stream<OUT, F_EX> map(Function<E, OUT, F_EX> f);

	List<E> toList() throws EX;
}

Attempt #3

try {
	List<Integer> supercharged = Stream
		.of("foo", "bar")
		// ⇝ Stream<Integer, IOException>
		.map(this::compute)
		// ⇝ Stream<Integer, InterruptedException>
		.map(this::supercharge)
		.toList();
} catch (InterruptedException ex) {
	// good
} catch (IOException ex) {
	// isn't declared, so can't be caught
	// ⇝ compile error
}

Attempt #3

That’s not correct:

  • only last function’s exception type is captured

  • other checked exceptions can’t be caught

⇝ Need to capture all exception types.

Attempt #4

Merge exceptions in generic parameter,
and let terminal operation throw that:

static <
	IN, OUT,
	NEW_EX extends Exception,
	STREAM_EX extends NEW_EX,
	F_EX extends NEW_EX>
Stream<OUT, NEW_EX> map(
		Stream<IN, STREAM_EX> stream,
		Function<IN, OUT, F_EX> f) {
	// ...
}

Attempt #4

try {
	List<Integer> supercharged = Stream
		// ⇝ Stream<Integer, IOException>
		.map(
			// ⇝ Stream<Integer, FileNotFoundException>
			Stream.map(
				// ⇝ Stream<String, RuntimeException>
				Stream.of("foo", "bar"),
				this::throwsFileNotFoundException),
			this::throwsZipException)
		.toList();
} catch (IOException ex) {
	// nice
}

Attempt #4

try {
	List<Integer> supercharged = Stream
		// ⇝ Stream<Integer, Exception>
		.map(
			// ⇝ Stream<Integer, IOException>
			Stream.map(
				// ⇝ Stream<String, RuntimeException>
				Stream.of("foo", "bar"),
				this::compute),
			this::supercharge)
		.toList();
} catch (Exception ex) {
	// argh!
}

Attempt #4

That’s not good:

  • common case sucks:
    Stream<SomeThing, RuntimeException>

  • map as static methods sucks

  • catching Exception sucks
    (exceptions don’t generalize well)

⇝ Need to keep exception types distinct.

Attempt #5

Create multiple Stream interfaces
that differ by number of exceptions:

interface Stream<E> {

	<OUT, F_EX extends Exception>
	StreamEx1<OUT, F_EX>
	map(Function<E, OUT, F_EX> f);

	List<E> toList();
}

Attempt #5

interface StreamEx1<E, EX extends Exception> {

	<OUT, F_EX extends Exception>
	StreamEx2<OUT, EX, F_EX>
	map(Function<E, OUT, F_EX> f);

	List<E> toList() throws EX;
}

Attempt #5

interface StreamEx2<E,
		EX0 extends Exception,
		EX1 extends Exception> {

	<OUT, F_EX extends Exception>
	StreamExN<OUT> map(Function<E, OUT, F_EX> f);

	List<E> toList() throws EX0, EX1;
}

Attempt #5

interface StreamExN<E> {

	<OUT, F_EX extends Exception>
	StreamExN<OUT> map(Function<E, OUT, F_EX> f);

	List<E> toList() throws Exception;
}

Attempt #5

try {
	List<Integer> supercharged = Stream
		// ⇝ Stream<String>
		.of("foo", "bar")
		// ⇝ StreamEx1<Integer, IOException>
		.map(this::compute)
		// ⇝ StreamEx2<Integer, IOException,
		//             InterruptedException>
		.map(this::supercharge)
		.toList();
} catch (IOException ex) {
	// good
} catch (InterruptedException ex) {
	// great
}

Attempt #5

That’s correct and usable!
(Which is a first.)

  • but it leads to many additional interfaces

  • together with primitive specializations
    ⇝ combinatorial explosion 💣²

  • functions may declare multiple exceptions
    ⇝ need multiple overloads for all operations
    ⇝ combinatorial explosion 💣³

⇝ Need variadic generics.

Attempt #6

Put all exceptions into one type parameter:

interface Function<
		IN, OUT, EXs... extends Exception> {
	OUT apply(IN input) throws EXs;
}

interface Stream<E, EXs... extends Exception> {
	<OUT, F_EXs... extends Exception>
	Stream<OUT, EXs | F_EX>
	map(Function<E, OUT, F_EXs> f);

	List<E> toList() throws EXs;
}

Attempt #6

try {
	List<Integer> supercharged = Stream
		// ⇝ Stream<String>
		.of("foo", "bar")
		// ⇝ Stream<Integer, IOException>
		.map(this::compute)
		// ⇝ Stream<Integer, IOException,
		//          InterruptedException>
		.map(this::supercharge)
		.toList();
} catch (InterruptedException ex) {
	// good
} catch (IOException ex) {
	// great
}

Attempt #6

All around great with one downside:

  • Java doesn’t allow that

  • neither Function nor Stream compiles

😕

Attempt #7

Screw everything, just handle errors via return type:

List<Integer> supercharged = Stream
	.of("foo", "bar")
	// ⇝ Stream<Try<Integer>>
	.map(this::compute)
	// ⇝ Stream<Try<Integer>>
	.map(this::supercharge)
	.toList();

Already works today.
More on that.

Summary

  • streams' laziness split in two:

    • passing a throwing function (intermediate op)

    • handling the exception (terminal op)

  • for classic try-catch:

    • needs generics to carry exception type(s) forward

    • there’s no good solution in today’s Java

  • but there are acceptable alternatives

Higher-Level Summary

It doesn’t make sense to introduce someting that:

  • has serious shortcomings in practice

  • prevents a much better solution down the road

  • particuarly if an acceptable alternative exists

Just because something isn’t perfect,
doesn’t mean every (partial) fix should be implemented.

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

What We Want

Immutable collections:

  • interfaces without mutating methods and

  • a guarantee that no mutations occur

As opposed to unmodifiable collections:

  • interfaces without mutating methods or

  • mutating methods that throw exceptions

Let’s think this through

(On the example of lists.)

Assume we had ImmutableList:

  • like List today

  • but without any mutation

⇝ How is it related to List?

ImmutableList super List

immutable collections mutable extends immutable

Good:

  • ImmutableList has no mutating methods

ImmutableList super List

Bad:

List<Agent> agents = new ArrayList<>();
// compiles because `List` extends `ImmutableList`
ImmutableList<Agent> section4 = agents;
// prints nothing
section4.forEach(System.out::println);

// now lets mutate `section4`
agents.add(new Agent("Motoko"));
// prints "Motoko" - how did she get in here?!
section4.forEach(System.out::println);

List super ImmutableList

immutable collections immutable extends mutable

Good:

  • ImmutableList isn’t extended
    and thus actually immutable

List super ImmutableList

Bad:

  • ImmutableList has mutating methods that throw

  • ImmutableList can be passed as List
    ⇝ it’s reasonable to assume that mutation is allowed
    ⇝ runtime exceptions

Only really work well locally,
i.e. not across API boundaries.

Immutability as Absence

Easy to mistake immutability as an absence:

  • take a List

  • remove mutating methods

  • profit

No!

That just gives you UnmodifiableList!

Immutability as a Feature

An UnmodifiableList offers no mutating methods,
without making immutability guarantees.

There are two things to add:

  • make it mutable by adding the according methods

  • make it immutable by adding the according guarantees

(Im)Mutability Clash

Immutability is not an absence of mutation, it’s a guarantee there won’t be mutation

— Your's truly

(Im)mutability is inherited by subtypes!

If one of two types extends the other,
one of them contains both properties.
⇝ 💣

How to make this work

Solution:

  • don’t make the two lists inherit one another

  • instead, have a new supertype for both

immutable collections both extend unmodifiable

In Practice

interface SecretService {

	void payAgents(UnmodifiableList<Agent> agents);
	void sendOnMission(ImmutableList<Agent> agents);
	void downtime(List<Agent> agents);

	UnmodifiableList<Agent> teamRoster();
	ImmutableList<Agent> teamOnMission();
	List<Agent> team();

}

In Practice

But such code already exists
and often looks like this:

interface SecretService {

	void payAgents(List<Agent> agents);
	void sendOnMission(List<Agent> agents);
	void downtime(List<Agent> agents);

	List<Agent> teamRoster();
	List<Agent> teamOnMission();
	List<Agent> team();

}

Retrofit new hierarchy

To benefit from new types,
we need to use them (duh!), but:

  • List to ImmutableList
    is source-incompatible ⇝ rewrite

  • return type List to UnmodifiableList
    is source-incompatible ⇝ rewrite

  • parameter type List to UnmodifiableList
    is bytecode-incompatible ⇝ recompile

Imagine this for the JDK,
all libraries, frameworks, and your code!

Retrofit new hierarchy

Alternative:

  • duplicate existing methods
    with a new name and new types

  • deprecate old variants

Huge task that takes forever!

Summary

  • immutable collection types are a great thing to have

  • proper implementations of List and ImmutableList
    can never extend one another

  • this complicates their introduction into existing APIs

  • requires rewriting and recompiling code
    across the entire Java ecosystem

Higher-Level Summary

It doesn’t make sense to introduce someting that:

  • requires rewriting/recompiling the world

  • splits the ecosystem into old and new

  • for incremental benefits

Just because something would be nice to have
doesn’t mean it’s nice to get.

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

What We Want

The ability to easily chain method calls
on possibly null instances:

// argh!
String street = null;
if (customer != null && customer.address() != null)
	street = customer.address().street();

// ugh
Optional<String> streetOpt = customerOpt
	.flatMap(Customer::address)
	.flatMap(Address::street);

// yay (?)
String street = customer?.address()?.street();

Let’s think this through

How does null come into the world?

  • intentional: allowed absence of a value

  • accidental:

    • forbidden absense of a value

    • uninitialized variable

    • botched error handling

    • etc.

Intentional vs Accidental

How to fix a null-related bug
(e.g. NullPointerException)?

Bad

add a null check and move on

Good

  • follow null back to its source

  • determine whether its intentional or accidental

  • fix code accordingly

Intentional vs Accidental

The problem with null:

  • is not avoiding the exception

  • it’s figuring out what null means

What’s more work?
What would ?. make easier?

Sidenote: Optional

That’s why I like Optional so much:

  • use Optional where absence is allowed

  • make null always illegal

⇝ Bake intentional vs accidental into the code.

?. Elsewhere

But doesn’t ?. work elsewhere?

Indeed! E.g. Kotlin:

var street = customer?.address?.street

Why does it work there?

?. Elsewhere

Kotlin has more than ?.:

// this is a compile error
var street: String = customer?.address?.street
// need to declare `street` as nullable
var street: String? = customer?.address?.street

⇝ Type system needs to know about null!

?. Elsewhere

Together:

  • ?.:

    • makes null handling easier

    • proliferates null

  • nullable types:

    • require null handling

    • minimize accidental null

Summary

  • the challenge with null is determining its meaning

  • ?. doesn’t help with that

  • instead it makes the wrong choice easier

  • unless nullable types hem them in

Higher-Level Summary

It doesn’t make sense to introduce someting that:

  • makes the wrong action easier

  • requires non-existent features to work well

Just because something works well in one language
doesn’t mean it’ll work well in another.

Why don’t they just…​

... Let Us Add Fields To Records?!
... Let Streams Handle Checked Exceptions?!
... Introduce Immutable Collections?!
... Introduce ?. For null-safe Member Selection?!
... Introduce Nullable Types?!

What We Want

A way to explicitly (dis)allow null.

Given a type Foo:

  • Foo! excludes null

  • Foo? allows null

What We Want

String? print(String! message) {
	// [...]
}

// compile errors
String! message = null;
print(null);
String! printed = print("foo");

News

Good

Creating nullable types is relatively easy.

Bad

Adopting them is a lot of work.

Adopting Nullable Types

Third case:

  • Foo! excludes null

  • Foo? allows null

  • Foo:

    • legacy type

    • makes no assertion re null

    • must be treated like Foo?

All existing declarations use legacy types.

Adopting Nullable Types

All existing declarations need to be updated.

This is like the introduction of generics, but:

  • order of magnitude more warnings

  • order of magnitude more work

  • harder work

    • heterogeneous containers are very rare

    • null isn’t

Summary

  • creating nullable types is relatively easy

  • adopting them is a lot of work

Like many forms of improved type checking, nullable types is one of those things you can do at the beginning, but is very hard to graft onto an existing ecosystem.

— Brian Goetz

Higher-Level Summary

It doesn’t make sense to introduce someting that:

  • requires rewriting/recompiling the world

  • splits the ecosystem into old and new

  • for incremental benefits

Just because something would be nice to have
doesn’t mean it’s nice to get.

Summary

"Not a bad feature" is not enough!

  • not every problem needs an immediate solution
    (particularly not if it prevents a better one later)

  • not every proposal is a solution
    just because it works in other languages

  • not every solution can be prioritized

  • not every solution can be retrofitted

  • not every retrofit may be worth it
    (particularly if it splits the ecosystem)

So long…​

37% off with
code fccparlog

bit.ly/the-jms

More

Slides at slides.nipafx.dev
⇜ Get my book!

Follow Nicolai

nipafx.dev
/nipafx

Follow Java

inside.java // dev.java
/java    //    /openjdk

Image Credits