Expert Java 8

Lambdas, Streams, Optionals
For Afficionados

Developer Advocate

Java Team at Oracle

Public Service Announcement

  • you need to be proficient with Java 8

  • some of what follows is merely my opinion

  • slides at slides.nipafx.dev

Streams

In APIs
Exception Handling
Finding Elements
Performance

Streams

In APIs
Exception Handling
Finding Elements
Performance

Using streams in your APIs.

Interesting Stream Properties

  • streams are unmodifiable

  • streams are easy to transform

  • streams are lazy

⇝ Good way to enrich data in layers.

Enrichment Example

Map<String, User> userById;
Map<String, Address> addrById;

Stream<User> users() {
	return userById.values().stream();
}

Stream<UserAddress> userAddresses() {
	return users()
		.map(user -> UserAddress.of(
				user, addrById.get(user.id())));
}

Enrichment Example

Map<Address, List<Order>> ordersByAddr;

Stream<Order> ordersFor(UserAdress userAddr) {
	return ordersByAddr
		.get(userAddr.address()).stream();
}

Stream<Delivery> deliveriesFor(UserAdress userAddr) {
	return ordersFor(userAddr)
		.map(order -> Delivery.of(userAddr, order));
}

Stream<Delivery> deliveries() {
	return userAddresses().flatMap(this::deliveriesFor);
}

Guidelines

  • streams can only be traversed once
    ⇝ ideally, returned streams can be recreated endlessly

  • during traversal collection must not be mutated
    ⇝ stick to layers instead of cycles
    (can be harder than it sounds)

  • decide carefully where to call parallel()
    ⇝ close to terminal operation is a good default

Accepting Streams

Returning streams is ok.

What about passing them as parameters?

  • streams can only be traversed once!

  • caller must assume stream is traversed

  • works for obvious consumer functions

  • possibly a transformed stream can be returned

Stream Param Example

// consuming the passed stream
void addUsers(Stream<User> users) {
	users.forEach(this.users::add)
}

// transforming the passed stream
Stream<UserAddress> userAddresses(Stream<User> users) {
	return users
		.map(user -> UserAddress.of(
				user, addrById.get(user.id())));
}

Reflection On APIs

Returning Streams

  • returning streams is great
    (unmodifiable but transformable)

  • preferably if streams can be recreated

  • can be used to gradually enrich data

Reflection On APIs

Passing Streams

  • caller must assume stream is traversed

  • works for obvious consumers

  • transformations can be hard to track

  • never return a traversed stream
    (obvious, right?!)

Reflection On APIs

But Look Out

  • streams can only be traversed once

  • no mutation during traversal

  • don’t make chains too long or
    debuggability suffers

Streams

In APIs
Exception Handling
Finding Elements
Performance

Handling checked exceptions in Streams.

Setting the Scene

Stream<User> parse(Stream<String> strings) {
	// compile error:
	// "Unhandled exception ParseException"
	return strings.map(this::parse);
}

User parse(String userString) throws ParseException {
	// ...
}

Which options do we have?

Try in Lambda

Stream<User> parse(Stream<String> strings) {
	return strings
		.map(string -> { try {
				return parse(string);
			} catch (ParseException ex) {
				return null;
			}})
		.filter(Objects::nonNull);
}
  • super ugly

  • requires extra clean-up step

  • handling exception locally can be hard

  • troublesome elements "disappear"

Try in Method

Stream<User> parse(Stream<String> strings) {
	return strings
			.map(this::tryParse)
			.filter(Objects::nonNull);
}

private User tryParse(String string) {
	try { return parse(string); }
	catch (ParseException ex) { return null; }
}
  • somewhat ugly

  • requires extra clean-up step ("far away")

  • handling exception locally can be hard

  • troublesome elements "disappear"

Sneaky Throws

How to "trick the compiler":

static Function<T, R> hideException(
		CheckedFunction<T, R, Exception> function) {
	return element -> {
		try {
			return function.apply(element);
		} catch (Exception ex) {
			return sneakyThrow(ex);
		}
	};
}

// sneakyThrow does shenanigans with generics
// and unchecked casts to "confuse the compiler"

Sneaky Throws

Stream<User> parse(Stream<String> strings) {
	return strings
		.map(Util.hideException(this::parse));
}
  • very surprising (hides a bomb in the stream!)

  • stream executor has to handle exception

  • can’t try-catch(ParseException) because
    checked exceptions need to be declared

  • exception aborts stream pipeline

Please never do that!

Wrap in Unchecked

Another Util method:

static Function<T, R> uncheckException(
		CheckedFunction<T, R, Exception> function) {
	return element -> {
		try {
			return function.apply(element);
		} catch (Exception ex) {
			// add special cases for RuntimeException,
			// InterruptedException, etc.
			throw new IllegalArgumentException(
				element, ex);
		}
	};
}

Wrap in Unchecked

Stream<User> parse(Stream<String> strings) {
	return strings
		.map(Util.uncheckException(this::parse));
}
  • stream executor has to handle exception

  • exception aborts stream pipeline

Remove Trouble

Another Util method:

static Function<T, Optional<R>> wrapOptional(
		CheckedFunction<T, R, Exception> function) {
	return element -> {
		try {
			return Optional.of(
				function.apply(element));
		} catch (Exception ex) {
			return Optional.empty();
		}
	};
}

Remove Trouble

Stream<User> parse(Stream<String> strings) {
	return strings
		.map(Util.wrapOptional(this::parse))
		// Java 9: .flatMap(Optional::stream)
		.filter(Optional::isPresent)
		.map(Optional::get);
}
  • requires extra clean-up step
    (at least supported by compiler)

  • troublesome elements "disappear"

Expose With Try

Try<T> is similar to Optional:

  • has two states (error or success)

  • allows to process them with functions

  • parameterized in type of success result

Another Util method:

static Function<T, Try<R>> wrapTry(
		CheckedFunction<T, R, Exception> function) {
	return element -> Try.of(
		() -> function.apply(element));
}

Expose With Try

Stream<Try<User>> parse(Stream<String> strings) {
	return strings
		.map(Util.wrapTry(this::parse));
}
  • requires external library (e.g. Vavr)

  • encodes possibility of failure in API

  • makes error available to caller

  • error is encoded as Exception/Throwable

Expose With Either

Either<L, R> is similar to Optional:

  • has two states (left or right)

  • allows to process them with functions

  • parameterized in types of left and right

  • if used for failure/success, exception goes left
    (by convention)

Expose With Either

Another Util method:

static Function<T, Either<EX, R>> wrapEither(
		CheckedFunction<T, R, EX> function) {
	return element -> {
		try {
			return Either.right(
				function.apply(element));
		} catch (Exception ex) {
			// add special cases for RuntimeException,
			// InterruptedException, etc.
			return Either.left((EX) ex);
		}
	};
}

Expose With Either

Stream<Either<ParseException, User>> parse(
		Stream<String> strings) {
	return strings
		.map(Util.wrapEither(this::parse));
}
  • requires external library (e.g. Vavr)

  • encodes possibility of failure in API

  • makes error available to caller

  • error has correct type

Reflection on Exceptions

  • don’t be smart and "trick the compiler"

  • return a clean stream, no null!

  • ideally, use types to express possibility of failure

Streams don’t cooperate well with checked exceptions.

See that as a chance to use functional concepts
for greater good of code base!

Streams

In APIs
Exception Handling
Finding Elements
Performance

Be careful how you find!

Finding First or Any

Stream::findFirst and findAny:

  • return an arbitrary element from the Stream

  • if stream has encounter order,
    findFirst returns first element

Often used after a filter.

Find Example

Optional<User> findUser(String id) {
	return users.stream()
		.filter(user -> user.id().equals(id))
		.findFirst();
}

Same as the loop:

Optional<User> findUser(String id) {
	for (User user : users)
		if (user.id().equals(id))
			return Optional.of(user);
	return Optional.empty();
}

Small Observation

I sometimes see the following:

  • code’s correctness depends on only
    one element passing the filter

  • but there are no additional checks

⇝ The easy solution might be the wrong one!

(Applies to the loop as well.)

Finding Only

Make sure there is only one element:

Optional<User> findUser(String id) {
	return users.stream()
		.filter(user -> user.id().equals(id))
		.reduce(toOnlyElement());
}

static BinaryOperator toOnlyElement() {
	return (element, otherElement) -> {
		throw new IllegalArgumentException();
	};
}

(Stream::collect is an alternative to reduce.)

Properties of Finding Only

Upsides:

  • guarantees correctness by failing fast

  • expresses intent

Downsides:

  • materializes entire stream

Reflection On Finding

If correctness depends on only one element
surviving an ad-hoc filter:

  • findFirst, findAny do not suffice

  • use a reducer or collector to assert uniqueness

  • comes with a performance penalty

Additional Sources

Stream

In APIs
Exception Handling
Finding Elements
Performance

Getting a gut feeling

Performance Caveat

Don’t get carried away with performance!

Before optimizing loops vs streams:

  • gather performance requirements

  • reliably measure performance

  • act when requirements are not met

  • determine that stream is at fault

Performance Caveat

Performance measurements are hard
and I’m not good at them!

  • concrete numbers apply only to my setup

  • focus on how they relate

  • even that differs across setups

If it’s important, benchmark yourself!

Use JMH for that.

Stream Performance

Simple benchmark:
find max in int[500_000]

// takes 0.123
int m = Integer.MIN_VALUE;
for (int i = 0; i < intArray.length; i++)
	if (intArray[i] > m)
		m = intArray[i];

// takes 0.599
Arrays.stream(intArray).max();

20% performance ⇝ streams suck 😱

Really, though?

Couple of things to note:

  • are 500'000 elements few or many?
    ⇝ vary number of elements

  • finding max is really cheap!
    ⇝ try other operations

  • arrays are rare; lists more common
    ⇝ use different collections

Let’s vary things a little.

Data structures

Throw a List<Integer> into the mix:

iterationarraylist

for

0.123

0.700

stream

0.599

3.272

⇝ That didn’t change much…​

Cost Per Operation

Benchmark more operations:

  • max: finding the maximum

  • sum: summing int (ignoring overflows)

  • arithmetic: handful of bit shifts and mults

  • string: convert to string, xor char by char

Cost Per Operation

--- Array ---

iterationmaxsumarithstring

for

0.123

0.186

4.405

49.533

stream

0.599

1.394

4.100

52.236

--- List ---

iterationmaxsumarithstring

for

0.700

0.714

4.099

49.943

stream

3.272

3.584

7.776

64.989

Cost Per Operation

Define Q as abstract cost of operation:

  • max and sum are very cheap

  • arithmetic is x3 to x20

  • string is another x10

⇝ Reasonably complex operations
dominate total run time.

Number Of Elements

From 500'000 to 50'000'000 elements:

structureopiterationspeedup

array

sum

stream

x6.62

list

max

stream

x1.16

list

sum

stream

x1.08

list

max

for

x0.89

array

sum

for

x0.88

array

max

for

x0.7

Number Of Elements

Define N as number of elements.

For larger N, these speed up:

  • more complex iteration mechanisms

  • over simple operations

⇝ This benefits streams.

Taken together

Streams are always slower than loops, but:

  • doesn’t matter if N*Q is small

  • gets better the larger N*Q gets

  • asymptotically approaching x1

⇝ Unless specific perf reqs are in the way,
streams are a good default.

Streams are easy to parallelize…​

Parallel Performance

Parallelization adds overhead:

  • decomposing the problem

  • task / thread management

  • composing the result

Only benefit: more resources!

(May be limited by memory bandwidth.)

Parallel Performance

Super simplified model
(remember N and Q):

N*Q should be five digits (at least)

⇝ Parallel streams are no good default!

Parallel Performance

For details on parallel performance:

Thinking in Parallel
by Stuart Marks and Brian Goetz at JavaOne 2016
(start at 25:20 for performance)

Reflection on Performance

  • stream overhead is real
    but almost disappears for relevant sizes

  • performance is rarely a reason against streams

  • consider parallel streams carefully

  • measure and compare to requirements!

Optional

Everybody's Favorite Bike Shed!

Usage Patterns
Value-Based Class
Not a Monad

Optional

Usage Patterns
Value-Based Class
Not a Monad

The Java community strongly disagrees
on how to best use Optional.

Some insights into the discussion…​

Basic Rules

First some basic rules:

  • never, ever, ever call get()/orElseThrow()
    without checking isPresent() first

  • prefer functional style
    (map, flatMap, ifPresent, orElse, …​)

  • make everyone setting Optional to null
    buy a round of drinks or wear a silly hat

Basic Rules

Nobody (?) wants to see …​

Optional.ofNullable(mango)
	.ifPresent(System.out::println);

... instead of …​

if (mango != null)
	System.out.println(mango);

Different Opinions

  • don’t use it unless
    absolutely necessary

  • use it as return value

  • use it everywhere

Don’t Use It!

Assumptions

  • API is verbose and invites misuse

  • makes stack traces harder to debug

  • not serializable

  • unsupported by various frameworks

  • dereferencing reduces performance

  • instances increase memory consumption

  • no benefits over explicit null handling

Don’t Use It!

Conclusions

  • Optional sucks

  • only use it if existing API returns it

  • unpack quickly!

(Mark Struberg, Stephen Connolly, Hugues Johnson)

Limited Return Value

Assumptions

  • was designed as a return value

  • not serializable

  • long-lived instances increase
    memory consumption

  • boxing method arguments is verbose

Limited Return Value

Conclusions

  • use as return value if
    returning null is error-prone

  • no instance variables

  • no method parameters

  • instances should generally be short-lived

(Stuart Marks, Brian Goetz)

⇝ This should be your default choice!

Return Value

Assumptions

  • returning null is always error-prone

  • rest as before

Conclusions

  • use as return value whenever
    value can be absent

  • rest as before

(Stephen Colebourne)

Use Everywhere!

Assumptions

  • using Optional instead of null
    lifts null-handling into the type system

  • makes any null an implementation error
    (great for debugging)

  • performance arguments can be discarded
    unless proven to be relevant

Use Everywhere!

Conclusions

  • avoid optionality through good design
    (good recommendation in general)

  • use Optional instead of null everywhere

  • consider providing overloads
    for optional method parameters

(Mario Fusco, me)

Use Everywhere!

Overload Example

String bar(Optional<String> drink) {
	return drink.map(this::bar)
			.orElseGet(this::bar);
}

String bar(String drink) { /* ... */ }

String bar() { /* ... */ }

Reflection on Usage

Whatever you decide:

  • pick my recommendation! :)

  • make it a team decision

  • put it into your code style

  • learn over time

Relaxing rules is easier
than making them stricter!

Additional Sources

Optional

Usage Patterns
Value-Based Class
Not a Monad

Optional implements a new "pattern"
that requires us to be careful with what we do.

Value-Based Class?

Did you RTFM?

This is a value-based class; use of identity-sensitive operations […​] on instances of Optional may have unpredictable results and should be avoided.

What does it mean?

Value Types In Future Java

A future Java will contain value types:

  • pass by value
    (copied when passed as params)

  • immutable

  • no identity

Very similar to today’s primitives.

No Identity?

Class instances have identity:

  • each new Integer(5) creates a new instance

  • they are not identical (!=, different locks, …​)

Value types will have no identity:

  • there are no two different int 5

  • only their value matters

But Isn’t This Java 8?

From value types to value-based classes:

  • value types require wrappers/boxes
    (just like primitives do today)

  • value-based classes might turn out
    wrapping value types

  • as an optimization the JVM will
    create and destroy them at will

⇝ Wrappers have identity but it is unstable

Identity Crisis

ZonedDateTime getLastLogin(User user);
void storeMessage(ZonedDateTime time, String message);

String lastLoginMessage(User user) {
	ZonedDateTime lastLogin = getLastLogin(user); (1)
	String message = "Was " + lastLogin;
	storeMessage(lastLogin, message); (2)
	return message;
}
  1. might return an instance or a value

  2. might receive an instance or a value

Requirements For VBC

declaration site
  • final and immutable

  • equals, hashCode, toString
    must only rely on instance state

  • …​

use site
  • no use of ==, identity hash code,
    locking, serialization

(None of this is checked by the JVM.)

VBC in Java

Java 8

java.util

Optional[Double, Long, Int]

java.time

Duration, Instant, Period,
Year, YearMonth, MonthDay,
Local…​, Offset…​, Zoned…​ ZoneId, ZoneOffset

java.time.chrono

HijrahDate, JapaneseDate, MinguaDate, ThaiBuddhistDate

VBC in Java

Java 9

java.lang

ProcessHandle, Runtime.Version

java.util

types returned by collection factory methods

Java 12

java.lang.constant

ConstantDesc, DynamicCallSiteDesc, DynamicConstantDesc

Reflection on VBC

With Optional and other value-based classes:

  • never rely on their identity

  • mainly no ==, locking, serialization

If this works out,
performance hit all but disappears!

Additional Sources

Optional

Usage Patterns
Value-Based Class
Not a Monad

Optional saves us from null
at the expense of breaking Monad Laws.

(No math, I promise!)

Left Identity

For a Monad, this should always be true:

Objects.equals(
	ofNullable(x).flatMap(f),
	f.apply(x));

But:

Function f = s -> of("mango")
Optional ofMap =
	ofNullable(null).flatMap(f);
Optional apply = f.apply(null);
// Optional[] != Optional["mango"]

Associativity

For a Monad, this should always be true:

Objects.equals(
	ofNullable(x).map(f).map(g),
	ofNullable(x).map(f.andThen(g)));

But:

Function f = s -> null;
Function g = s -> "mango";
Optional map = of("kiwi").map(f).map(g);
Optional then = of("kiwi").map(f.andThen(g));
// Optional[] != Optional["mango"]

Root Cause Analysis

  • Optional maps null to empty()

  • flatMap and map are not executed
    on empty optionals

  • the first occurrence of null/empty
    stops the chain of executions

So What?

  • refactoring can change
    which code gets executed

  • functions that can "recover" from null
    might not get executed

  • particularly error-prone when
    functions have side effects
    (they generally should not, but it happens)

Reflection on Monads

  • be aware that Optional is no well-behaved monad

  • see it as a way to avoid handling null

  • be aware that refactoring can cause problems
    if null was special cased

Additional Sources

Default Methods

Fluent Decorators
Interface Evolution

Default Methods

Fluent Decorators
Interface Evolution

Fluent implementation of
the decorator pattern

Decorator Pattern

decorator
Component component = /*...*/;
Component decorated =
	new SomeDecorator(
		new AnotherDecorator(component));

Fluent Decorators

With lambda expressions and default methods
we can apply decorators fluently:

Component component = /*...*/;
Component decorated = DecoratingComponent.from(component)
	.some() // applies `SomeDecorator`
	.another("param") // applies `AnotherDecorator`
	.decorate(YetAnotherDecorator::new);

How?

Fluent Decorators

decorator 8

The DecoratingComponent interface:

  • extends Component

  • is implemented by all decorators

  • offers methods that wrap this
    in decorator and return it

Decorating Component

interface DecoratingC extends Component {

	static DecoratingC from(Component component) {
		return new DecoratingComponent() {
			// implement by forwarding to component
		};
	}

	default DecoratingC decorate(
		Function<DecoratingC, DecoratingC> decorator) {
		return decorator.apply(this);
	}

}

Generic Decoration

This generic decoration
allows chains like the following:

Component component = /*...*/;
Component decorated = DecoratingComponent.from(component)
	.decorate(SomeDecorator::new);
	.decorate(c -> new AnotherDecorator(c, "param"));
	.decorate(YetAnotherDecorator::new);

Specific Decoration

interface DecoratingC extends Component {

	default DecoratingC some()
		return decorate(SomeDecorator::new);
	}

	default DecoratingC another(String s)
		return decorate(
			c -> new AnotherDecorator(c, s));
	}

}

Fluent Decoration

Real-life example:

HyperlinkListener listener =
	this::changeHtmlViewBackgroundColor;
listener = DecoratingHyperlinkListener.from(listener)
	.onHoverMakeVisible(urlLabel)
	.onHoverSetUrlOn(urlLabel)
	.logEvents()
	.decorate(l ->
		new OnActivateHighlightComponent(l, urlLabel))
	.decorate(OnEnterLogUrl::new);

Why Default Methods?

Why not put these methods on AbstractDecorator?

  • clumps up responsibilities:

    • enabling easy implementation of Component

    • decorating instances of Component

    (change for different reasons)

  • requires implementation of abstract helper class

  • makes abstract helper class prominent

Reflection On Decorator

To implement fluent deocators:

  • add an additional interface

  • add default method for generic decoration

  • maybe add methods for specific decoration

interface DecoratingComponent extends Component {
	static DC from(Component component);
	default DC decorate(Function<DC, DC> decorator);
	default DC log(Level level);
}

Default Methods

Fluent Decorators
Interface Evolution

Evolving interfaces without breaking code.

Interface Evolution

If your code has clients that
you have no control over…​

  • open-source library

  • internal library

  • extensible application

... then evolving interfaces
always breaks code.

Default methods to the rescue!

General Approach

New Version
  • interface is transitional (old and new outline)

  • default methods ensure existing code works

Transition
  • client moves from old to new outline

  • default methods ensure code keeps working

New Version
  • removes old outline

Adding Methods

Reasonable default impl exists:

New Version
  • add the method with default impl

  • internal impls can override

  • internal callers use new method

Transition
  • external callers use the method

That’s it.

Adding Methods

No reasonable default impl exists:

New Version
  • add method with default impl throwing UOE

  • override method in all internal impls

Transition
  • external impls override the method

  • external callers use the method

New Version
  • make method abstract

  • internal callers use new method

Removing Methods

No external impls exist:

New Version
  • deprecate method

  • internal callers stop calling method

Transition
  • external callers stop using the method

New Version
  • remove the method

(No default methods required.)

Removing Methods

External impls exist:

New Version
  • deprecate method

  • provide default impl throwing UOE

  • internal callers stop calling method

Transition
  • external callers stop using the method

  • external impls of the method are removed

New Version
  • remove the method

Replacing Methods

Applies with new signature (name, parameters, …​),
where methods are "functionally equivalent".

Otherwise it’s a matter of adding new
and removing old method.

Replacing Methods

New Version
  • add new with default impl calling old

  • provide default impl of old calling new

  • deprecate old

  • internal impls override new instead of old

  • internal callers use new instead of old

Wtf, circular call?

  • ensures it does not matter which version is impl’d

  • must be thoroughly documented; tests help

Replacing Methods

Transition
  • external impls override new instead of old

  • external callers use new instead of old

New Version
  • make new abstract

  • remove old

Reflection On Evolution

If clients can be expected to update their code
default methods allow interface evolution
without breaking client code.

Mode is always the same:

  • release version with transitional outline

  • give clients time to update

  • release version with new outline

Additional Source

So long…​

37% off with
code fccparlog

bit.ly/the-jms

More

Slides at slides.nipafx.dev
⇜ Get my book!

Follow Nicolai

nipafx.dev
/nipafx

Follow Java

inside.java // dev.java
/java    //    /openjdk

Image Credits