The Java Module System

Nicolai Parlog

codefx.org / @nipafx

JUG Nürnberg

@JUGNBG

Let’s get started!

there’s much to talk about
⇝ this can only cover parts of the JPMS
⇝ we may have to skip some details
slides at slides.codefx.org/jpms

Impedance Mismatch

Project Jigsaw

Java Module System Basics

Beyond The Basics

Runtime Images With jlink

Migration Challenges

Impedance Mismatch

Where the JVM disagrees with us

How do you think about Software?

What is it made of?

How I think about Software

interacting parts
parts have
- names
- dependencies
- capabilities
creates a graph

How the JVM thinks about it

parts are packaged as JARs
to the JVM JARs
- have no names
- dependencies are unexpressed
- have no coherent surface
JVM rolls them into one big ball of mud

Consequences

JAR hell
- unexpressed, transitive dependencies
- shadowing, version conflicts
- complex class loading
slow
unsecure
maintenance nightmare

Project Jigsaw

Teaching the JVM
about that graph

Quick History

2005: first discussions about modularity in JDK
2008: Project Jigsaw is created
2011-2014: exploratory phase; ends with JSR 376
2015: prototype is released
2017: Java 9 gets released with module system

Goals

Reliable Configuration
Strong Encapsulation
Scalable Systems (esp. the JDK)
Security, Performance, Maintainability

Non-Goals

Version Selection
Multiple Versions
Package Isolation

Means

Introducing modules, which

have a name
express dependencies
encapsulate internals

Everything else follows from here!

Concepts & Features

Modules, Readability, Accessibility
Implied Readability, Qualified Exports
Modular JARs, Module Path, Module Graph
Services
Unnamed Modules, Automatic Modules
Reflection, Layers
Run-time Images

Java Module System Basics

Getting started...

Modules

Pieces of a puzzle

These are the nodes in our graph.

Definition

Modules

have a unique name
express their dependencies
export specific packages
(and hide the rest)

Implementation

Modules are JARs with a module-info.class
(aka Modular JAR)
gets generated from module-info.java
(in sources root, e.g. src/main/java):
```
module $module_name {
	requires $other_module;
	exports $api_package;
}
```
this is called a Module Declaration or a
Module Descriptor

Readability

Putting the pieces together

Readability brings edges into our graph.

It is the basis for Reliable Configuration.

Definition

For two modules A and B with

module A {
	requires B;
}

we say

A requires B
A depends on B
A reads B
B is readable by A

Reliable Configuration

Java will only compile/launch when

every dependency is fulfilled
there are no cycles
there is no ambiguity

Consequences

boost for reliability
module system is strict and rigid
(no way to easily alter dependencies)
module name changes are not supported
accidental long cycles will cause problems

Accessibility

Hiding internals

Accessibility governs which types a module can see.

It builds on top of Readability.

It is the basis for Strong Encapsulation.

Definition

A type in one module is only accessible
by code in another module if

the type is public
the package is exported
the second module reads the first

Strong Encapsulation

public is no longer public
even reflection doesn’t work
more fine-grained mechanisms exist:
- for module authors in module declaration
- for module users as command line arguments

Consequences

great boost for maintainability
major reason for community unrest
critical APIs survive longer
(e.g. sun.misc.Unsafe — see JEP 260)
life gets tougher for reflection-based
libraries and frameworks

Jigsaw Advent Calendar

A simple example

Structure

Code

public static void main(String[] args) {
	List<SurpriseFactory> factories = List.of(
		new ChocolateFactory(), new QuoteFactory());
	Calendar cal = Calendar.create(factories);
	System.out.println(cal.asText());
}

No Module

modularization is not required
JARs continue to work as today!

(Unless you do forbidden things, more on that later.)

we can just put the application
on the class path as before

(Boring...)

No Module

Compilation, Packaging, Execution

# compile
$ javac -d classes ${*.java}
# package with manifest
$ jar --create --file jars/advent.jar
    --manifest ${manifest}
    ${*.class}
# run
$ java -jar jars/advent.jar

To look up javac, jar, java basics:
tiny.cc/cheat-modules

A single module

Modularization

Create module-info.java in sources root:

module advent {
	// java.base is implicitly required
	// requires no other modules
	// exports no API
}

(Still Boring...)

A single module

Compilation, Packaging, Execution

# compile with module-info.java
$ javac -d classes ${*.java}
# package with module-info.class
# and specify main class
$ jar --create
    --file mods/advent.jar
    --main-class advent.Main
    ${*.class}
# run by specifying a module path
# and a module to run (by name)
$ java --module-path mods --module advent

A single module

The Module Path

used by java, javac, and other tools
to look up dependencies
accepts a list of artifacts or folders
intended for modular artifacts

Example use:

# `mods` is a directory with JARs
$ java --module-path mods # ...

A single module

Readability & Accessibility

Multiple Modules

module surprise {
	// requires no other modules
	exports org.codefx.advent.surprise;
}

module calendar {
	requires surprise;
	exports org.codefx.advent.calendar;
}

module factories {
	requires surprise;
	exports org.codefx.advent.factories;
}

module advent {
	requires calendar;
	requires factories;
	requires surprise;
}

Multiple Modules

Compilation, Packaging, Execution

# compile all modules at once
$ javac -d classes
    --module-source-path "src"
    --module advent
# package one by one, eventually advent
$ jar --create
    --file mods/advent.jar
    --main-class advent.Main
    ${*.class}
# launch the application
$ java --module-path mods --module advent

Multiple Modules

The Module Source Path

With multi-module compilation, several
modules can be compiled at once.

used by javac to look up source files
defines a folder structure
uses * as placeholder for module name

Example:

--module-source-path "./*/src/main/java"

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Transitive Dependencies

Not all dependencies are created equal:

most are used within a module
some are used on the boundary
between modules

Transitive dependencies are about the latter.

Exposing dependencies

Example in calendar exposing surprise:

public static Calendar create(
		List<SurpriseFactory> factories) {
	// ...
}

⇝ Module calendar is unusable without surprise!

Try and Error?

How can users of exposing module
determine required dependencies?

Try and error?

No!

Implied Readability

Exposing module can mark dependencies
on exposed modules with
requires transitive:

module A {
	requires transitive B;
}

A reads B as usual
modules reading A will read B
without having to require it

⇝ A implies readability of B

Implied Readability

Applied to the advent calendar:

module calendar {
	requires transitive surprise;
	// ...
}

Further Applications

Implied readability is surprisingly versatile

aggregator modules
splitting modules up
even merging modules
renaming modules

Aggregator Modules

Making it easier to consume
calendar, factories, surprise:

module adventcalendar {
	requires transitive calendar;
	requires transitive factories;
	requires transitive surprise;
}

Splitting Modules

If factories gets split into
api, chocolate, and quotes:

module factories {
	requires transitive factory.api;
	requires transitive factory.chocolate;
	requires transitive factory.quotes;
}

Merging Modules

If calendar, factories, surprise
are merged into adventcalendar:

module calendar {
	requires transitive adventcalendar;
}

module factories {
	requires transitive adventcalendar;
}

module surprise {
	requires transitive adventcalendar;
}

Careful: Users suddenly depend on a large module!

Renaming Modules

If factories becomes surprisefactories:

module factories {
	requires transitive surprisefactories;
}

Summary

With A requires transitive B:

A reads B
any module reading A reads B

Applications:

make API usable without further dependencies
aggregator modules
splitting, merging, renaming modules

More at codefx.org:

Implied Readability

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Optional Dependencies

Not all dependencies are equally required:

some are needed for a module to function
some can be absent and code adapts
some are only needed to enhance
another module’s features

Optional dependencies are about the latter two.

Adapting Code

Use case:

a library may be absent from some deployments
code is aware and does not call absent modules

Example:

each SurpriseFactory implementation
has its own module
advent works without any specific factory

Adapting Code

Enhancing A Module

Use case:

a project may provide usability functions
for other libraries
such code will not be called if library modules
are absent

Example:

hypothetical library uber-lib
provides usability functions for various libraries

Enhancing A Module

Conundrum

With what we know so far:

for code to compile against another module
that module has to be required
a required module has to be present
at launch time

⇝ If a module’s types are used
it has to be present at run time

(Reliable configuration!)

Optional Dependencies

Dependency can be marked requires static:

module A {
	requires static B;
}

at compile time: A requires B as usual
at run time:
- if B is present, A reads B
- otherwise, app can launch
  but access to B can fail

Adapting Code

For advent and the two factories:

module advent {
	requires calendar;
	requires surprise;
	requires static factory.chocolate;
	requires static factory.quote;
}

Adapting Code

Checking whether module is present:

Optional<SurpriseFactory> createChocolateFactory() {
	if (isModulePresent("factory.chocolate"))
		return Optional.of(new ChocolateFactory());
	else
		return Optional.empty();
}

Enhancing A Module

For uber-lib:

module uber.lib {
	requires static com.google.guava;
	requires static org.apache.commons.lang;
	requires static io.vavr;
	requires static com.aol.cyclops;
}

Assumptions:

nobody calls into Guava part without using Guava
no runtime checks necessary

Summary

With A requires static B:

at compile time: A requires B as usual
at runtime: B may be absent

Two kinds of applications:

modules with code adapting to absence
utility libraries that aren’t called
without that dependency

More at codefx.org:

Optional Dependencies

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Dependency Inversion?

Service Locator Pattern

Consumers and implementations
of an API should be decoupled.

Service locator pattern:

service registry as central arbiter
implementors inform registry
consumers call registry to get implementations

Services in JPMS

In the JPMS:

modules declare which services they use
modules declare which services they provide
ServiceLoader is the registry
code can interact with it to load services

Service Declarations

Module declarations:

// consumer
module A {
	uses some.Service;
}

// provider
module B {
	provides some.Service
		with some.Type;
}

(A and B need access to some.Service)

Loading Services

A never "sees" providers like B
module system picks up all providers
A can get providers from ServiceLoader

ServiceLoader.load(Service.class)

Factory Services

module advent {
	requires surprise;
	uses surprise.SurpriseFactory;
}

module factory.chocolate {
	requires surprise;
	provides surprise.SurpriseFactory
		with factory.chocolate.ChocolateFactory;
}

module factory.quote {
	requires surprise;
	provides surprise.SurpriseFactory
		with factory.quote.QuoteFactory;
}

Factory Services

Factory Services

public static void main(String[] args) {
	List<SurpriseFactory> factories = ServiceLoader
		.load(SurpriseFactory.class).stream()
		.map(Provider::get)
		.collect(toList());
	Calendar cal = Calendar.create(factories);
	System.out.println(cal.asText());
}

Summary

To decouple API consumers and providers:

consumer uses Service
provider provides Service with Impl

Module system is service locator;
request implementations from ServiceLoader:

ServiceLoader.load(Service.class)

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Qualified Exports

So far, exported packages are visible
to every reading module.

What if a set of modules wants to share code?

Known Problem

Similar to utility classes:

if class should be visible outside of package,
it has to be public ⇝ visible to everybody
if package should be visible outside of module,
it has to be exported ⇝ visible to everybody

Module system fixes the former.
What about the latter?

Qualified Exports

module A {
	exports some.pack to B;
}

B can access some.pack
as if regularly exported
other modules can not access it
as if not exported at all

Factory Utilities

To ease implementation of SurpriseFactory:

create new module factory
add class AbstractSurpriseFactory
export that package only to
factory implementation modules

module factory {
	requires transitive surprise;
	exports factory
		to factory.chocolate, factory.quote;
}

Summary

With A exports pack to B:

only B can access types in some.pack
other modules behave as if some.pack
is not exported

Use to share sensible code between modules.

Java Module System Beyond The Basics

Transitive Dependencies

Optional Dependencies

Services

Qualified Exports

Reflective Access

Exporting a package makes it public API:

modules can compile code against it
clients expect it to be stable

What if a type is only meant
to be accessed via reflection?

(Think Spring, Hibernate, etc.)

Open Packages

module A {
	opens some.pack;
}

at compile time:
types in some.pack are not accessible
at run time:
all types and members in some.pack
are accessible

A qualified variant (opens to) exists.

Open Modules

open module A {
	// no more `opens` directives
}

The same as open packages
but for all of them!

Summary

With open modules or open packages:

code can be made accessible
at run time only
particularly valuable to open
for reflection

Use to make types available for reflection
without making them public API.

Creating Runtime Images With JLink

Custom-Made For Your Application

Remember This?

Benefiting From JDK Modularization

Create a JDK install with just the code you need:

know which modules your app uses (⇝ jdeps)
create an image with those modules (⇝ jlink)

This is about jlink.

A Minimal JDK Image

Create with jlink:

$ jlink
#   define output folder for the image
    --output jdk-minimal
#   where to find modules? (obsolete in ⑪)
    --module-path $JAVA_HOME/jmods
#   which modules to add (includes dependencies!)
    --add-modules java.base

Try it out:

$ jdk-minimal/bin/java --list-modules
> java.base

Image For A Backend

Say you use JAXP, JDBC, and JUL:

$ jlink
	--output jdk-backend
	--module-path $JAVA_HOME/jmods # obsolete in ⑪
	--add-modules java.xml,java.sql,java.logging

Image For A Backend

$ jdk-backend/bin/java --list-modules
> java.base
> java.logging
> java.sql
> java.transaction.xa
> java.xml

Image Including Your App And Dependencies

To create an image for your app:

all JARs need to be modularized!
including dependencies

Unless you use Gunnar Morling’s ModiTect,
which creates module descriptors on the fly.

Linking And Launching

Creating the image:

$ jlink
	--output app-image
	--module-path $JAVA_HOME/jmods:mods
	--add-modules advent
#   services are not resolves automatically
	--add-modules factory.surprise,factory.chocolate

Launching the app:

app-image/bin/java --module advent

Creating A Launcher

You can even create a launcher:

$ jlink
    --output app-image
    --module-path $JAVA_HOME/jmods:mods
    --add-modules advent,...
#   --launcher <name>=<module>[/<mainclass>]
    --launcher calendar=advent

Launching the app:

app-image/bin/calendar

More Features

automatic service binding
(with --bind-services)
various optimizations
(size and launch performance)
plugin API (not yet public)
cross OS image generation

Summary

You can use jlink to:

create a runtime image
with just the right modules
create an application image
including your code

This should make certain kinds of deploys
smaller and easier.

Migration Challenges

What to look out for
when running on JDK 11

Break Stuff

Some internal changes break existing code!

Just by running on JDK 11
(even without modularizing the application).

Of Modules And JARs

Modularized JDK and legacy JARs have to cooperate.

Two requirements:

for the module system to work,
everything needs to be a module
for compatibility, the class path
and regular JARs have to keep working

The Unnamed Module

The Unnamed Module
contains all JARs on the class path
(including modular JARs).

has no name (surprise!)
can read all modules
exports all packages

Inside the unnamed module
"the chaos of the class path" lives on.

Challenges

internal APIs
Java EE modules
split packages
runtime images

Internal APIs

internal APIs are:
- all in sun.*
- most in com.sun.*
  (unless marked @jdk.Exported)
encapsulated at compile time
accessible at run time
for some time
critical APIs may survive longer
(e.g. sun.misc.Unsafe)

What to look for?

JDeps can report internal dependencies:

$ jdeps --jdk-internals
	-recursive --class-path 'libs/*'
	scaffold-hunter-2.6.3.jar

> batik-codec.jar -> JDK removed internal API
>     JPEGImageWriter -> JPEGCodec
> guava-18.0.jar -> jdk.unsupported
>     Striped64 -> Unsafe
> scaffold-hunter-2.6.3.jar -> java.desktop
>     SteppedComboBox -> WindowsComboBoxUI

What else to look for?

look for reflection, especially
- Class::forName
- AccessibleObject::setAccessible
recursively check your dependencies!

What to do?

fix your code
contact library developers
look for alternatives
(in the JDK or other libraries)
consider command line flags
--add-exports, --add-opens, or
--illegal-access

Java EE Modules

java.activation (javax.activation)
java.corba (CORBA packages)
java.transaction (javax.transaction)
java.xml.bind (javax.xml.bind.*)
java.xml.ws (JAX-WS packages)
java.xml.ws.annotation (javax.annotation)

These were

deprecated for removal in ⑨
removed in ⑪

What to look for?

JDeps shows dependencies on platform modules:

$ jdeps -summary sh-2.6.3.jar

> sh-2.6.3.jar -> java.base
> sh-2.6.3.jar -> java.datatransfer
> sh-2.6.3.jar -> java.desktop
> sh-2.6.3.jar -> java.logging
> sh-2.6.3.jar -> java.prefs
> sh-2.6.3.jar -> java.sql
> sh-2.6.3.jar -> java.xml

What to do?

Pick a third-party implementation.

Split Packages

packages should have a unique origin
no module must read the same package
from two modules

The implementation is even stricter:

no two modules must contain
the same package (exported or not)
split packages on class path
are inaccessible

Examples

some libraries split java.xml.*, e.g. xml-apis
some JBoss modules split, e.g.,
java.transaction, java.xml.ws
jsr305 splits javax.annotation

What to look for?

JDeps reports split packages:

$ jdeps -summary
	-recursive --class-path 'libs/*'
	project.jar

> split package: javax.annotation
>     [jrt:/java.xml.ws.annotation,
>         libs/jsr305-3.0.2.jar]

What to do?

Your artifacts:

rename one of the packages
merge package into the same artifact
merge the artifacts
place both artifacts on the class path

Otherwise:

upgrade the JDK module with the artifact
--patch-module with the artifact’s content

Run-Time Images

new JDK/JRE layout
internal JARs are gone (e.g. rt.jar, tools.jar)
JARs are now JMODs
application class loader is no URLClassLoader
(no way to append to its class path)
new URL schema for run-time image content

What to look for?

does the code rummage around
in the JDK / JRE folder?
are URLs to JDK classes / resources handcrafted?
search for casts to URLClassLoader

Obsolete

Compact Profiles
Extension Mechanism
Endorsed Standards Override Mechanism
Boot Class Path Override
JRE version selection with -version:N

But wait, there’s more!

Yes, yes, there’s more:

Java 11 Migration Guide

Background:

And there are new version strings:

goodbye 1.9.0_31, hello 9.0.1

General Advice I

The most relevant for most applications:

internal APIs
split packages
Java EE modules

General Advice II

get your code in shape
(and prevent relapses)
check your dependencies and tools
if any are suspicious
(automatically true for IDEs, build tools):
- make sure they’re alive
- get them up to date!
- or look for alternatives
download Java 11 and try it!

About Nicolai Parlog

37% off with
code fccparlog

tiny.cc/jms

Follow

💻 codefx.org
🐦 @nipafx

Want More?

Slides at slides.codefx.org

⇜ Get my book!

You can hire me:

training (Java 8-12, JUnit 5)
consulting (Java 8-12)

Image Credits

Introduction

puzzle-people: Kevin Dooley (CC-BY 2.0)
binary-code: Christiaan Colen (CC-BY-SA 2.0)
graph: chef_ele (CC-BY 2.0)
ball-of-mud-2: Andi Gentsch (CC-BY-SA 2.0)
jar-hell: Wellcome Library, London (CC-BY 4.0)

Project Jigsaw

flag-amsterdam: George Rex (CC-BY-SA 2.0)
puzzle-cubed: David Singleton (CC-BY 2.0)

Java Module System

puzzle-piece-green:
StockMonkeys.com (CC-BY 2.0)
puzzle-pieces-put-together:
Ken Teegardin (CC-BY-SA 2.0)
iceberg:
NOAA’s National Ocean Service (CC-BY 2.0)
advent-calendar: Tina D (CC-BY 2.0)
class and module diagrams:
Nicolai Parlog (CC-BY-NC 4.0)

Incremental Modularization

keep-out: Brian Smithson (CC-BY 2.0)
garbage-only: Peter Kaminski (CC-BY 2.0)
golden-gate: Nicolas Raymond (CC-BY 2.0)
confusion: Procsilas Moscas (CC-BY 2.0)
module diagrams:
Nicolai Parlog (CC-BY-NC 4.0)

Migration Challenges

broken-glass:
Eric Schmuttenmaer (CC-BY-SA 2.0)
internals: David French (CC-BY 2.0)
cut: Jinx! (CC-BY-SA 2.0)
cells: Jonathan Lin (CC-BY-SA 2.0)
obsolete: Trevor King (CC-BY 2.0)
sign: Duncan Harris (CC-BY-SA 2.0)

Rest

question-mark: Milos Milosevic (CC-BY 2.0)
bundles: Danumurthi Mahendra (CC-BY 2.0)

The Java Module System

Let’s get started!

Table of contents

Impedance Mismatch

Where the JVM disagrees with us

How do you think about Software?

What is it made of?

How I think about Software

How the JVM thinks about it

Consequences

Project Jigsaw

Teaching the JVMabout that graph

Quick History

Goals

Non-Goals

Means

Concepts & Features

Java Module System Basics

Getting started...

Modules

Pieces of a puzzle

Definition

Implementation

Readability

Putting the pieces together

Definition

Reliable Configuration

Consequences

Accessibility

Hiding internals

Definition

Strong Encapsulation

Consequences

Jigsaw Advent Calendar

A simple example

Structure

Code

No Module

No Module

Compilation, Packaging, Execution

A single module

Modularization

A single module

Compilation, Packaging, Execution

A single module

The Module Path

A single module

Readability & Accessibility

Multiple Modules

Multiple Modules

Multiple Modules

Compilation, Packaging, Execution

Multiple Modules

The Module Source Path

Java Module System Beyond The Basics

Java Module System Beyond The Basics

Transitive Dependencies

Exposing dependencies

Try and Error?

Implied Readability

Implied Readability

Further Applications

Aggregator Modules

Splitting Modules

Merging Modules

Renaming Modules

Summary

Java Module System Beyond The Basics

Optional Dependencies

Adapting Code

Adapting Code

Enhancing A Module

Enhancing A Module

Conundrum

Optional Dependencies

Adapting Code

Adapting Code

Enhancing A Module

Summary

Java Module System Beyond The Basics

Teaching the JVM
about that graph

What to look out for
when running on JDK 11