The Java 9 Module System

Impedance Mismatch

Where the JVM disagrees with us

How do you think about Software?

What is it made of?

How I think about Software

  • interacting parts

  • parts have

    • names

    • dependencies

    • capabilities

  • creates a graph

How the JVM thinks about it

  • parts are packaged as JARs

  • to the JVM JARs

    • have no names

    • dependencies are unexpressed

    • have no coherent surface

  • JVM rolls them into one big ball of mud

Consequences

  • JAR hell

    • unexpressed, transitive dependencies

    • shadowing, version conflicts

    • complex class loading

  • slow

  • unsecure

  • maintenance nightmare

Project Jigsaw

Teaching the JVM about that graph

Quick History

2005

first discussions about modularity in JDK

2008

Project Jigsaw is created

2011-2014

exploratory phase; ends with JSR 376

2015

prototype is released

2017

Java 9 gets released with module system

Goals

  • Reliable Configuration

  • Strong Encapsulation

  • Scalable Systems (esp. the JDK)

  • Security, Performance, Maintainability

Non-Goals

  • Multiple Versions

  • Version Selection

Means

Introducing modules, which

  • have a name

  • express dependencies

  • encapsulate internals

Everything else follows from here!

Concepts & Features

  • Modules, Readability, Accessibility

  • Implied Readability, Qualified Exports

  • Modular JARs, Module Path, Module Graph

  • Services

  • Unnamed Modules, Automatic Modules

  • Reflection, Layers

  • Run-time Images

Java Module System Basics

Getting started...

Modules

Pieces of a puzzle

These are the nodes in our graph.

Definition

Modules

  • have a unique name

  • express their dependencies

  • export specific packages
    (and hide the rest)

Implementation

  • Modules are JARs with a module-info.class
    (aka Modular JAR)

  • gets generated from module-info.java:

    module $module_name {
    	requires $other_module;
    	exports $api_package;
    }
  • this is called a Module Declaration or a
    Module Descriptor

Readability

Putting the pieces together

Readability brings edges into our graph.

It is the basis for Reliable Configuration.

Definition

For two modules A and B with

module A {
	requires B;
}

we say

  • A requires B

  • A depends on B

  • A reads B

  • B is readable by A

Reliable Configuration

Java will only compile/launch when

  • every dependency is fulfilled

  • there are no cycles

  • there is no ambiguity

Consequences

  • boost for reliability

  • module system is strict and rigid
    (no way to easily alter dependencies)

  • accidental long cycles will cause problems

  • module name changes are not supported

Accessibility

Hiding internals

Accessibility governs which types a module can see.

It builds on top of Readability.

It is the basis for Strong Encapsulation.

Definition

A type in one module is only accessible
by code in another module if

  • the type is public

  • the package is exported

  • the second module reads the first

Strong Encapsulation

  • public is no longer public

  • even reflection doesn’t work

  • more fine-grained mechanisms exist:

    • for module authors in module declaration

    • for module users as command line arguments

Consequences

  • great boost for maintainability

  • major reason for community unrest

  • critical APIs survive until Java 10
    (e.g. sun.misc.Unsafe — see JEP 260)

  • life gets tougher for reflection-based
    libraries and frameworks

Jigsaw Advent Calendar

A simple example

Find it on GitHub!

Structure

advent calendar structure

Code

public static void main(String[] args) {
	List<SurpriseFactory> fac = asList(
		new ChocolateFactory(), new QuoteFactory());
	Calendar cal = Calendar.create(fac);
	println(cal.asText());
}
_

No Module

  • modularization is not required

  • JARs continue to work as today!

(Unless you do forbidden things, more on that later.)

  • we can just put the application
    on the class path as before

(Boring...)

No Module

Compilation, Packaging, Execution

# compile
javac -d classes/advent ${*.java}
# package with manifest
jar --create --file jars/advent.jar
	--manifest ${manifest}
	${*.class}
# run
java -jar jars/advent.jar

A single module

Modularization

advent calendar module single
module advent {
	// java.base is implicitly required
	// requires no other modules
	// exports no API
}

(Still Boring...)

A single module

Compilation, Packaging, Execution

# compile with module-info.java:
javac -d classes/advent ${*.java}
# package with module-info.class
#	and specify main class:
jar --create --file mods/advent.jar
	--main-class advent.Main
	${*.class}
# run by specifying a module path
#	and a module to run (by name):
java --module-path mods --module advent

A single module

Readability & Accessibility

advent calendar readability accessibility

Multiple Modules

b2e21fbf

Multiple Modules

advent calendar module multi
module surprise {
	// requires no other modules
	exports org.codefx.advent.surprise;
}
module calendar {
	requires surprise;
	exports org.codefx.advent.calendar;
}
module factories {
	requires surprise;
	exports org.codefx.advent.factories;
}
module advent {
	requires calendar;
	requires factories;
	requires surprise;
}

Multiple Modules

Compilation, Packaging, Execution

# compile all modules at once
javac -d classes
	--module-source-path "src"
	--module advent
# package one by one, eventually advent:
jar --create --file mods/advent.jar
	--main-class advent.Main
	${*.class}
# launch the application:
java --module-path mods --module advent

Advanced Java Module System

Going Deeper...

Transitive Dependencies

Not all dependencies are created equal:

  • most are used within a module

  • some are used on the boundary
    between modules

Transitive dependencies are about the latter.

Exposing dependencies

Example in calendar exposing surprise:

public static Calendar createWithSurprises(
		List<SurpriseFactory> factories) {
	// ...
}

⇝ Module calendar us unusable without surprise!

Try and Error?

How can users of exposing module
determine required dependencies?

Try and Error?

No!

Implied Readability

Exposing module can mark dependencies
on exposed modules
with requires transitive:

module A {
	requires transitive B;
}
  • A reads B as usual

  • modules reading A will read B
    without having to require it

A implies readability of B

Implied Readability

Applied to the advent calendar:

module calendar {
	requires transitive surprise;
	// ...
}

module factories {
	requires transitive surprise;
	// ...
}

Further Applications

Implied readability is surprisingly versatile

  • aggregator modules

  • splitting modules up

  • even merging modules

  • renaming modules

Aggregator Modules

Making it easier to consume
calendar, factories, surprise:

module adventcalendar {
	requires transitive calendar;
	requires transitive factories;
	requires transitive surprise;
}

Splitting Modules

If factories gets split into
api, chocolate, and quotes:

module factories {
	requires transitive factories.api;
	requires transitive factories.chocolate;
	requires transitive factories.quotes;
}

Merging Modules

If calendar, factories, surprise
are merged into adventcalendar:

module calendar {
	requires transitive adventcalendar;
}

module factories {
	requires transitive adventcalendar;
}

module surprise {
	requires transitive adventcalendar;
}

Careful: Users suddenly depend on a large module!

Renaming Modules

If factories becomes surprisefactories:

module factories {
	requires transitive surprisefactories;
}

Summary

With A requires transitive B:

  • A reads B

  • any module reading A reads B

Applications:

  • make sure API is usable
    without further dependencies

  • aggregator modules

  • splitting, merging, renaming modules

Optional Dependencies

Not all dependencies are equally required:

  • some are needed for a module to function

  • some can be absent and code adapts

  • are only needed to enhance
    another module’s features

Optional dependencies are about the latter two.

Use Cases

Adapting code:

  • a library may be absent from some deployments

  • code is aware and does not call absent modules

Enhancing a module:

  • a project may provide usability functions
    for other libraries

  • such code can not be called if module absent

Conundrum

With what we know so far:

  • for code to compile against another module
    that module has to be required

  • a required module has to be present
    at launch time

⇝ If a module’s types are used
it has to be present at run time

(Reliable configuration!)

Optional Dependencies

Dependency can be marked requires static:

module A {
	requires static B;
}
  • at compile time: A requires B as usual

  • at run time:

    • if B is present, A reads B

    • otherwise, app can launch
      but access to B can fail

Adapting Code

Assuming each factory has its own module,
advent may not need both all the time:

module advent {
	requires calendar;
	requires factories;
	requires static factories.chocolate;
	requires static factories.quote;
}

Adapting Code

Checking whether module is present:

Optional<SurpriseFactory> createChocolateFactory() {
	if (isModulePresent("factories.chocolate"))
		return Optional.of(new ChocolateFactory());
	else
		return Optional.empty();
}

Enhancing A Module

Hypothetical library uber-lib provides
usability functions for various libraries.

module uber.lib {
	requires static com.google.guava;
	requires static org.apache.commons.lang;
	requires static org.apache.commons.io;
	requires static io.vavr;
	requires static com.aol.cyclops;
}

Assumptions:

  • nobody calls into Guava part without using Guava

  • no runtime checks necessary

Summary

With A requires static B:

  • at compile time: A requires B as usual

  • at runtime: B may be absent

Use to prevent absence of optional dependencies
from preventing application launch:

  • modules with code adapting to absense

  • utility libraries that aren’t called
    without that dependency

Services

Consumers and implementations
of an API should be decoupled.

Service locator pattern:

  • service registry as central arbiter

  • implementors inform registry

  • consumers call registry to get implementations

In Java:

  • ServiceLoader is the registry

  • modules can interact with it

Services

Module Declarations:

// consumer
module A {
	uses service.Interface;
}

// provider
module B {
	provides service.Interface
		with some.Type;
}

Services

  • A never "sees" providers like B

  • module system picks up all providers

  • A can get providers from ServiceLoader

ServiceLoader.load(service.Interface.class)

Factory Services

module advent {
	requires calendar;
	uses surprise.SurpriseFactory;
}

module factory.chocolate {
	requires surprise;
	provides surprise.SurpriseFactory
		with factory.quote.ChocolateFactory;
}

module factory.quote {
	requires surprise;
	provides surprise.SurpriseFactory
		with factory.quote.QuoteFactory;
}

Factory Services

public static void main(String[] args) {
	List<SurpriseFactory> fac = new ArrayList<>();
	ServiceLoader.load(SurpriseFactory.class)
			.forEach(fac::add);
	Calendar cal = Calendar.createWithSurprises(fac);
	System.out.println(cal.asText());
}

Summary

To decouple API consumers and providers:

  • consumer uses Service

  • provider provides Service with Impl

Module system is service locator;
request implementations from ServiceLoader:

ServiceLoader.load(Service.class)`

Qualified Exports

So far, exported packages are visible
to every reading module.

What if a set of modules wants to share code?

Known Problem

Similar to utility classes:

  • if class should be visible outside of package,
    it has to be public ⇝ visible to everybody

  • if package should be visible outside of module,
    it has to be exported ⇝ visible to everybody

Module system fixes the former.
What about the latter?

Qualified Exports

module A {
	exports some.pack to B;
}
  • B can access some.pack
    as if regularly exported

  • other modules can not access it
    as if not exported at all

Factory Utilities

To ease implementation of SurpriseFactory:

  • create new module factory

  • add class AbstractSurpriseFactory

  • export that package only to
    factory implementation modules

module factory {
	requires transitive surprise;
	exports factory
		to factory.chocolate, factory.quote;
}

Summary

With A exports pack to B:

  • only B can access types in pack

  • other modules behave as if pack
    not exported

Use to share sensible code between modules.

Reflective Access

Exporting a package makes it public API:

  • modules can compile code against it

  • clients expect it to be stable

What if a type is only meant
to be accessed via reflection?

(Think Spring, Hibernate, etc.)

Open Packages

module A {
	opens some.pack;
}
  • at compile time:
    types in some.pack are not accessible

  • at run time:
    all types and members in some.pack
    are accessible

A qualified variant (opens to) exists.

Open Modules

open module A {
	// no more `opens` clauses
}

The same as open packages
but for all of them!

Summary

With open modules or open packages:

  • code can be made accessible
    at compile time only

  • particularly valuable to open
    for reflection

Use to make types available for reflection
without making them public API.

Command Line Arguments

The module system is pretty strict but…​

  • …​ not all modules are well-designed

  • …​ not all use cases were intended

What to do then?

Command Line Arguments to the rescue!
(I often call them "escape hatches".)

Command Line Arguments

All command line flags can be applied
to javac and java!

(Which is weird for --add-opens.)

When used during compilation,
they do not change the resulting
module descriptor!

Add Modules

If a module is not required,
it might not make it into the module graph.

Help it get there with --add-modules:

	java --module-path mods
#	--add-modules <module>(,<module>)*
		--add-modules factories
		--module advent

Add JavaEE Modules

JavaEE modules are not resolved by default!

  • java.activation

  • java.annotations.common

  • java.corba

  • java.transaction

  • java.xml.bind

  • java.xml.ws

They need to be added with --add-modules!

Add Readability Edges

Maybe a module in the graph is not readable
by another but you need it to be.

Add readability edges with --add-reads:

	java --module-path mods
#	--add-reads <module>=<target>(,<more>)*
		--add-reads advent=factories
		--module advent

Add Exports

A common case:

A module needs types that
the owning module doesn’t export.

Export them with --add-exports:

	java --module-path mods
#	--add-exports <module>/<package>=<target>
		--add-reads factories/factories.quotes=advent
		--module advent

Use target ALL-UNNAMED to export to all modules.

Open Packages

Another common case:

A module reflects on types from a package that
the owning module doesn’t open.

Open packages with add-opens:

	java --module-path mods
#	--add-opens <module>/<package>=<target>
		--add-opens factories/factories.quotes=advent
		--module advent

Use target ALL-UNNAMED to open to all modules.

(It is not possible to open an entire module.)

Summary

Edit module graph with:

  • --add-modules to add modules

  • --add-reads to add readability edges

  • --add-exports to export packages to modules

  • --add-opens to open packages to modules

The latter two accept ALL-UNNAMED as target.

Migration

Moving Into Module Land

Why Is It Even An Option?

  • most module systems are "in or out",

  • but modularized JDK and legacy JARs
    have to cooperate!

  • so migration has to be possible

Enablers

Migration is enabled by two features:

  • Unnamed Module(s)

  • Automatic Modules

And the fact that module and class path coexist:

  • modular JARs can be put on either

  • "regular" JARs can be put on either

The Unnamed Module

Why The Class Path "Just Works"

Definition

The Unnamed Module
contains all JARs on the class path
(including modular JARs).

  • has no name (surprise!)

  • can read all modules

  • exports all packages

Inside the unnamed module
"the chaos of the class path" lives on.

Example

Put all your JARs on the class path.

migration unnamed

No Access

  • what if your code was modularized?
    and your dependencies were not?

  • proper modules can not depend on
    "the chaos on the class path"

  • this is not possible:

    module advent {
    	requires unnamed;
    }

No Access

migration unnamed dependency

Automatic Modules

From Modules To The Class Path

Definition

An Automatic Module
is created for each "regular" JAR
on the module path.

  • gets a name based on the file name

  • can read all modules
    (including the Unnamed Module)

  • exports all packages

Example

  • put guava-19.0.jar on the module path

  • then this works:

    module advent {
    	requires guava;
    }

Example

migration automatic

What Goes Where?

Class PathModule Path

Regular JAR

Unnamed Module

Automatic Module

Modular JAR

Unnamed Module

Named Module

Migration Strategies

Three strategies emerge:

  • bottom-up migration

  • top-down migration

  • inside-out migration

Bottom-Up Migration

Works best for projects without
unmodularized dependencies
(libraries).

  • turn project JARs into modules

  • they still work on the class path

  • clients can move them to the module path
    whenever they want

Top-Down Migration

Required for projects with
unmodularized dependencies
(applications).

  • turn project JARs into modules

Top-Down Migration

  • modularized dependencies:

    • require direct ones

    • put all on the module path

  • unmodularized dependencies:

    • require direct ones with automatic name

    • put direct ones on the module path

    • put others on the class path

Top-Down Migration

When dependencies get modularized:

  • hopefully the name didn’t change

  • if they are already on the module path,
    nothing changes

  • otherwise move them there

  • check their dependencies

Inside-Out Migration?

What about published projects with
unmodularized dependencies
(libraries)?

  • top-down seems to work

  • but there’s a problem
    with automatic module names!

Inside-Out Migration?

Unstable Module Names

  • automatic module names
    are based on JAR names

  • file names can differ
    across build environments

  • module name can change
    when project gets modularized

⇝ Automatic module names are unstable.

Inside-Out Migration?

Impossible Module Requirements

  • dependencies might require the same
    module by different names

  • the module system does not support that

  • there is no way to launch that application!

⇝ JDK team recommends not to publish
modules that depend on automatic modules!

No Inside-Out Migration

Frameworks/libraries can only publish modules
once all dependencies were modularized.

This will make modularization of the ecosystem
take much, much longer! 😭

Compatibility

What to look out for
when running on JDK 9

Break Stuff

Some internal changes can break existing code!

Just by running on JDK 9
(even without modularizing the application).

JEP 261 contains a list of risks.

Internal APIs

  • internal APIs are:

    • all in sun.*

    • most in com.sun.*
      (unless marked @jdk.Exported)

  • most if them are encapsulated

  • critical APIs will survive until Java 10
    (e.g. sun.misc.Unsafe — see JEP 260)

  • six deprecated methods adding/removing
    PropertyChangeListener got removed
    from LogManager and Pack200

What to look for?

  • use jdeps (preferably from JDK 9)

    • either manually:
      jdeps --jdk-internals {.class/.jar}

    • or as part of your build (e.g. with JDeps Mvn)

  • look for reflection, especially

    • Class::forName

    • AccessibleObject::setAccessible

  • recursively check your dependencies!

What to do?

  1. contact library developers

  2. look for alternatives
    (in the JDK or other libraries)

  3. consider command line flags
    --add-exports, --add-opens, or
    --permit-illegal-access

Split Packages

  • packages should have a unique origin

  • no module must read the same package
    from two modules

The implementation is even stricter:

  • no two modules must contain
    the same package (exported or not)

  • split packages on class path
    are inaccessible

Examples

  • some libraries split java.xml.*, e.g. xml-apis

  • some JBoss modules split, e.g.,
    java.transaction, java.xml.ws

  • jsr305 splits javax.annotation

What to look for?

  • search your code and dependencies
    for java(x) packages (jdeps can help)

  • no tool support (yet?)

What to do?

  1. is the split on purpose / necessary?

  2. find other ways to solve the problem

  3. upgradeable modules
    to replace run-time modules

  4. command line --patch-module
    to add individual classes

Run-Time Images

  • new JDK/JRE layout

  • internal JARs are gone (e.g. rt.jar, tools.jar)

  • JARs are now JMODs

  • application class loader is no URLClassLoader

  • new URL schema for run-time image content

What to look for?

  • does the code rummage around
    in the JDK / JRE folder?

  • search for casts to URLClassLoader

  • are URLs to JDK classes / resources handcrafted?

Obsolete

  • Endorsed Standards Override Mechanism

  • Extension Mechanism

  • Boot Class Path Override

But wait, there’s more!

  • yes, yes, there’s more

  • you should really check JEP 261

  • and JEP 223 (new version strings)

    • goodbye 1.9.0_31

    • hello 9.1.4

General Advice I

The most relevant for most applications:

  • internal APIs

  • split packages

General Advice II

  • get your code in shape
    (and prevent relapses)

  • check your dependencies and tools

  • if any are suspicious
    (automatically true for IDEs, build tools):

    • make sure they’re alive

    • get them up to date!

    • or look for alternatives

  • get an EA-build and try it!

About Nicolai Parlog

37% off with
code fccparlog

tiny.cc/j9ms

Want More?

⇜ Get my book!

I write a mean newsletter,
currently mostly about Java 9.

You can hire me.

What About OSGi?

Brief comparison
of Jigsaw and OSGi

Jigsaw vs. OSGi

OSGi Bundles:

  • are JARs with a descriptor (MANIFEST.MF)

  • have a name

  • import packages or bundles

  • define public API by exporting packages

Jigsaw vs. OSGi

JigsawOSGi

Versioning

not at all

packages and modules

Run-time Behavior

mostly static

dynamic

Services

declarative via ServiceLoader

declarative or programmatically;
more flexible

Class Loaders

operates below

one per bundle

Image Credits

Introduction

Project Jigsaw

Java Module System

Migration

Compatibility

Rest