Java Modules In Real Life

Nicolai Parlog

nipafx.dev / @nipafx

IntelliJ LiveStream

@intellijidea

Developer Advocate

Java Team at Oracle

Java Modules In Real Life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where's the exit?!
When (not) to use modules?

Slides at slides.nipafx.dev/modules-irl

Java Modules In Real Life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where’s the exit?!
When (not) to use modules?

Managing dependencies

Does this compile
(when built with Maven)?

<dependency>
	<groupId>com.google.guava</groupId>
	<artifactId>guava</artifactId>
	<version>31.1-jre</version>
</dependency>
import org.checkerframework.checker.units.qual.Prefix;
public class Transitive {
	private final Prefix prefix = Prefix.centi;
}

Should it?

Managing dependencies

Now it doesn’t
(regardless of build tool):

module com.example.main {
	requires com.google.common;
}

⇝ Modules make dependencies more explicit.

Self-contained applications

Explicit dependencies in action:

# create the image
$ jlink
    --module-path mods
    --add-modules com.example.main
    --output app-image

# list contained modules
$ app-image/bin/java --list-modules
> com.example.app
# other app modules
> java.base
# other java/jdk modules

⇝ Modules make it easy to create application images.

Defining APIs

Which packages can I use?

junit pioneer packages

Defining APIs

Exactly these:

module org.junitpioneer {
	exports org.junitpioneer.jupiter;
	exports org.junitpioneer.jupiter.cartesian;
	exports org.junitpioneer.jupiter.params;
	exports org.junitpioneer.vintage;
}

⇝ Modules clarify APIs.

⇝ Modules add module-internal accessibility.
(enforced by compiler and runtime).

Project-internal APIs

How to stop projects integrating your code
from using all your APIs?

module com.example.lib {
	exports com.example.lib.internal
		to com.example.main;
}

⇝ Modules add project-internal accessibility.
(enforced by compiler and runtime).

Illegal access errors

  • in Spring LDAP:

    org.springframework.ldap...AbstractContextSource
⇝ com.sun.jndi.ldap.LdapCtxFactory

  • in Error Prone:

    com.google.errorprone.BaseErrorProneJavaCompiler
⇝ com.sun.tools.javac.api.BasicJavacTask

  • in veraPDF-library:

    org.verapdf.gf.model.impl.external.GFPKCSDataObject
⇝ sun.security.pkcs.PKCS7

Illegal access

Dependencies on internals of…​

  • JDK

  • frameworks

  • libraries

... are a risk.

⇝ Modules make these explicit
(with errors or --add-exports)
and incentivize fixing them.

Explicit services

Which services does a JAR use?

module java.sql {
	uses java.sql.Driver;
}

⇝ Modules make consuming services explicit.

Simple services

No more files in META-INF/services/ — instead:

module com.example.sql {
	provides java.sql.Driver
		with com.example.sql.ExampleDriver;
}

⇝ Modules make providing services simpler.

New abstraction

What describes a project
(and how to look it up):

  • name ⇝ build tool

  • API ⇝ 🤷🏾‍♂️

  • dependencies ⇝ build tool

  • services ⇝ META-INF/services/

  • high-level documentation ⇝ 🤷🏼‍♀️

⇝ Modules express this in one file.

High-level documentation

Module declaration is a great place to document:

  • central abstraction, contract, design

  • unexpected dependencies

  • unusual API

  • allowance of reflective access

  • service interactions

Evolving architecture

Module declarations:

  • define and document a project

  • are verified by compiler and runtime

  • can be evaluated by other tools

  • are obvious to review

⇝ Modules are a living representation
of a project’s architecture.

Java modules in real life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where’s the exit?!
When (not) to use modules?

Incremental modularization

Why consider it?

  • partial modularization gives you
    partial benefits

  • can avoid some roadblocks

  • makes the process more relaxed

Incremental modularization

Why is it even an option?

  • most module systems are "in or out"

  • but modularized JDK and legacy JARs
    have to cooperate!

  • there is a boundary between
    legacy and modules

Incremental modularization means
moving that boundary.

Enablers

Incremental modularization is enabled by two features:

  • unnamed module(s)

  • automatic modules

And the fact that module and class path coexist:

  • modular JARs can be put on either

  • "regular" JARs can be put on either

The unnamed module

Contains all JARs on the class path
(including modular JARs).

  • has no name (surprise!)

  • can read all modules

  • exports all packages

Inside the unnamed module,
"the chaos of the class path" lives on.

⇝ Why the class path "just works".

No Access

  • what if your code was modularized
    and your dependencies were not?

  • proper modules can not depend on
    "the chaos on the class path"

  • this is not possible:

    module com.example.app {
	requires unnamed;
}

Automatic modules

One is created for each "regular" JAR
on the module path.

  • name defined by manifest entry
    AUTOMATIC-MODULE-NAME or
    derived from JAR name

  • can read all modules
    (including the unnamed module)

  • exports all packages

What goes where?

Class PathModule Path

Regular JAR

Unnamed Module

Automatic Module

Modular JAR

Unnamed Module

Explicit Module

Unnamed or named module?
The user decides, not the maintainer!

Modularization strategies

Three strategies emerge:

  • bottom up

  • top down

  • inside out

Bottom up

Works best for projects without
unmodularized dependencies.

  • pick a JAR at bottom of dependency tree

  • turn it into a module

  • put it and all dependencies on module path

  • continue with siblings or parent

(Modular JARs still work on to the class path,
so users are free to put them on any path.)

Top down

Good approach for projects with
unmodularized dependencies

  • pick JAR at top of dependency tree

  • turn it into a module
    (require explicit and auto modules)

  • put it on the module path

  • continue with children

Where to put dependencies?

Top down

Where to put dependencies?

Direct dependencies:

Put all on the module path.

Transitive dependencies:

Put only those required by explicit modules
on the module path (transitively).

Top down

When dependencies get modularized:

  • if the name changed, update declarations

  • were they already on the module path?

    • yes ⇝ nothing changes for them

    • no ⇝ move them there

  • check their dependencies

Top down

If you publish modules, be careful
with automatic module names:

  • automatic module name may
    be based on JAR name

  • file names can differ
    across build environments

  • module name can change
    when project gets modularized

⇝ Such automatic module names are unstable.

Top down

Manifest entry:

  • projects can publish module name with
    manifest entry AUTOMATIC-MODULE-NAME

  • assumption is that it won’t change
    when project gets modularized

  • that makes these names stable

⇝ It is ok to publish modules
that depend on automatic modules
whose names are based on manifest entry.

Inside out

Bottom up and top down can be combined:

  • pick a JAR anyhwere in your dependency tree

  • turn it into a module

  • put it and all dependencies on module path

  • place transitive dependencies as for top down

  • continue with any other JAR

Recommendations

In order of precedence:

  1. no unmodularized dependencies ⇝ bottom up

  2. bottom up or top down

  3. roadblocks ⇝ continue elsewhere (inside out)

Remember, partial modularization brings partial benefits!

Java Modules In Real Life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where’s the exit?!
When (not) to use modules?

Roadblocks

Before we look at specific situations:

  • most problems originate in dependencies

  • often stem from automatic modules

  • can often be fixed by demoting them to class path

Automatic Culprits

Many problems come from JARs on the module path
that aren’t ready to be modules.

Minimize number of automatic modules!

Only put on module path:

  • your modular JARs

  • the JARs required by modular
    JARs on the module path

That deals with most transitive dependencies.

Automatic Culprits

If your code directly depends on
a troublesome automatic module:

  • put problematic JARs on class path

  • subprojects that depend on them:

    • do not modularize

    • define automatic module name

    • put on module path

⇝ Modularize elsewhere.

Roadblocks

Some common or tricky roadblocks:

  • split packages

  • broken module descriptors

  • reflective access

  • unsupported media type

More details on GitHub:
nipafx/module-system-woes

Split packages

The module system requires each package
to belong to exactly one module.

# compiler
error: package exists in another module: ...
	package ...;

#run time — between two modules
Error occurred during initialization of boot layer
java.lang.reflect.LayerInstantiationException:
	Package ... in both module ... and module ...

#run time — between module and class path
Exception in thread "main" NoClassDefFoundError: ...
	at ...

Split package solution

The (only) solution:
Ask maintainers to…​

  • reorganize packages across JARs/modules, or

  • provide uber JAR that contains all code

Split package workarounds

Workarounds exist:

  • merge the modules

  • create a bridge module

  • use the unnamed module

  • patch the module

Merging modules

Projects that don’t publish artifacts
can merge the splitting JARs:

  • create a subproject that…​

    • depends on the splitting JARs

    • merges them (e.g. shading in Maven)

    • maybe contains a module descriptor

  • modular code depends on that subproject

Alternatively, set the subproject up separately
and install the JAR in your local Nexus.

Creating a bridge

Projects that can encapsulate their use
of splitting JARs in one subproject,
can make that a bridge:

  • create a subproject that…​

    • depends on the splitting JARs

    • contains all code that uses them

    • becomes an automatic module
      (i.e. no module descriptor!)

  • modular code depends on that subproject

  • put splitting JARs on class path

More workarounds

The other workarounds (not shown here):

  • manipulate dependencies
    with command line flags

  • lead to IDE errors in projects
    that directly depend on them

Work best for transitive dependencies.

Reflective access

Reflection no longer "just works".

Exception in thread "main" InaccessibleObjectException:
	Unable to make ... accessible:
	module ... does not "opens ..." to module ...

Reflective access

Solution:

Open packages for reflection
in module declaration.

Workaround:

Open packages for reflection
at launch with --add-opens.

In module declaration

Analyze which parts of your code
need to be reflected over, e.g.:

  • Spring controllers

  • JPA entities

  • classes for JSON or XML

Open packages in module declaration:

module com.example.app {
	opens com.example.app.controllers;
	opens com.example.app.json;
}

In module declaration

Consider only opening packages
to the modules that reflect:

module com.example.app {
	opens com.example.app.controllers
		to spring.beans, spring.core, spring.context;
	opens com.example.app.json
		 to com.fasterxml.jackson.databind;
}

  • better security

  • better documentation

At launch

For access to modules you don’t create:

java --add-opens
	com.example.lib/com.example.lib.values=$MODULE

Where $MODULE is:

  • the name of the reflecting module

  • ALL-UNNAMED for reflection from class path

Guesswork

Dependencies may not report errors from reflection.

For quick experiments, open your modules:

open module com.example.app {
	// no more `opens` directives
}

If error vanishes, it was an issue with reflection.

Unsupported media type

Projects that aren’t prepared for modules:

  • can have various run-time issues

  • sometimes react poorly by
    hiding the underlying cause

⇝ Search the log for module-related errors.

Searching the log

Search terms for module system errors:

  • "module", "lang.module", "module path"

  • "layer", "boot layer"

  • "visible", "exported", "public", "illegal", "access"

Sometimes, projects just swallow errors. 😔

⇝ Take the module system out of the equation.

Suspending modules

Everything* that works on the module path
also works on the class path.

(* except services in module-info.java)

When debugging a weird error:

Healing the world

Two categories of problems in dependencies:

  • they do something they shouldn’t

  • they don’t tell you that
    you need to do something

Such cases need to be fixed on their end!

⇝ Makes the Java ecosystem more reliable for everybody.

Java Modules In Real Life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where’s the exit?!
When (not) to use modules?

Getting out

If roadblocks are insurmountable,
getting out is easy:

find . -name module-info.java -type f -delete

Preparations

But first:

  • move documentation elsewhere

  • declare services the old way

  • update command line options
    (e.g. --add-exports and paths)

  • move from jlink to jpackage

Lost work

What about all the work?!

Some benefits remain:

  • good while it lasted

  • cleaner architecture

  • documentation

Getting out

Not a great step,
but at least it’s possible!

⇝ Makes it feasible to experiment.

Java Modules In Real Life

Why use modules?
Incremental modularization
What are common or tricky roadblocks?
Where’s the exit?!
When (not) to use modules?

Observations & Assumptions

Observations:

  • modules require & enforce a decent architecture

  • (re)factoring modules << (re)factoring code

  • most problems originate in dependencies

Assumption:

  • barring problematic dependencies,
    modules are a net-positive

Those damn dependencies!

Why are dependencies so often the problem?

  • almost all still baseline against JDK ≤ 8

  • makes creating/maintaining modules tougher

  • number of modularized apps is small
    and thus demand is small (but growing)

But more and more ship as modules:
sormuras/modules tracks 4294 unique modules
(as of July 27th, 2022).

Project size

What impact does project size have?

Simplified:

SizeEffortBenefit

small

📉

📉

large

📈

📈

⇝ Project size doesn’t matter (much).

Project type

General benefits for all kinds of projects:

  • managing dependencies

  • preventing (accidental) use
    of internal (JDK) APIs

  • new abstraction

  • etc.

Project type

Frameworks and libraries:

  • benefit from strong encapsulation

  • shouldn’t limit users

Applications benefit from:

  • strong encapsulation (!)

  • self-contained images (possibly)

⇝ Project type has little practical impact,
but reused code should consider users.

Project age

New projects:

  • module declarations and architecture
    evolve side by side

  • new dependencies can be vetted

Existing projects:

  • modules must be retrofitted

  • architecture may require updating
    (which can be the goal!)

  • existing dependencies must work

⇝ New projects are a better fit.

Domain (knowledge)

Impact of domain (knowledge) on dependencies:

  • maintenance of domain-specific dependencies?

  • choice of domain-specific dependencies?

  • understanding of domain to evaluate 👆🏾
    and to analyze unexpected problems

⇝ Easier if domain isn’t niche
and you understand it well.

When (not)?

Best place to start:

  • new projects

  • in well-understood,
    well-maintained domain

  • particularly if reused

Next best place: your current project!
(Unless it has lots of outdated dependencies.)

How to

Do:

  • Accept that partial modularization brings
    partial benefits. Then start small.

  • Use module declarations to analyze, guide,
    document, and review architecture.

Don’t:

  • Get stuck trying to fix dependencies:
    identify root cause, open an issue,
    put on class path, wait for (or contribute) fix.

  • Forget that modules are a seat belt, not a rocket.

Everything will be okay in the end. If it’s not okay, it’s not the end.

— John Lennon

So long…​

37% off with
code fccparlog

bit.ly/the-jms

More

Slides at slides.nipafx.dev
⇜ Get my book!

Follow Nicolai

nipafx.dev
/nipafx

Follow Java

inside.java // dev.java
/java    //    /openjdk

Image Credits