JUnit 5

Next Generation Testing
on the JVM

Nicolai Parlog

codefx.org / @nipafx

ETC

#EuroTestConf

Heads Up

JUnit 5 is work in progress!

This is based on Milestone 3
(released 30th of November 2016).

Give feedback!

JUnit 5 Links:

GitHub repository

github.com/junit-team/junit5

User guide

junit.org/junit5/docs/snapshot/user-guide

Twitter

@junitteam

Articles on my blog

blog.codefx.org/tag/junit-5/

Public Service Announcement

This is the short version of the talk!

Find the slides under
slides.codefx.org

You will also find longer versions there.
(Pick one from 2017.)

Basics

Basics
Extensions
Architecture
Tools & Setup

Declaring Tests

class JUnit5Test {

	@Test
	void someTest() {
		assertTrue(true);
	}

}

Disabling Tests

@Test
@Disabled("Y U No Pass?!")
void failingTest() {
	assertTrue(false);
}

Seamless Extensions

@Test
@DisabledOnFriday
void failingTest() {
	assertTrue(false);
}

Lifecycle Methods

@BeforeAll
static void beforeAll() { ... }

@BeforeEach
void beforeEach() { ... }

@AfterEach
void afterEach() { ... }

@AfterAll
static void afterAll() { ... }

Basic Assertions

@Test
void someTest() {
	...
	assertEquals(
		expected,
		actual,
		"Should be equal.");
}

Advanced Assertions

@Test
void assertAllProperties() {
	Address ad = new Address(
	  "City", "Street", "42");

	assertAll("address",
	  () -> assertEquals("C", ad.city),
	  () -> assertEquals("Str", ad.street),
	  () -> assertEquals("63", ad.number)
	);
}

Assert All

Output if assertAll fails:

org.opentest4j.MultipleFailuresError:
	address (3 failures)
	expected: <C> but was: <City>
	expected: <Str> but was: <Street>
	expected: <63> but was: (42)

Nesting Tests

class CountTest {
	// lifecycle and tests
	@Nested
	class CountGreaterZero {
		// lifecycle and tests
		@Nested
		class CountMuchGreaterZero {
			// lifecycle and tests
		}
	}
}

Naming Tests

@DisplayName("A count")
class CountTest {
	@Nested
	@DisplayName("when greater zero")
	class CountGreaterZero {
		@Test
		@DisplayName("is positive")
		void isPositive() { ... }
	}
}

Nesting And Naming

The effects of @Nested and @DisplayName:

Nested and DisplayName

Tests Parameters

@Test
void someTest(MyServer server) {
	// do something with `server`
}

JUnit 5 Basics

Summary

  • test declaration much like before

  • lifecycle much like before

  • many details were improved

  • @Nested and @DisplayName
    make a nice couple

  • parameter injection

That's all very nice but how is it
Next Generation Testing?

Extensions

Basics
Extensions
Architecture
Tools & Setup

Approach To Extension

From JUnit 5’s Core Principles:

Prefer extension points over features

Quite literally,
JUnit 5 has Extension Points

Extension Points

  • Test Instance Post Processor

  • BeforeAll Callback

  • Test and Container Execution Condition

  • BeforeEach Callback

  • Parameter Resolution

  • Before Test Execution

  • After Test Execution

  • Exception Handling

  • AfterEach Callback

  • AfterAll Callback

Implementing Extensions

  • one interface for each extension point

  • method arguments capture context

public interface BeforeEachCallback
		extends Extension {

	void beforeEach(
		TestExtensionContext context);
}

  • an extension can use multiple points
    to implement its feature

Benchmark Extension

We want to benchmark our tests!

  • for each test method

  • write the elapsed time to console

How?

  • before test execution: store test launch time

  • after test execution: print elapsed time

Benchmark Extension

public class BenchmarkExtension implements
		BeforeTestExecutionCallback,
		AfterTestExecutionCallback {

	private long launchTime;

	// ...
}

Benchmark Extension

@Override
public void beforeTestExecution(
		TestExtensionContext context) {
	launchTime = currentTimeMillis();
}

@Override
public void afterTestExecution(
		TestExtensionContext context) {
	printf("Test '%s' took %d ms.%n",
		context.getDisplayName(),
		currentTimeMillis() - launchTime);
}

Other Examples

Remember This?

@Test
@DisabledOnFriday
void failingTest() {
	assertTrue(false);
}

Let’s see how it works!

Disabled Extension

public class DisabledOnFridayCondition
		implements TestExecutionCondition {
	@Override
	public ConditionEval.Result evaluate(
			TestExtensionCtx. context) {
		if (isFriday())
			return disabled("Weekend!");
		else
			return enabled("Fix it!");
	}
}

Other Examples

What about parameter injection?

@Test
void someTest(MyServer server) {
	// do something with `server`
}

Parameter Injection

public class MyServerParameterResolver
		implements ParameterResolver {
	@Override
	public boolean supports(
			ParameterContext p, ...) {
		return MyServer.class
			== p.getParameter().getType();
	}
	@Override
	public Object resolve( ... ) {
		return new MyServer();
	}
}

Applying Extensions

How do we apply extensions?

@ExtendWith(DisabledOnFridayCondition.class)
class JUnit5Test {
	...
}

That’s technical and verbose…​ :(

Applying Extensions

Meta-annotations to the rescue!

  • JUnit 5’s annotations are meta-annotations

  • JUnit 5 checks recursively for annotations

⇝ We can create our own annotations!

Creating Annotations

@ExtendWith(DisabledOnFridayCondition.class)
public @interface DisabledOnFriday { }

@Test
@Tag("integration")
@ExtendWith(BenchmarkExtension.class)
@ExtendWith(MyServerParameterResolver.class)
public @interface IntegrationTest { }

@IntegrationTest
@DisabledOnFriday
void testLogin(MyServer server) { ... }

Extensions

Summary

  • flexibility because of many extension points

  • extensions compose well

  • customizable due to meta-annotations

(We left out some details.)

That's all very nice but how is it
Next Generation Testing?

Architecture

Basics
Extensions
Architecture
Tools & Setup

JUnit 4 Architecture

  • a single JAR (ignoring Hamcrest)

  • used by

    • developers

    • extensions

    • IDEs, build-tools

  • no separation of concerns

JUnit 4 Architecture

  • tools provide us with awesome features!

  • but API is not powerful enough

I know, I’ll use reflection!

  • nothing was safe!

  • bound tools to implementation details

  • made maintenance and evolution very hard

Dead End

Part of JUnit’s success is its great tool support!

But the same tools locked development in.

The success of JUnit as a platform prevents the development of JUnit as a tool.
(Johannes Link)

Approach in JUnit 5

Separation of concerns:

  1. an API to write tests against

  2. a mechanism to discover and run tests

  3. an API for tools to run tests

Approach in JUnit 5

Separation of concerns V 2.0:

  1. an API to write tests against

  2. a mechanism to discover and run tests

    1. specific engine per variant of tests
      (e.g. JUnit 4 or JUnit 5)

    2. orchestration of engines

    3. API between them

  3. an API for tools to run tests

JUnit 5 Modules

architecture limited lean

Subprojects & Modules

JUnit Jupiter 5.0.0-M3

  • junit-jupiter-api

  • junit-jupiter-engine

JUnit Vintage 4.12.0-M3

  • junit-vintage-engine

JUnit Platform 1.0.0-M3

  • junit-platform-engine

  • junit-platform-runner

  • a lot more

Architecture

Summary

  • clear separation of concerns

  • API for developers

  • API for tools

That's all very nice but how is it
Next Generation Testing?

Because it opens up the platform!

Moar Engines!

  • want to run JUnit 4 tests?
    ⇝ create an engine for it

  • want TestNG to have support like JUnit?
    ⇝ create an engine for it

  • want to write tests in natural language?
    ⇝ create an engine for it

Moar Engines!

architecture lean

Open Platform

Once JUnit 5 adoption sets in:

  • tools are decoupled from implementation details

  • tools can support all frameworks (almost) equally well

  • new frameworks start with full tool support

  • developers can try out new things

A new generation of test frameworks might arise!

Open Platform

JUnit’s success as a platform
becomes available to everybody.

This heralds the
next generation of testing on the JVM!

Architecture

Summary

  • clear separation of concerns:
    APIs for developers, tools,
    and new frameworks

  • opens up the platform

  • tool support for everybody!

(There’s even more to the story.)

Tools & Setup

Basics
Extensions
Architecture
Tools & Setup

Writing Tests

As Easy As Pie!

Add this:

org.junit.jupiter
junit-jupiter-api
5.0.0-M3

Have fun!

Running Tests

Native support is w.i.p.

  • Maven: work started #1206

  • Gradle: work started #1037

  • IntelliJ: since 2016.2; 2016.3.1 required for M3

  • Eclipse: EA support with 4.7 M4
    with some extra work #488566

  • NetBeans: not even an issue

Running Tests

As Part Of JUnit 4

  • individual classes:

    @RunWith(JUnitPlatform.class)
public class JUnit5Test { ... }

  • all classes:

    @RunWith(JUnitPlatform.class)
@SelectPackages({ "my.test.package" })
public class JUnit5TestSuite { }

Running Tests

With Build Tools

JUnit 5 team provides rudimentary
Gradle plugin and Maven Surefire provider
(see user guide for details)

Running Tests

From Console

There is a console launcher:

# run all tests
junit-platform-console \
	-cp ${path_to_compiled_test_classes} \
	--scan-class-path
# run a specific test
junit-platform-console \
	-cp ${path_to_compiled_test_classes} \
	--select-class \
	${fully_qualified_test_class_name}

Tools & Setup

Summary

  • you can start writing tests right away

  • only IntelliJ has native support

  • running with JUnit 4 is a good compromise

(Read about the setup details.)

Next Generation Testing On The JVM

  • new API is an incremental improvement
    full of thoughtful details

  • dynamic tests are very useful

  • extension model looks very promising

  • architecture opens up the platform

  • tool support is not there yet

(Read more about JUnit 5.)

Questions?

Find Me

codefx.org
sitepoint.com/java
@nipafx / +NicolaiParlog

Want More?

one-day course Expert Java 8
Voxxed Days Zürich, 22.02.

or buy my book about Java 9

or hire me!

Image Credits