Feed multiple parameters in JUnit 5 test method

Question

I want to provide my codebase with "polymorphic" test cases. Specifically, there are going to be multiple implementations of a Graph interface and would like to reuse the test code for all of them ( ALGraph , AMGraph , ...).

I'd like to develop my test methods along the following lines

    @ParameterizedTest
    @MethodSource("graphFactory")
    // Note: JUnit 5 won't allow the following additional argument source
    @ValueSource(ints = {0, 31415, -31415})
    void testInsertDeleteNode(Graph g, Integer v) {
        g.insertNode(new Node<>(v));
        assertTrue(g.containsNode(new Node<>(v)));
        assertEquals(1, g.vertices().size());

        g.deleteNode(new Node<>(v));
        assertFalse(g.containsNode(new Node<>(v)));
        assertEquals(0, g.vertices().size());
    }

but the way JUnit is built is preventing me from accomplishing this scheme.

So basically I'd like to provide a cartesian product of multiple arguments to my tests. Is that possible with the out-of-the-box argument providers ( ValueSource , NullSource , ...) or do I forcibly need to set up customized ones with the aid of @MethodSource ?

Answer 1

Update 2020-10-07 via beatngu13

Since v1.0.0, @CartesianProductTest is available as part of the JUnit Pioneer extension pack for JUnit 5. Find details at https://junit-pioneer.org/docs/cartesian-product

JUnit 5 Sample

It's not supported out of the box -- but there already exists a feature request at https://github.com/junit-team/junit5/issues/1427

Find an example and proof of concept solution here: https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-extensions

@CartesianProductTest({"0", "1"})
void threeBits(String a, String b, String c) {
    int value = Integer.parseUnsignedInt(a + b + c, 2);
    assertTrue((0b000 <= value) && (value <= 0b111));
}

@CartesianProductTest
@DisplayName("S ⨯ T ⨯ U")
void nFold(String string, Class<?> type, TimeUnit unit, TestInfo info) {
    assertTrue(string.endsWith("a"));
    assertTrue(type.isInterface());
    assertTrue(unit.name().endsWith("S"));
    assertTrue(info.getTags().isEmpty());
}

static CartesianProductTest.Sets nFold() {
    return new CartesianProductTest.Sets()
            .add("Alpha", "Omega")
            .add(Runnable.class, Comparable.class, TestInfo.class)
            .add(TimeUnit.DAYS, TimeUnit.HOURS);
}

Yields a test plan like:

Answer 2

In your example you might also consider using a property-based testing framework like jqwik . The first way to use it would be very similar to your example:

import net.jqwik.api.*;
import static org.junit.jupiter.api.Assertions.*;

@Property(generation = GenerationMode.EXHAUSTIVE)
void testInsertDeleteNode(
        @ForAll("graphFactory") Graph g, 
        @ForAll("ints") Integer v) 
{
    g.insertNode(new Node<>(v));
    assertTrue(g.containsNode(new Node<>(v)));
    assertEquals(1, g.vertices().size());

    g.deleteNode(new Node<>(v));
    assertFalse(g.containsNode(new Node<>(v)));
    assertEquals(0, g.vertices().size());
}

@Provide
Arbitrary<Graph> graphFactory() {
    return Arbitraries.of(new ALGraph(), new AMGraph());
}

@Provide
Arbitrary<Integer> ints() {
    return Arbitraries.of(0, 31415, -31415);
}

By using generation = GenerationMode.EXHAUSTIVE you tell the engine to generate the cartesian product of possible parameter values. In fact, if the number of combinations is <= 1000 jqwik would go for the cartesion product all by itself.

Depending on the number of differen Graph implementations you could also consider an approach that uses concrete subclasses for each implementation and specify the tests themselves in an interface (or an abstract superclass):

interface GraphTest<G extends Graph> {

    G createGraph();

    @Property(generation = GenerationMode.EXHAUSTIVE)
    default void testInsertDeleteNode(@ForAll("ints") Integer v) {
        Graph g = createGraph();
        g.insertNode(new Node<>(v));
        assertTrue(g.containsNode(new Node<>(v)));
        assertEquals(1, g.vertices().size());

        g.deleteNode(new Node<>(v));
        assertFalse(g.containsNode(new Node<>(v)));
        assertEquals(0, g.vertices().size());
    }

    @Provide
    default Arbitrary<Integer> ints() {
        return Arbitraries.of(0, 31415, -31415);
    }
}

class ALGraphTest implements GraphTest<ALGraph> {
    @Override
    public ALGraph createGraph() {
        return new ALGraph();
    }
}

class AMGraphTest implements GraphTest<AMGraph> {
    @Override
    public AMGraph createGraph() {
        return new AMGraph();
    }
}

Since you're already using the JUnit 5 platform, using jqwik requires a single additional dependency .