Skip to content

 
  • Test-Driven Development Training and Coaching
  • Automated Acceptance and Web Testing
  • Test-Driven Development Training and Coaching
  • Test-Driven Development Training and Coaching
  • Automated Acceptance and Web Testing
  • Test-Driven Development Training and Coaching
  • Continuous Integration and Continuous Delivery with Jenkins
  • Expertise in Automated Acceptance Tests and ATDD

Data-driven unit testing in Java

Data-driven testing is a powerful way of testing a given scenario with different combinations of values. In this article, we look at several ways to do data-driven unit testing in JUnit.

Suppose, for example, you are implementing a Frequent Flyer application that awards status levels (Bronze, Silver, Gold, Platinum) based on the number of status points you earn. The number of points needed for each level is shown here:

level

minimum status points

result level

Bronze

0

Bronze

Bronze

300

Silver

Bronze

700

Gold

Bronze

1500

Platinum

Our unit tests need to check that we can correctly calculate the status level achieved when a frequent flyer earns a certain number of points. This is a classic problem where data-driven tests would provide an elegant, efficient solution.

Data-driven testing is well-supported in modern JVM unit testing libraries such as Spock and Spec2. However, some teams don’t have the option of using a language other than Java, or are limited to using JUnit. In this article, we look at a few options for data-driven testing in plain old JUnit.

Parameterized Tests in JUnit

JUnit provides some support for data-driven tests, via the Parameterized test runner. A simple data-driven test in JUnit using this approach might look like this:

@RunWith(Parameterized.class)
public class WhenEarningStatus {

    @Parameters(name = "{index}: {0} initially had {1} points, earns {2} points, should become {3} ")
    public static Iterable<Object[]> data() {
        return Arrays.asList(new Object[][]{
                {Bronze, 0,    100,  Bronze},
                {Bronze, 0,    300,  Silver},
                {Bronze, 100,  200,  Silver},
                {Bronze, 0,    700,  Gold},
                {Bronze, 0,    1500, Platinum},
        });
    }

    private Status initialStatus;
    private int initialPoints;
    private int earnedPoints;
    private Status finalStatus;

    public WhenEarningStatus(Status initialStatus, int initialPoints, int earnedPoints, Status finalStatus) {
        this.initialStatus = initialStatus;
        this.initialPoints = initialPoints;
        this.earnedPoints = earnedPoints;
        this.finalStatus = finalStatus;
    }

    @Test
    public void shouldUpgradeStatusBasedOnPointsEarned() {
        FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678")
                                            .named("Joe", "Jones")
                                            .withStatusPoints(initialPoints)
                                            .withStatus(initialStatus);

        member.earns(earnedPoints).statusPoints();

        assertThat(member.getStatus()).isEqualTo(finalStatus);
    }
}

You provide the test data in the form of a list of Object arrays, identified by the _@Parameterized@ annotation. These object arrays contain the rows of test data that you use for your data-driven test. Each row is used to instantiate member variables of the class, via the constructor.

When you run the test, JUnit will instantiate and run a test for each row of data. You can use the name attribute of the @Parameterized annotation to provide a more meaningful title for each test.

There are a few limitations to the JUnit parameterized tests. The most important is that, since the test data is defined at a class level and not at a test level, you can only have one set of test data per test class. Not to mention that the code is somewhat cluttered - you need to define member variables, a constructor, and so forth.

Fortunatly, there is a better option.

Using JUnitParams

A more elegant way to do data-driven testing in JUnit is to use [https://code.google.com/p/junitparams/|JUnitParams]. JUnitParams (see [http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22JUnitParams%22|Maven Central] to find the latest version) is an open source library that makes data-driven testing in JUnit easier and more explicit.

A simple data-driven test using JUnitParam looks like this:

@RunWith(JUnitParamsRunner.class)
public class WhenEarningStatusWithJUnitParams {

    @Test
    @Parameters({
            "Bronze, 0,   100,  Bronze",
            "Bronze, 0,   300,  Silver",
            "Bronze, 100, 200,  Silver",
            "Bronze, 0,   700,  Gold",
            "Bronze, 0,   1500, Platinum"

    })
    public void shouldUpgradeStatusBasedOnPointsEarned(Status initialStatus, int initialPoints, 
                                                       int earnedPoints, Status finalStatus) {
        FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678")
                                            .named("Joe", "Jones")
                                            .withStatusPoints(initialPoints)
                                            .withStatus(initialStatus);

        member.earns(earnedPoints).statusPoints();

        assertThat(member.getStatus()).isEqualTo(finalStatus);
    }
}

Test data is defined in the @Parameters annotation, which is associated with the test itself, not the class, and passed to the test via method parameters. This makes it possible to have different sets of test data for different tests in the same class, or mixing data-driven tests with normal tests in the same class, which is a much more logical way of organizing your classes.

JUnitParam also lets you get test data from other methods, as illustrated here:

    @Test
    @Parameters(method = "sampleData")
    public void shouldUpgradeStatusFromEarnedPoints(Status initialStatus, int initialPoints, 
                                                    int earnedPoints, Status finalStatus) {
        FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678")
                .named("Joe", "Jones")
                .withStatusPoints(initialPoints)
                .withStatus(initialStatus);

        member.earns(earnedPoints).statusPoints();

        assertThat(member.getStatus()).isEqualTo(finalStatus);
    }

    private Object[] sampleData() {
        return $(
                $(Bronze, 0,   100, Bronze),
                $(Bronze, 0,   300, Silver),
                $(Bronze, 100, 200, Silver)
        );
    }

The $ method provides a convenient short-hand to convert test data to the Object arrays that need to be returned.

You can also externalize

    @Test
    @Parameters(source=StatusTestData.class)
    public void shouldUpgradeStatusFromEarnedPoints(Status initialStatus, int initialPoints, 
                                                    int earnedPoints, Status finalStatus) {
        ...
    }

The test data here comes from a method in the StatusTestData class:

    public class StatusTestData {
        public static Object[] provideEarnedPointsTable() {
            return $(
                    $(Bronze, 0,   100, Bronze),
                    $(Bronze, 0,   300, Silver),
                    $(Bronze, 100, 200, Silver)
            );
        }
    }

This method needs to be static, return an object array, and start with the word "provide".

Getting test data from external methods or classes in this way opens the way to retrieving test data from external sources such as CSV or Excel files.

JUnitParam provides a simple and clean way to implement data-driven tests in JUnit, without the overhead and limitations of the traditional JUnit parameterized tests.

Testing with non-Java languages

If you are not constrained to Java and/or JUnit, more modern tools such as Spock (https://code.google.com/p/spock/) and Spec2 provide great ways of writing clean, expressive unit tests in Groovy and Scala respectively. In Groovy, for example, you could write a test like the following:

class WhenEarningStatus extends Specification {

    def "should earn status based on the number of points earned"() {
        given:
        def member = FrequentFlyer.withFrequentFlyerNumber("12345678")
                .named("Joe", "Jones")
                .withStatusPoints(initialPoints)
                .withStatus(initialStatus);

        when:
        member.earns(earnedPoints).statusPoints()

        then:
        member.status == finalStatus

        where:
        initialStatus | initialPoints | earnedPoints | finalStatus
        Bronze        | 0             | 100          | Bronze
        Bronze        | 0             | 300          | Silver
        Bronze        | 100           | 200          | Silver
        Silver        | 0             | 700          | Gold
        Gold          | 0             | 1500         | Platinum
    }
}

John Ferguson Smart is a specialist in BDD, automated testing, and software life cycle development optimization, and author of BDD in Action and other books. John runs regular courses in Australia, London and Europe on related topics such as Agile Requirements Gathering, Behaviour Driven Development, Test Driven Development, and Automated Acceptance Testing.

Add a comment

BDD Requirements Management with JBehave, Thucydides and JIRA - Part 2

Thucydides is an open source library designed to make practicing Behaviour Driven Development easier. Thucydides plays nicely with BDD tools such as JBehave, or even more traditional tools like JUnit, to make writing automated acceptance tests easier, and to provide richer and more useful living documentation. In this series of articles, we look at the tight one and two-way integration that Thucydides offers with JIRA. The first article discussed basic one-way integration with JIRA. In this article, we will looking at taking that integration further. We will see how to insert links to the Thucydides reports into JIRA, how to update the state of JIRA issues based on the Thucydides test outcomes, and how to report on JIRA versions and releases in the Thucydides reports.

Add a comment

BDD Requirements Management with JBehave, Thucydides and JIRA - Part 1

Thucydides is an open source library designed to make practicing Behaviour Driven Development easier. Thucydides plays nicely with BDD tools such as JBehave, or even more traditional tools like JUnit, to make writing automated acceptance tests easier, and to provide richer and more useful living documentation. In a series of two articles, we will look at the tight one and two-way integration that Thucydides offers with JIRA.

The rest of this article assumes you have some familiarily with Thucydides. For a tutorial introduction to Thucydides, check out the Thucydides Documentation or this article for a quick introduction.

Add a comment

Running parallel acceptance tests using JBehave, Thucydides and Bamboo

This article was writen by Simeon Ross, who works in a large government organization. In it, he describes how he sets up parallel testing in JBehave and Thucydides using Bamboo. Simeon has also put a sample project illustrating the approach on Github

The place that I work for has a data mart project pulling from many sources and it must be accurate. To that end we decided to write a number of JBehave style, data driven JBehave tests utilising Thucydides to ensure its integrity. This was a success until the data grew and the tests were taking too long to complete - some of our stories had massive example tables! While we could speed up the process by tuning the database with indexes we decided that it since it was still in development it could be time wasted if the data/approach was wrong and that wasn't getting away from the issue of the sheer number of tests so we came to the conclusion that the first attack would be running the tests in parallel.

Add a comment

It ain't just reds and greens: Automated Acceptance Testing and quaternary test outcomes

Although they seem simple enough on the surface, test outcomes are actually quite complicated beasts. Traditional unit tests, and basic TDD tests, have just two states, passing or failing, represented by red and green in the famous "RED-GREEN-REFACTOR" dicton. In Behaviour Driven Development (BDD), on the other hand, we have the additional concept of 'pending' tests: tests that have been specified (for example, in a Cucumber or JBehave story) but not yet implemented. When we report on test results, we need to be able to distinguish these three states, as a pending test has very different semantics to a failing test. Pending means it's not yet done yet, but this may well be as expected, especially towards the start of a sprint. A failing test, on the other hand, needs fixing. Now.

Add a comment

Learning Corner


Upcoming courses


Upcoming Events