Chapter 16. Running Thucydides tests in parallel batches

Web tests make good candidates for concurrent testing, in theory at least, but the implementation can be tricky. For example, although it is easy enough to configure both JUnit and easyb to run tests in parallel, running several webdriver instances of Firefox in parallel on the same display, for example, tends to become unreliable.

The natural solution in this case is to split the web tests into smaller batches, and to run each batch on a different machine and/or on a different virtual display. When each batch has finished, the results can be retrieved and aggregated into the final test reports.

However splitting tests into batches by hand tends to be tedious and unreliable – it is easy to forget to add a new test to a batch, for example, or have unevenly-distributed batches.

The latest version of Thucydides lets you do this automatically, by splitting your test cases evenly into batches of a given size. In practice, you run a build job for each batch. You need to specify two parameters when you run each build: the total number of batches being run (thucydides.batch.count), and the number of the batch being run in this build (thucydides.batch.number).

For example, the following will divide the test cases into 3 batches (thucydides.batch.count), and only run the first test in each batch (thucydides.batch.number):

mvn verify -Dthucydides.batch.count=3 -Dthucydides.batch.number=1

This will only work with the JUnit integration. However this feature is also supported in easyb (as of easyb version 1.5), though using different parameters. When using the Thucydides easyb integration, you also need to provide the equivalent options for easyb:

mvn verify -Deasyb.batch.count=3 -Deasyb.batch.number=1

If you have both easyb and JUnit Thucydides tests, you will need to specify both options.