Chapter 2. Basic concepts of Acceptance and Regression Testing

To get the most out of Thucydides, it is useful to understand some of the basic principles behind Acceptance Test Driven Development. Thucydides is commonly used for both Automated Acceptance Tests and Regression tests, and the principles discussed here apply, with minor variations, to both.

Acceptance Test Driven Development, or ATDD, is an advanced form of Test Driven Development (TDD) in which automated acceptance criteria — defined in collaboration with users — drive and focus the development process. This helps ensure that everyone understands what features are under development.

One of the important things about ATDD is the idea of "Specification by Example". Specification by Example refers to the use of relatively concrete examples to illustrate how a system should work, as opposed to more formally written specifications.

Let’s look at an example. In many projects, requirements are expressed as simple stories along the following lines:

In order to earn money to buy a new car
As a car owner
I want to sell my old car online

If we were implementing an online car sales web site that helps people achieve this goal, we would typically define a set of acceptance criteria to flesh out this story. For example, we might have the following criteria in our list of acceptance criteria:

and so on.

A tester planning the tests for these acceptance criteria might draw up a test plan outlining of the way she expects to test these criteria. For the first criteria, she might start off with a high-level plan like the following:

Each of these steps might need to be broken down into smaller steps.

These steps are often fleshed out with more concrete details:

For our purposes, Regression Tests can be defined as end-to-end tests that ensure that an application behaves as expected, and that it continues to behave as expected in future releases. Whereas ATDD Acceptance Tests are defined very early on in the piece, before development starts, Regression Tests involve an existing system. Other than that, the steps involved in defining and automating the tests are very similar.

Now different project stakeholders will be interested in different levels of detail. Some, such as project managers and management in general, will be interested only in which application features work, and which need to be done. Others, such as business analysts and QA, will be interested in the details of how each acceptance scenario is implemented, possible down to the screen level.

Thucydides helps you structure your automated acceptance tests into steps and sub-steps like the ones illustrated above. This tends to make the tests clearer, more flexible and easier to maintain. In addition, when the tests are executed, Thucydides produces illustrated, narrative-style reports like the one in Figure 2.1, “A test report generated by Thucydides”.

figs/test-report.png

Figure 2.1. A test report generated by Thucydides


When it comes to implementing the tests themselves, Thucydides also provides many features that make it easier, faster and cleaner to write clear, maintainable tests. This is particularly true for automated web tests using Selenium 2, but Thucydides also caters for non-web tests as well. Thucydides currently works well with JUnit and easyb - integration with other BDD frameworks is in progress.