An Introduction to BDD Test Automation with Serenity and JUnit

Serenity BDD (previously known as Thucydides) is an open source reporting library that helps you write better structured, more maintainable automated acceptance criteria, and also produces rich meaningful test reports (or "living documentation") that not only report on the test results, but also what features have been tested. And for when your automated acceptance tests exercise a web interface, Serenity comes with a host of features that make writing your automated web tests easier and faster.

1. BDD fundamentals

But before we get into the nitty-gritty details, let’s talk about Behaviour Driven Development, which is a core concept underlying many of Serenity’s features. Behaviour Driven Development, or BDD, is an approach where teams use conversations around concrete examples to build up a shared understanding of the features they are supposed to build.

For example, suppose you are building a site where artists and craftspeople can sell their good online. One important feature for such a site would be the search feature. You might express this feature using a story-card format commonly used in agile projects like this:

  In order for buyers to find what they are looking for more efficiently
  As a seller
  I want buyers to be able to search for articles by keywords

To build up a shared understanding of this requirement, you could talk through a few concrete examples. The converstaion might go something like this:

"So give me an example of how a search might work."
"Well, if I search for wool, then I should see only woolen products."
"Sound’s simple enough. Are there any other variations on the search feature that would produce different outcomes?"
"Well, I could also filter the search results; for example, I could look for only handmade woolen products."

And so on. In practice, many of the examples that get discussed become "acceptance criteria" for the features. And many of these acceptance criteria become automated acceptance tests. Automating acceptence tests provides valuable feedback to the whole team, as these tests, unlike unit and integrationt tests, are typically expressed in business terms, and can be easily understood by non-developers. And, as we will se later on in this article, the reports that are produced when these teste are executed give a clear picture of the state of the application.

2. Serenity BDD and JUnit

In this article, we will learn how to use Serenity BDD using nothing more than JUnit, Serenity BDD, and a little Selenium WebDriver. Automated acceptance tests can use more specialized BDD tools such as Cucumber or JBehave, but many teams like to keep it simple, and use more conventional unit testing tools like JUnit. This is fine: the essence of the BDD approach lies in the conversations that the teams have to discuss the requirements and discover the acceptance criteria.

2.1. Writing the acceptance test

Let’s start off with a simple example. The first example that was discussed was searching for wool. The corresponding automated acceptance test for this example in JUnit looks like this:

@RunWith(SerenityRunner.class)                                                   (1)
public class WhenSearchingByKeyword {

    @Managed(driver="chrome", uniqueSession = true)                              (2)
    WebDriver driver;

    @Steps                                                                       (3)
    BuyerSteps buyer;

    @Test
    public void should_see_a_list_of_items_related_to_the_specified_keyword() {  (4)
        // GIVEN
        buyer.opens_etsy_home_page();
        // WHEN
        buyer.searches_for_items_containing("wool");
        // THEN.
        buyer.should_see_items_related_to("wool");
    }
}

1	The Serenity test runner sets up the test and records the test results
2	This is a web test, and Serenity will manage the WebDriver driver for us
3	We hide implementation details about how the test will be executed in a "step library"
4	Our test itself is reduced to the bare essential business logic that we want to demonstrate

There are several things to point out here. When you use Serenity with JUnit, you need to use the SerenityRunner test runner. This instruments the JUnit class and instantiates the WebDriver driver (if it is a web test), as well as any step libraries and page objects that you use in your test (more on these later).

The @Managed annotation tells Serenity that this is a web test. Serenity takes care of instantiating the WebDriver instance, opening the browser, and shutting it down at the end of the test. You can also use this annotation to specify what browser you want to use, or if you want to keep the browser open during all of the tests in this test case.

The @Steps annotation tells Serenity that this variable is a Step Library. In Serenity, we use Step Libraries to add a layer of abstraction between the "what" and the "how" of our acceptance tests. At the top level, the step methods document "what" the acceptance test is doing, in fairly implementation-neutral, business-friendly terms. So we say "searches for items containing wool", not "enters wool into the search field and clicks on the search button". This layered approach makes the tests both easier to understand and to maintain, and helps build up a great library of reusable business-level steps that we can use in other tests.

2.2. The Step Library

The Step Library class is just an ordinary Java class, with methods annotated with the @Step annotation:

public class BuyerSteps {

    HomePage homePage;                                          (1)
    SearchResultsPage searchResultsPage;

    @Step                                                       (2)
    public void opens_etsy_home_page() {
        homePage.open();
    }

    @Step
    public void searches_for_items_containing(String keywords) {
        homePage.searchFor(keywords);
    }

    @Step
    public void should_see_items_related_to(String keywords) {
        List<String> resultTitles = searchResultsPage.getResultTitles();
        resultTitles.stream().forEach(title -> assertThat(title.contains(keywords)));
    }
}
//end:tail

1	Step libraries often use Page Objects, which are automatically instantiated
2	The `@Step` annotation indicates a method that will appear as a step in the test reports

For automated web tests, the step library methods do not call WebDriver directly, but rather they typically interact with Page Objects.

2.3. The Page Objects

Page Objects encapsulate how a test interacts with a particular web page. They hide the WebDriver implementation details about how elements on a page are accessed and manipulated behind more business-friendly methods. Like steps, Page Objects are reusable components that make the tests easier to understand and to maintain.

Serenity automatically instantiates Page Objects for you, and injects the current WebDriver instance. All you need to worry about is the WebDriver code that interacts with the page. And Serenity provides a few shortcuts to make this easier as well. For example, here is the page object for the Home page:

@DefaultUrl("http://www.etsy.com")                      (1)
public class HomePage extends PageObject {              (2)

    @FindBy(css = "button[value='Search']")
    WebElement searchButton;

    public void searchFor(String keywords) {
        $("#search-query").sendKeys(keywords);          (3)
        searchButton.click();                           (4)
    }
}
// end:tail[]

1	What URL should be used by default when we call the `open()` method
2	A Serenity Page Object must extend the `PageObject` class
3	You can use the `$` method to access elements directly using CSS or XPath expressions
4	Or you may use a member variable annotated with the `@FindBy` annotation

And here is the second page object we use:

public class SearchResultsPage extends PageObject {

    @FindBy(css=".listing-card")
    List<WebElement> listingCards;

    public List<String> getResultTitles() {
        return listingCards.stream()
                .map(element -> element.getText())
                .collect(Collectors.toList());
    }
}
// end:tail[]

In both cases, we are hiding the WebDriver implementation of how we access the page elements inside the page object methods. This makes the code both easier to read and reduces the places you need to change if a page is modified.

This approach encourages a very high degree of reuse. For example, the second example mentioned at the start of this article involved filtering results by type. The corresponding automated acceptance criteria might look like this:

    @Test
    public void should_be_able_to_filter_by_item_type() {
        // GIVEN
        buyer.opens_etsy_home_page();
        // WHEN
        buyer.searches_for_items_containing("wool");
        int unfilteredItemCount = buyer.get_matching_item_count();
        // AND
        buyer.filters_results_by_type("Handmade");
        // THEN
        buyer.should_see_items_related_to("wool");
        // AND
        buyer.should_see_item_count(lessThan(unfilteredItemCount));
    }

    @Test
    public void should_be_able_to_view_details_about_a_searched_item() {
        // GIVEN
        buyer.opens_etsy_home_page();
        // WHEN
        buyer.searches_for_items_containing("wool");
        buyer.selects_item_number(5);
        // THEN
        buyer.should_see_matching_details();
    }

Notice how most of the methods here are reused from the previous steps: in fact, only two new methods are required.

3. Reporting and Living Documentation

Reporting is one of Serenity’s fortes. Serenity not only reports on whether a test passes or fails, but documents what it did, in a step-by-step narrative format that inculdes test data and screenshots for web tests. For example, the following page illustrates the test results for our first acceptance criteria:

Figure 1. Test results reported in Serenity

But test outcomes are only part of the picture. It is also important to know what work has been done, and what is work in progress. Serenity provides the @Pending annotation, that lets you indicate that a scenario is not yet completed, but has been scheduled for work, as illustrated here:

@RunWith(SerenityRunner.class)
public class WhenPuttingItemsInTheShoppingCart {

    @Pending @Test
    public void shouldUpdateShippingPriceForDifferentDestinationCountries() {
    }
}

This test will appear in the reports as Pending (blue in the graphs):

Figure 2. Test result overview

We can also organize our acceptance tests in terms of the features or requirements they are testing. One simple approach is to organize your requirements in suitably-named packages:

|----net
| |----serenity_bdd
| | |----samples
| | | |----etsy
| | | | |----features                                       (1)
| | | | | |----search                                       (2)
| | | | | | |----WhenSearchingByKeyword.java
| | | | | | |----WhenViewingItemDetails.java
| | | | | |----shopping_cart                                (3)
| | | | | | |----WhenPuttingItemsInTheShoppingCart.java
| | | | |----pages
| | | | | |----HomePage.java
| | | | | |----ItemDetailsPage.java
| | | | | |----RegisterPage.java
| | | | | |----SearchResultsPage.java
| | | | | |----ShoppingCartPage.java
| | | | |----steps
| | | | | |----BuyerSteps.java

1	All the test cases are organized under the `features` directory.
2	Test cass related to the search feature
3	Test cases related to the ‘shopping cart’ feature

Serenity can use this package structure to group and aggregate the test results for each feature. You need to tell Serenity the root package that you are using, and what terms you use for your requirements. You do this in a special file called (for historical reasons) thucydides.properties, which lives in the root directory of your project:

thucydides.test.root=net.serenity_bdd.samples.etsy.features
thucydides.requirement.types=feature,story

With this configured, Serenity will report about how well each requirement has been tested, and will also tell you about the requirements that have not been tested:

Figure 3. Serenity reports on requirements as well as tests

4. Conclusion

Hopefully this will be enough to get you started with Serenity.

That said, we have barely scratched the surface of what Serenity can do for your automated acceptance tests. You can read more about Serenity, and the principles behind it, by reading the Users Manual, or by reading BDD in Action, which devotes several chapters to these practices. And be sure to check out the online courses at Parleys.

You can get the source code for the project discussed in this article on GitHub.