BDD oriented testing in a Vaadin project

Marcio Dantas
On Sep 13, 2018 6:00:00 AM

In this blog post, I will showcase how Behaviour-Driven Development (BDD) can make your functional tests  (commonly called UI tests, too) more readable and as a consequence easier to maintain. The comparison “with vs. without BDD” is done by rewriting some UI tests from Vaadin Bakery App Starter (Vaadin 8 and Spring) with JBehave on top of Vaadin Test Bench 5.

Sample application links:

What is BDD in the first place?

If you have never heard of BDD, it originates from and builds on the Test-Driven Development  (TDD) mindset. While on TDD you write unit tests ahead of the actual implementation code, when using BDD you’ll be writing plain text files in natural language (English, for instance) describing how the application should behave in specific story scenarios. Through frameworks like JBehave or Cucumber, these regular text files become computer-interpretable and hence can be used on testing automation.

BDD and the Page Object pattern are recommended on the Vaadin docs  as best practices for UI testing. As a BDD story resembles a specification, people occasionally take a further step and use it to specify all the system’s requirements , an approach known as Specification by Example.

How to run BDD stories using JBehave?

After initially setting up JBehave on your project, creating a new story involves:

  1. Writing a story file using “Given, When and Then” syntax.
  2. Mapping story steps to Java
  3. Running the story
  4. Checking the results

Let’s use a practical example to see how this looks. We will rewrite the (Vaadin Bakery App Starter without BDD) UI test with JBehave. Take some time to understand it before we start.

    public class MenuIT extends AbstractIT {

        public void adminSeesAdminMenus() {
            MenuElement menu = $(MenuElement.class).first();
            Assert.assertNotNull(menu.getMenuLink("Users "));
            Assert.assertNotNull(menu.getMenuLink("Products "));
        public void baristaDoesNotSeeAdminMenus() {
            MenuElement menu = $(MenuElement.class).first();
            Assert.assertNull(menu.getMenuLink("Users "));
            Assert.assertNull(menu.getMenuLink("Products "));
    } UI test from Vaadin 8 with Spring Bakery App Starter

Writing a story file using the “Given, When and Then” syntax

The following story was written based on the original test code:

    As an admin
    I want to see menu items restricted to admins
    and other users shouldn't be able to do it

    Scenario: admin sees menu items restricted to admins
    Given I log in as admin
    Then I see menu link Users
    And I see menu link Products
    Scenario: barista doesn't see menu items restricted to admins
    Given I log in as barista
    Then I don't see menu link Users
    And I don't see menu link Products

MenuIT.story: MenuIT tests as a JBehave BDD story

The narrative is an optional metadata which summarizes what the story is about. Scenarios are executable specifications on how the system should behave in a given situation.

The .story file contains plain English sentences. All stakeholders involved in a project (even a customer with no technical expertise) should be able to understand it.

Mapping story steps to Java

You need to map each step (Given, When or Then) of the story to a Java method so JBehave knows how the steps are performed.

Time to change to reflect this:

    public class MenuIT extends AbstractStory {
        public List<Class<? extends AbstractStep>> getStepClasses() {
                return Collections.singletonList(StorySteps.class);

        public static class StorySteps extends AbstractStep {
                @Then("I see menu link $linkCaption ")
                public void assertMenuLinkExists(String linkCaption) {

                @Then("I don 't see menu link $linkCaption")
                public void assertMenuLinkDoesntExist(String linkCaption) {

                private WebElement getMenuLink(String linkCaption) {
                        MenuElement menu = $(MenuElement.class).first();
                        return menu.getMenuLink(linkCaption);
    } mapping MenuIT.story into Java

  • MenuIT now is an AbstractStory which is a JUnit runnable entry point and bootstraps JBehave and Vaadin Test Bench.
  • As an AbstractStory, MenuIT needs to return all classes containing step mappings used by the story.
  • StorySteps contains two parameterized steps: @Then("I see menu link $linkCaption") and @Then("I don't see menu link $linkCaption")
  • AbstractStep has some generic steps like: @Given("I log in as admin") and @Given("I log in as barista").

The steps can be reused. I also strongly recommend you to adopt the Page Object pattern to encapsulate and reuse code interacting with Vaadin Test Bench and Selenium. This will make your test codebase much more modular. But let’s do it in baby steps. This could be a topic for a future blog post.

Running the story is a JUnit test, just as before. So you can still run it directly from your favorite IDE or using the Maven command:

 mvn clean verify -Pit

Checking the results

If everything goes fine, your logs should look similar to this:

Running story org/vaadin/bdd/ui/views/MenuIT.story

Scenario: admin sees menu items restricted to admins
Using timeout for story MenuIT.story of 30000 secs.
Vaadin TestBench 5 registered to YOUR_USERNAME (Pro Tools subscription)
Given I log in as admin
Then I see menu link Users
And I see menu link Products

Scenario: barista doesn't see menu items restricted to admins 
Given I log in as barista 
Then I don 't see menu link Users
And I don't see menu link Products 

Process finished with exit code 0

JBehave report: successful run

If a step fails then JBehave will report that specific one and thus, all subsequent ones are skipped. This is how it looks like:

    Running story org/vaadin/bdd/ui/views/MenuIT.story

    Scenario: admin sees menu items restricted to admins
    Using timeout for story MenuIT.story of 30000 secs.
    Vaadin TestBench 5 registered to YOUR_USERNAME (Pro Tools subscription)
    Given I log in as admin
    Then I see menu link Super uper admin (FAILED)
    And I see menu link Products (NOT PERFORMED)
        at org.vaadin.bdd.ui.views.MenuIT$StorySteps.assertMenuLinkExists(

JBehave report: step failure with stacktrace

It makes it really obvious to understand what the code that failed was trying to achieve, and hence start investigating what went wrong. In the sample application, the Vaadin Test Bench screenshot on failure feature is also used, which takes a screenshot at the moment the test fails, that helps a lot as well.

“With vs. without BDD” comparison

To have a glimpse at how BDD could improve your UI test maintainability, let’s tackle a bigger test this time:

        public void updateOrderInfo() {
            StorefrontViewElement storeFront = loginAsBarista();
            OrderEditViewElement orderEdit = storeFront.selectOrder(1);
            OrderState oldState = OrderState.forDisplayName(orderEdit.getStateLabel().getText());
            Assert.assertEquals("Cancel button has wrong caption", "Cancel", orderEdit.getEditOrCancel().getCaption());
            Assert.assertEquals("Save button has wrong caption", "Save", orderEdit.getOk().getCaption());
            OrderInfo currentOrder = orderEdit.getOrderInfo();
            OrderInfo updatedOrder = new OrderInfo();
            LocalDate newDate = currentOrder.dueDate.plusDays(1);
            updatedOrder.dueDate = newDate;
            int nextStateIndex = (oldState.ordinal() + 1) % OrderState.values().length;
            OrderState newState = OrderState.values()[nextStateIndex];
            updatedOrder.state = newState;
            Customer currentCustomer = currentOrder.customer;
            Customer updatedCustomer = new Customer();
            updatedCustomer.setFullName(currentCustomer.getFullName() + "-updated");
            updatedCustomer.setPhoneNumber(currentCustomer.getPhoneNumber() + "-updated");
            updatedCustomer.setDetails(currentCustomer.getDetails() + "-updated");
            updatedOrder.customer = updatedCustomer;
            updatedOrder.pickupLocation = "Store".equals(currentOrder.pickupLocation) ? "Bakery" : "Store";
            updatedOrder.products = new ArrayList<>();
            for (int i = 0; i < currentOrder.products.size(); i++) {
                ProductOrderData updatedProduct = new ProductOrderData();
                ProductOrderData currentProduct = currentOrder.products.get(i);
                updatedProduct.setComment(currentProduct.getComment() + "-updated");
                updatedProduct.setQuantity(currentProduct.getQuantity() + 1);
                // Product is intentionally kept the same as we do not know what
                // products there are in the DB
            int updatedTotal = 0;
            for (ProductOrderData data : updatedOrder.products) {
                updatedTotal += data.getQuantity() * data.getPrice();
            NumberFormat format = NumberFormat.getNumberInstance(Locale.US);
            DollarPriceConverter convert = new DollarPriceConverter();
   = convert.convertToPresentation(updatedTotal, new ValueContext(Locale.US));
            Assert.assertEquals("Save failed", "Edit", orderEdit.getEditOrCancel().getCaption());
 updateOrderInfo original test

It might take some time to understand this one. The intention wasn’t to frighten you, this was taken from a sample application... In real life, we know things can get much harder.

After spending some time understanding the test and rewriting it with BDD, this is the resulting .story file:

    Scenario: barista is able to update order details
    Given I log in as barista
    When I select the first order
    Then Cancel button should be displayed
    And Save button should be displayed
    Given I change the order date
    And I change the order state
    And I change the order customer
    And I change the pickup location
    And I change the order products
    And I change the order total
    When I click Save
    Then Edit button should be displayed
    And the order data should be updated    

UpdateOrderInfoIT.story: updateOrderInfo test as BDD story

Now it’s possible to follow up on what the test does in plain English and 14 sentences. You can check the full implementation on the sample application  and it’s possible to easily see all the changes made on the original application on GitHub , as well.

Final remarks

Every team has a different culture, experience and preferences, so there is no right or wrong here. But if you do think this approach makes your UI tests easier to maintain, then the code samples can help you get started quickly with JBehave on your Vaadin applications.

Nice testing! Cheers!


Marcio Dantas
Software engineer, beloved dad and clean code enthusiast.
Other posts by Marcio Dantas