Table of Contents

20.1. Overview
20.2. Installing Vaadin TestBench
20.2.1. Test Development Installation
20.2.2. A Distributed Testing Environment
20.2.3. Downloading and Unpacking the Installation Package
20.2.4. Installation Package Contents
20.2.5. Example Contents
20.2.6. Installing the Recorder
20.2.7. Installing Browser Drivers
20.2.8. Test Node Configuration
20.3. Preparing an Application for Testing
20.4. Using Vaadin TestBench Recorder
20.4.1. Starting the Recorder
20.4.2. Recording
20.4.3. Selectors
20.4.4. Playing Back Tests
20.4.5. Editing Tests
20.4.6. Exporting Tests
20.4.7. Saving Tests
20.5. Developing JUnit Tests
20.5.1. Starting From a Stub
20.5.2. Finding Elements by Selectors
20.5.3. Running JUnit Tests in Eclipse
20.5.4. Executing Tests with Ant
20.5.5. Executing Tests with Maven
20.5.6. Test Setup
20.5.7. Creating and Closing a Web Driver
20.5.8. Basic Test Case Structure
20.5.9. Waiting for Vaadin
20.5.10. Testing Tooltips
20.5.11. Scrolling
20.5.12. Testing Notifications
20.5.13. Testing Context Menus
20.5.14. Profiling Test Execution Time
20.6. Taking and Comparing Screenshots
20.6.1. Screenshot Parameters
20.6.2. Taking Screenshots on Failure
20.6.3. Taking Screenshots for Comparison
20.6.4. Practices for Handling Screenshots
20.6.5. Known Compatibility Problems
20.7. Running Tests in an Distributed Environment
20.7.1. Running Tests Remotely
20.7.2. Starting the Hub
20.7.3. Node Service Configuration
20.7.4. Starting a Grid Node
20.7.5. Mobile Testing
20.8. Known Issues
20.8.1. Testing the LoginForm
20.8.2. Using assertTextPresent and assertTextNotPresent
20.8.3. Exporting Recordings of the Upload Component
20.8.4. Running Firefox Tests on Mac OS X

This chapter describes the installation and use of the Vaadin TestBench.

Quality assurance is one of the cornerstones of modern software development. Extending throughout the entire development process, quality assurance is the thread that binds the end product to the requirements. In iterative development processes, with ever shorter release cycles and continuous integration, the role of regression testing is central. The special nature of web applications creates many unique requirements for regression testing.

In a typical situation, you are developing a web application with Vaadin and want to ensure that only intended changes occur in its behaviour after modifying the code, without testing the application manually every time. There are two basic ways of detecting such regressions. Screenshots are the strictest way, but often just checking the displayed values in the HTML is better if you want to allow some flexibility for themeing, for example. You may also want to generate many different kinds of inputs to the application and check that they produce the desired outputs.


Vaadin TestBench utilizes the Selenium WebDriver to control the browser from Java code, as illustrated in Figure 20.1, “Controlling the Browser with WebDriver”. It can open a new browser window to start the application, interact with the components for example by clicking them, and then get the HTML element values.

You can develop such WebDriver unit tests along your application code, for example with JUnit, which is a widely used Java unit testing framework. You can also use a recorder that runs in the browser to create JUnit test case stubs, which you can then refine further with Java. You can run the tests as many times as you want in your workstation or in a distributed grid setup.


The main features of Vaadin TestBench are:

  • Record JUnit test case stubs in browser

  • Develop tests in Java with the WebDriver

  • Validate UI state by assertions and screen capture comparison

  • Screen capture comparison with difference highlighting

  • Distributed test grid for running tests

  • Integration with unit testing

  • Test with browsers on mobile devices

Execution of tests can be distributed over a grid of test nodes, which speeds up testing. The grid nodes can run different operating systems and have different browsers installed. In a minimal setup, such as for developing the tests, you can use Vaadin TestBench on just a single computer.