Chapter 20. Vaadin TestBench
Table of Contents
- 20.1. Overview
- 20.2. Installing Vaadin TestBench
- 20.3. Preparing an Application for Testing
- 20.4. Using Vaadin TestBench Recorder
- 20.5. Developing JUnit Tests
- 20.5.1. Starting From a Stub
- 20.5.2. Finding Elements by Selectors
- 20.5.3. Running JUnit Tests in Eclipse
- 20.5.4. Executing Tests with Ant
- 20.5.5. Executing Tests with Maven
- 20.5.6. Test Setup
- 20.5.7. Creating and Closing a Web Driver
- 20.5.8. Basic Test Case Structure
- 20.5.9. Waiting for Vaadin
- 20.5.10. Testing Tooltips
- 20.5.11. Scrolling
- 20.5.12. Testing Notifications
- 20.5.13. Testing Context Menus
- 20.5.14. Profiling Test Execution Time
- 20.6. Taking and Comparing Screenshots
- 20.7. Running Tests in an Distributed Environment
- 20.8. Known Issues
This chapter describes the installation and use of the Vaadin TestBench.
Quality assurance is one of the cornerstones of modern software development. Extending throughout the entire development process, quality assurance is the thread that binds the end product to the requirements. In iterative development processes, with ever shorter release cycles and continuous integration, the role of regression testing is central. The special nature of web applications creates many unique requirements for regression testing.
In a typical situation, you are developing a web application with Vaadin and want to ensure that only intended changes occur in its behaviour after modifying the code, without testing the application manually every time. There are two basic ways of detecting such regressions. Screenshots are the strictest way, but often just checking the displayed values in the HTML is better if you want to allow some flexibility for themeing, for example. You may also want to generate many different kinds of inputs to the application and check that they produce the desired outputs.
Vaadin TestBench utilizes the Selenium WebDriver to control the browser from Java code, as illustrated in Figure 20.1, “Controlling the Browser with WebDriver”. It can open a new browser window to start the application, interact with the components for example by clicking them, and then get the HTML element values.
You can develop such WebDriver unit tests along your application code, for example with JUnit, which is a widely used Java unit testing framework. You can also use a recorder that runs in the browser to create JUnit test case stubs, which you can then refine further with Java. You can run the tests as many times as you want in your workstation or in a distributed grid setup.
The main features of Vaadin TestBench are:
-
Record JUnit test case stubs in browser
-
Develop tests in Java with the WebDriver
-
Validate UI state by assertions and screen capture comparison
-
Screen capture comparison with difference highlighting
-
Distributed test grid for running tests
-
Integration with unit testing
-
Test with browsers on mobile devices
Execution of tests can be distributed over a grid of test nodes, which speeds up testing. The grid nodes can run different operating systems and have different browsers installed. In a minimal setup, such as for developing the tests, you can use Vaadin TestBench on just a single computer.
Vaadin TestBench is based on the Selenium web browser automation library. With the Selenium WebDriver API, you can control browsers straight from Java code. The TestBench Recorder is based on the Selenium IDE.
Selenium is augmented with Vaadin-specific extensions, such as:
- Proper handling of Ajax-based communications of Vaadin
- Exporting test case stubs from the Recorder
- Performance testing of Vaadin applications
- Screen capture comparison
- Finding HTML elements using a Vaadin selector
The main components of Vaadin TestBench are:
-
Vaadin TestBench Java Library
-
Vaadin TestBench Recorder
The library includes WebDriver, which provides API to control a browser like a user would. This API can be used to build tests, for example, with JUnit. It also includes the grid hub and node servers, which you can use to run tests in a grid configuration.
The Vaadin TestBench Recorder is helpful for creating test case stubs. It is a Firefox extension that you install in your browser. It has a control panel to record test cases and play them back. You can play the test cases right in the recorder. You can then export the tests as JUnit tests, which you can edit further and then execute with the WebDriver.
Vaadin TestBench Library provides the central control logic for:
-
Executing tests with the WebDriver
-
Additional support for testing Vaadin-based applications
-
Comparing screen captures with reference images
-
Distributed testing with grid node and hub services
Requirements for recording test cases with Vaadin TestBench Recorder:
-
Mozilla Firefox
Requirements for running tests:
-
Java JDK 1.6 or newer
-
Browsers installed on test nodes as supported by Selenium WebDriver
- Google Chrome
- Internet Explorer
- Mozilla Firefox (10.x ESR recommended)
- Opera
- Mobile browsers: Android, iPhone
-
A build system, such as Ant or Maven, to automate execution of tests during build process (recommended)
Note that running tests on Firefox 10.x ESR is recommended because of the frequent release cycle of Firefox, which often cause tests to fail. Download the ESR release of Firefox from http://www.mozilla.org/en-US/firefox/organizations/all.html. Install it alongside your normal Firefox install (do not overwrite).
For Mac OS X, note the issue mentioned in Section 20.8.4, “Running Firefox Tests on Mac OS X”.
Continuous integration means automatic compilation and testing of applications frequently, typically at least daily, but ideally every time when code changes are committed to the source repository. This practice allows catching integration problems early and finding the changes that first caused them to occur.
You can make unit tests with Vaadin TestBench just like you would do any other Java unit tests, so they work seamlessly with continuous integration systems. Vaadin TestBench is tested to work with at least TeamCity and Hudson/Jenkins build management and continuous integration servers, which all have special support for the JUnit unit testing framework.
You can download Vaadin TestBench from Vaadin Directory and try it out for a free 30-day trial period, after which you are required to acquire the needed licenses. You can purchase licenses from the Directory. A license for Vaadin TestBench is also included in the Vaadin Pro Account subscription.