Documentation

Documentation versions (currently viewingVaadin 23)
Check out the new styling guides

Running Tests with Maven

A Maven build is divided into different lifecycle phases, of which the relevant ones are:

  • compile Compiles the code

  • test Runs unit tests which do not require a packaged project

  • pre-integration-test Makes preparations for integration tests

  • integration-test Executes integration tests

  • post-integration-test Cleans up after integration tests

TestBench tests naturally fit into the integration-test phase. The pre-integration-test phase is the place to start a server and deploy the package. The post-integration-test phase is where you would stop the server.

Tip
If you name your tests *Test, they will be run automatically in the test phase. If, instead, you name your TestBench tests *IT, they will be run automatically in the integration-test phase.
Note
Never execute TestBench tests in the test phase, because they cannot be run without a packaged or deployed project.

Starting the Server Automatically

For applications without external dependencies, it is often handy to start a test as part of the build. As an example, if you are using Jetty to run the project, you can use the jetty-maven-plugin to start the server in the pre-integration-test and stop it in the post-integration-test phase, as follows:

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.2.3.v20140905</version>
    <configuration>
        <stopPort>9966</stopPort>
        <stopKey>something-goes-here</stopKey>
    </configuration>
    <executions>
        <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The stopPort and stopKey are Jetty specific parameters which must be given so that Jetty is able to stop the correct server instance. A fully working example of running Jetty as part of the build can be found in https://github.com/vaadin/testbench-demo/blob/master/pom.xml.

If you are using Spring Boot, you can use the spring-boot-maven-plugin to achieve the same thing. See the Bakery starter for Spring for an example.

If you are using JavaEE, you can start TomEE, WildFly or a Liberty server in a similar way. See the Bakery starter for JavaEE (at the time of writing only available for Vaadin 8) for an example.

Executing Tests in the Integration Test Phase

In Maven, unit tests are executed by the maven-surefire-plugin, automatically included in all projects. On the other hand, integration tests are executed by the maven-failsafe-plugin, which needs to be included manually in the project as

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <trimStackTrace>false</trimStackTrace>
    </configuration>
</plugin>

The <executions> part is needed to actually execute the plugin during the integration-test phase. The <configuration> part is optional but, by including it, you get the full stack trace when an error occurs, which typically makes it easier to figure out what went wrong in a test.

Downloading Web Drivers Automatically

There is a plugin called driver-binary-downloader-maven-plugin which downloads web drivers for a given browser and platform and makes them available for the TestBench tests. By downloading these as part of the build, you do not need to do any setup on the machine where you are running the tests. The plugin can be enabled as follows:

<plugin>
    <groupId>com.lazerycode.selenium</groupId>
    <artifactId>driver-binary-downloader-maven-plugin</artifactId>
    <version>1.0.17</version>
    <configuration>
        <downloadedZipFileDirectory>${project.basedir}/webdriver/zips</downloadedZipFileDirectory>
        <rootStandaloneServerDirectory>${project.basedir}/webdriver</rootStandaloneServerDirectory>
        <customRepositoryMap>${project.basedir}/webdrivers.xml</customRepositoryMap>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>selenium</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This will download the web drivers defined in webdrivers.xml (called a repository map) in the project root during the test-compile phase, that is, before the integration tests start. The downloaded web drivers will be placed in the webdriver/zips folder in the project and unpacked to the webdriver folder. The file webdrivers.xml defines which version of the various webdrivers to download. An example can be found at https://github.com/vaadin/testbench-demo/blob/master/webdrivers.xml.

Tip
There is a repository map which is kept quite up to date available at https://github.com/Ardesco/selenium-standalone-server-plugin

In addition to downloading the web drivers, the location of the unpacked drivers must be passed to the maven-failsafe-plugin, so that the TestBench tests can find them during execution. This can be done by defining system properties in the <configuration> section of the maven-failsafe-plugin:

<configuration>
    <trimStackTrace>false</trimStackTrace>
    <systemPropertyVariables>
        <webdriver.chrome.driver>${webdriver.chrome.driver}</webdriver.chrome.driver>
        <!-- Similarly for other browsers -->
    </systemPropertyVariables>
</configuration>

2516DA74-34F6-4247-AAD3-44584BF5DBF3