Docs

Docs

Running Tests with Maven

Using Maven to test projects.

A Maven build is divided into different lifecycle phases. Below are the relevant ones:

  • compile Compiles the code;

  • test Runs unit tests which don’t require a packaged project;

  • pre-integration-test Makes preparations for integration tests;

  • integration-test Executes integration tests; and

  • post-integration-test Cleans up after integration tests.

TestBench tests fit into the integration-test phase. The pre-integration-test phase is the place to start a server and deploy the package. The post-integration-test phase is where you would stop the server.

Note
 Never execute TestBench tests in the test phase. They can’t be run without a packaged or deployed project.

+ If you name your tests *Test, they are run automatically in the test phase. If you instead name your TestBench tests *IT, they’re run automatically in the integration-test phase.

Starting the Server Automatically

For applications without external dependencies, it’s often handy to start a test as part of the build. As an example, if you’re using Jetty to run the project, you can use the jetty-maven-plugin to start the server in the pre-integration-test and stop it in the post-integration-test phase, as follows:

Source code
XML
<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.2.3.v20140905</version>
    <configuration>
        <stopPort>9966</stopPort>
        <stopKey>something-goes-here</stopKey>
    </configuration>
    <executions>
        <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The stopPort and stopKey are Jetty specific parameters which must be given so that Jetty is able to stop the correct server instance. A fully working example of running Jetty as part of the build can be found in https://github.com/vaadin/testbench-demo/blob/master/pom.xml.

If you’re using Spring Boot, you can use the spring-boot-maven-plugin to achieve the same thing. See the Bakery starter for Spring for an example.

If you’re using JavaEE, you can start TomEE, WildFly or a Liberty server in a similar way. See the Bakery starter for JavaEE for an example — this is available at this time only for Vaadin 8.

Executing Tests in Integration Test Phase

In Maven, unit tests are executed by the maven-surefire-plugin, automatically included in all projects. Integration tests are executed by the maven-failsafe-plugin instead, which needs to be included manually in the project as the following:

Source code
XML
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <trimStackTrace>false</trimStackTrace>
    </configuration>
</plugin>
XML
XML
XML

The <executions> part is needed to execute the plugin during the integration-test phase. The <configuration> part is optional, but by including it you get the full stack trace when an error occurs. This typically makes it easier to determine what went wrong in a test. Running failed tests multiple times is also possible using the rerunFailingTestsCount property.

Downloading WebDrivers Automatically

The webdrivermanager plugin downloads WebDrivers for a given browser and platform and makes them available for the TestBench tests. By downloading these as part of the build, you don’t need to do any setup on the machine where you are running the tests.

The plugin can be enabled as follows:

Source code
XML
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>6.1.0</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>com.google.guava</groupId>
                    <artifactId>guava</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.github.docker-java</groupId>
                    <artifactId>docker-java-transport-httpclient5</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>net.java.dev.jna</groupId>
                    <artifactId>jna</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>net.java.dev.jna</groupId>
                    <artifactId>jna-platform</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

WebDriverManager is convenient as it will detect the version of the installed browser in the system and then installs the correct version of the driver. This works both when running tests locally and CI/CD pipeline.

In addition to the dependency, the base class of the tests needs to setup the WebDriverManager, for example with Chrome it is done like below:

Source code
Java
    @BeforeAll
    public static void setupClass() {
        WebDriverManager.chromedriver().setup();
    }

2516DA74-34F6-4247-AAD3-44584BF5DBF3