Test automation for custom components using the Vaadin Testbench

Hello,

so far so good I managed to get some tests running with the vaadin test bench like it was described in tutorials provided by vaadin.
I wonder however if there is a concept on how to test the generated java part and also the facade java part of custom vaadin components
(build with polymer) completely automated. So I’m looking for a way to just press the “Run Tests” button of a component and want the
component then automatically be started locally on some predefined port followed by performing all the implemented unit tests.

Is a process like that defined somewhere to look it up?

Hi,

Did you have a look at https://vaadin.com/docs/v10/testbench/testbench-running-with-maven.html which describes how to start/stop a server and run tests automatically using Maven?

No, did not find that. It sounds great. Ill check it out. Thank you.

Do I understand it correctly that the whole testbench stuff is made for testing applications and not single components?
We are building flow components using polymer so we basically have three different code parts:

  1. The frontend code (html / css / javascript)
  2. The automatically generated java code
  3. The facade java code extending the generated java code (the code which we actually want to use in vaadin-applications)

The third part is the one for which we want to write unit tests and integration tests using the testbench. If I run the tests I need to be able to “start” the component somehow so that it is accessible for the unit tests on some address and port. And this fully automatically ofcourse. Did I just miss any part in the documentation describing how to start and show components in the browser out of the java side project or do I understand something very wrong?

Hi,

the main target is to test applications but it’s used to test all our components also.

When you have multiple layers, there are basically two strategies you can adopt:

  1. Test as much as possible on the lowest layer, then only test on the next layer what is added and assume the lower layers work
  2. Only test on the highest layer and test everything at once

The benefit of doing 1. is that you will immediately find problems if you make changes to your frontend code while the benefit of 2. is that you only have to write tests only in one place.

If you test web components on the Polymer level, you use polyserve and some test templates to “deploy” and configure the component you want to test.

If you test on Flow level, you use some embedded server like Jetty as described in the docs linked above and “deploy” and configure the component using a test view, like https://github.com/vaadin/vaadin-button-flow/blob/master/src/test/java/com/vaadin/flow/component/button/demo/ButtonView.java (ignore the javadocs)

Nice, thank you for your reply. I took the example you posted (vaadin-button-flow) and tried to reproduce it for one of my components. So I added a pom.xml to my project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.my.group</groupId>
    <artifactId>ranking-bar</artifactId>
    <version>0.2.0</version>

    <properties>
        <flow.version>1.0-SNAPSHOT</flow.version>
        <jetty.version>9.3.7.v20160115</jetty.version>
        <maven.failsafe.plugin.version>2.20</maven.failsafe.plugin.version>
        <maven.resources.plugin.version>3.0.2</maven.resources.plugin.version>
    </properties>

    <repositories>
        <repository>
            <id>vaadin-prereleases</id>
            <url>https://maven.vaadin.com/vaadin-prereleases/</url>
        </repository>
        <repository>
            <id>webjars-bintray</id>
            <url>https://dl.bintray.com/webjars/maven/</url>
        </repository>
        <repository>
            <id>vaadin-addons</id>
            <url>http://maven.vaadin.com/vaadin-addons</url>
        </repository>
        <repository>
            <id>nexus-snapshots</id>
            <url>https://my.nexus.com/repository/maven-snapshots/</url>
        </repository>
        <repository>
            <id>nexus-releases</id>
            <url>https://my.nexus.com/repository/maven-releases/</url>
        </repository>
    </repositories>

    <build>
        <plugins>
            <plugin>
                <groupId>org.eclipse.jetty</groupId>
                <artifactId>jetty-maven-plugin</artifactId>
                <version>${jetty.version}</version>
                <configuration>
                    <httpConnector>
                        <port>9998</port>
                    </httpConnector>
                    <scanIntervalSeconds>-1</scanIntervalSeconds>
                    <stopPort>9999</stopPort>
                    <stopWait>5</stopWait>
                    <stopKey>foo</stopKey>
                    <!-- Use TestScope because the test classes are in the
                        test package. -->
                    <useTestScope>true</useTestScope>
                    <webAppConfig>
                        <containerIncludeJarPattern>^$</containerIncludeJarPattern>
                    </webAppConfig>
                </configuration>
                <executions>
                    <!-- start and stop jetty (running our app) when running
                        integration tests -->
                    <execution>
                        <id>start-jetty</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-jetty</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${maven.failsafe.plugin.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <systemPropertyVariables>
                        <webdriver.chrome.driver>${webdriver.chrome.driver}</webdriver.chrome.driver>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <version>${maven.resources.plugin.version}</version>
                <executions>
                    <execution>
                        <id>copy-resources-for-demo-website</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/test-classes</outputDirectory>
                            <overwrite>true</overwrite>
                            <resources>
                                <resource>
                                    <directory>src/test/java</directory>
                                </resource>
                                <resource>
                                    <directory>src/main/webapp</directory>
                                    <targetPath>META-INF/resources</targetPath>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>com.googlecode.maven-download-plugin</groupId>
                <artifactId>download-maven-plugin</artifactId>
                <version>1.3.0</version>
                <executions>
                    <execution>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>wget</goal>
                        </goals>
                        <configuration>
                            <url>https://raw.githubusercontent.com/vaadin/flow/master/drivers.xml</url>
                            <outputFileName>drivers.xml</outputFileName>
                            <outputDirectory>${project.build.directory}</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.sonatype.plugins</groupId>
                <artifactId>nexus-staging-maven-plugin</artifactId>
                <version>1.6.8</version>
                <extensions>true</extensions>
                <configuration>
                    <nexusUrl>${staging.server.url}</nexusUrl>
                    <serverId>${staging.server.id}</serverId>
                    <stagingProfileId>${staging.profile.id}</stagingProfileId>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>3.0.2</version>
                <configuration>
                    <finalName>${project.artifactId}</finalName>
                </configuration>
                <executions>
                    <execution>
                        <configuration>
                            <includes>
                                <include>**/demo/**</include>
                                <include>**/META-INF/resources/**</include>
                            </includes>
                            <classifier>demo</classifier>
                        </configuration>
                        <goals>
                            <goal>test-jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>3.0.1</version>
                <executions>
                    <execution>
                        <id>attach-sources</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-javadoc-plugin</artifactId>
                <version>2.10.4</version>
                <executions>
                    <execution>
                        <id>attach-javadocs</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <quiet>true</quiet>
                    <additionalparam>-Xdoclint:none</additionalparam>
                </configuration>
            </plugin>
        </plugins>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.eclipse.m2e</groupId>
                    <artifactId>lifecycle-mapping</artifactId>
                    <version>1.0.0</version>
                    <configuration>
                        <lifecycleMappingMetadata>
                            <pluginExecutions>
                                <pluginExecution>
                                    <pluginExecutionFilter>
                                        <groupId>com.googlecode.maven-download-plugin</groupId>
                                        <artifactId>download-maven-plugin</artifactId>
                                        <versionRange>[1.3.0,)</versionRange>
                                        <goals>
                                            <goal>wget</goal>
                                        </goals>
                                    </pluginExecutionFilter>
                                    <action>
                                        <execute>
                                            <runOnIncremental>false</runOnIncremental>
                                        </execute>
                                    </action>
                                </pluginExecution>
                            </pluginExecutions>
                        </lifecycleMappingMetadata>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.my.group</groupId>
            <artifactId>ranking-bar-generated</artifactId>
            <version>0.2.0</version>
        </dependency>

        <!-- Dependencies for the demo -->
        <dependency>
            <groupId>com.vaadin</groupId>
            <artifactId>flow-component-demo-helpers</artifactId>
            <version>${flow.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.vaadin</groupId>
            <artifactId>flow-test-generic</artifactId>
            <version>${flow.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <profiles>
        <profile>
            <id>runTests</id>
            <dependencies>
                <dependency>
                    <groupId>com.vaadin</groupId>
                    <artifactId>flow-server</artifactId>
                    <version>${flow.version}</version>
                    <scope>compile</scope>
                </dependency>
                <dependency>
                    <groupId>com.vaadin</groupId>
                    <artifactId>flow-client</artifactId>
                    <version>${flow.version}</version>
                    <scope>compile</scope>
                </dependency>
                <dependency>
                    <groupId>com.vaadin</groupId>
                    <artifactId>flow-data</artifactId>
                    <version>${flow.version}</version>
                    <scope>compile</scope>
                </dependency>
                <dependency>
                    <groupId>com.vaadin</groupId>
                    <artifactId>flow-html-components</artifactId>
                    <version>${flow.version}</version>
                    <scope>compile</scope>
                </dependency>
            </dependencies>
        </profile>
    </profiles>

</project>

And also created the demo class:

import com.vaadin.flow.demo.DemoView;
import com.vaadin.flow.router.Route;
import com.my.group.rankingbar.RankingBar;

@Route("ranking-bar")
public class RankingBarView extends DemoView {

    @Override
    protected void initView() {
        this.addDefaultRankingBar();
    }

    private void addDefaultRankingBar() {
        RankingBar rankingBar = new RankingBar();
        rankingBar.setId("default-ranking-bar");

        this.addCard("Default Ranking Bar", rankingBar);
    }
}

Now Im running mvn jetty:run -PrunTests and it is starting correctly and I can call it using http://localhost:9998/ranking-bar. But I got an issue here. The ranking bar is not showing up
due to:

GET http://localhost:9998/frontend/bower_components/ranking-bar/src/ranking-bar.html 404 (Not Found)

Do you maybe have an idea what I was missing?

Edit 1:

I was not able to start the vaadin-button-flow to compare the results due to:

[ERROR]
 Failed to execute goal on project vaadin-button-flow: Could not resolve dependencies for project com.vaadin:vaadin-button-flow:jar:1.0-SNAPSHOT: Failed to collect dependencies at org.webjars.bowergithub.vaadin:vaadin-button:jar:2.0.1 -> org.webjars.bowergithub.vaadin:vaadin-lumo-styles:jar:1.0.0 -> org.webjars.bowergithub.polymerelements:iron-icon:jar:2.0.1 -> org.webjars.bowergithub.polymerelements:iron-flex-layout:jar:1.3.7: Failed to read artifact descriptor for org.webjars.bowergithub.polymerelements:iron-flex-layout:jar:1.3.7: Could not transfer artifact org.webjars.bowergithub.polymerelements:iron-flex-layout:pom:1.3.7 from/to nexus-snapshots (https://nexus.gfk.com/repository/maven-snapshots/): Failed to transfer file: https://nexus.gfk.com/repository/maven-snapshots/org/webjars/bowergithub/polymerelements/iron-flex-layout/1.3.7/iron-flex-layout-1.3.7.pom. Return code is: 400, ReasonPhrase: Repository version policy: SNAPSHOT does not allow version: 1.3.7. -> [Help 1]

https://nexus.gfk.com/repository/maven-snapshots sound like some repository you have configured or your company is using.

GET http://localhost:9998/frontend/bower_components/ranking-bar/src/ranking-bar.html 404 (Not Found) sounds like you are missing the webjar for ranking-bar or the files are not in the correct folder

Thank you.

  1. You are totally right. It was the settings.xml for maven.
  2. The webjar is there but yes, maybe the wrong place, ill check it.

For the vaadin button the generated class is also doing this:

@HtmlImport("frontend://bower_components/vaadin-button/src/vaadin-button.html")

But this location is also not existing and still the application can find the vaadin-button.html.
Is there a configuration / annotation / whatever which is telling the application where to find it?

The webjar support maps frontend://bower_components to META-INF/resources/webjars to be able to support both Bower and webjars using the same import path. So frontend://bower_components/vaadin-button/src/vaadin-button.html should be META-INF/resources/webjars/vaadin-button/src/vaadin-button.html in a webjar, and that is found in http://central.maven.org/maven2/org/webjars/bowergithub/vaadin/vaadin-button/2.0.1/vaadin-button-2.0.1.jar

In my case the path is META-INF/resources/webjars/ranking-bar/0.2.0/src/ranking-bar.html so the version is the problem. Need to get rid of that. Thanks.

EDIT

Yes, all this is working very great when removing the version. Thank you very much!