Testing the UI without a browser

Martin Fowler has introduced a metaphor Test Pyramid, to suggest that you should only write a fraction of end-to-end tests compared to small unit tests. It is not because end-to-end tests, like the ones using Vaadin TestBench or Selenium, are worse. They are actually better as they test all parts of the application like the end-user sees it. But those are also far more expensive to write, and especially to maintain, when developing your application.

Part of the cost of automated end-to-end tests comes from the fact that even a small change in your app possibly affects multiple tests, which have a very weak link to the actual change. It is time-consuming to figure out which tests need to be updated and why. Part of this inconvenience comes from the fact that they are so slow to execute. Launching the full app and commanding browsers to interact with the server takes time and thus these tests are often performed only on CI servers.

This is why smaller unit or integration tests are often preferred. This is also why the MVP pattern is often applied. We can then test the UI logic, but separately without the view. These are the ways to make tests easier to write and execute. But what if your full UI test would only take 50 ms instead of 50 seconds to execute? What if you could write those in Java against the API that your UI library and your code provide?

Testing the UI, not the framework or browser

A different, but not new, approach to test your UI is to skip the rendering and browser part altogether. This approach is applied for example in the Vaadin guide in spring.io. With a high-level component-based approach, like Vaadin's Java APIs, you can speed up your UI test execution tremendously, by skipping the rendering part and browser, and writing the test as a basic JUnit test. We have thousands of automated UI tests for the framework itself; maybe you don't have to test that part again?

Here is a trivial example of the approach:

public static class HelloWorld extends VerticalLayout {
    Label msg = new Label();
    Button button = new Button("Click me");

    public HelloWorld() {
        addComponents(button, msg);
        button.addClickListener(e -> msg.setValue("Hello"));
    }

}

@Test
public void messageUpdates() {
    HelloWorld helloWorld = new HelloWorld();
    Assert.assertEquals("", helloWorldTest.msg.getValue());
    helloWorld.button.click();
    Assert.assertEquals("Hello", helloWorld.msg.getValue());
    long end = System.currentTimeMillis();
}