Testing the UI without a browser

Matti Tahvonen
Matti Tahvonen
On Dec 19, 2018 7:00:00 AM

Martin Fowler has introduced a metaphor Test Pyramid, to suggest that you should only write a fraction of end-to-end tests compared to small unit tests. It is not because end-to-end tests, like the ones using Vaadin TestBench or Selenium, are worse. They are actually better as they test all parts of the application like the end-user sees it. But those are also far more expensive to write, and especially to maintain, when developing your application.

testing-ui-without-browserPart of the cost of automated end-to-end tests comes from the fact that even a small change in your app possibly affects multiple tests, which have a very weak link to the actual change. It is time-consuming to figure out which tests need to be updated and why. Part of this inconvenience comes from the fact that they are so slow to execute. Launching the full app and commanding browsers to interact with the server takes time and thus these tests are often performed only on CI servers.

This is why smaller unit or integration tests are often preferred. This is also why the MVP pattern is often applied. We can then test the UI logic, but separately without the view. These are the ways to make tests easier to write and execute. But what if your full UI test would only take 50 ms instead of 50 seconds to execute? What if you could write those in Java against the API that your UI library and your code provide?

Testing the UI, not the framework or browser

A different, but not new, approach to test your UI is to skip the rendering and browser part altogether. This approach is applied for example in the Vaadin guide in With a high-level component-based approach, like Vaadin's Java APIs, you can speed up your UI test execution tremendously, by skipping the rendering part and browser, and writing the test as a basic JUnit test. We have thousands of automated UI tests for the framework itself; maybe you don't have to test that part again?

Here is a trivial example of the approach:

public static class HelloWorld extends VerticalLayout {
    Label msg = new Label();
    Button button = new Button("Click me");

    public HelloWorld() {
        addComponents(button, msg);
        button.addClickListener(e -> msg.setValue("Hello"));


public void messageUpdates() {
    HelloWorld helloWorld = new HelloWorld();
    Assert.assertEquals("", helloWorldTest.msg.getValue());;
    Assert.assertEquals("Hello", helloWorld.msg.getValue());
    long end = System.currentTimeMillis();

So instead of deploying an application to a web server and commanding the browser to type in text and clicking the button, we do the same thing programmatically with the server side API. We are testing the same thing, as long as you can expect that your UI framework, server, and browser work as they should.

For JUnit runner, executing the above takes about 50ms and a fraction of that when the JRE/JIT warms up. Doing a similar test with Selenium took 3216ms with my MacBook Pro, and that doesn't get too much faster when the JRE warms up. The first is 64 times faster. If you have a larger set of tests, you can expect this style of testing to be around 100 times faster than testing real end-to-end results with a Selenium based solution.

Problems and solutions

The problem is that in a non-trivial case it doesn't work that easily. Many Vaadin component APIs expect to be connected to the UI, Page, and Session, and will throw exceptions when you try to write a test for your real UI classes. Same with certain thread-local variables, which are often used by Vaadin developers. We can mock the context classes, but that's a lot of work.

Instead of mocking the classes yourself, there is a convenient community project called Karibu-Testing, that does exactly this for you. When using Karibu-Testing, handy helpers like UI.getCurrent() work just as with real deployment. The project is built in Kotlin, but can just as well be used with Java, too. The library supports Spring Boot projects and both Vaadin 8 and 10+ and there are examples for many different project setups.

This approach to UI testing sure can't replace your Vaadin TestBench  tests, which verify your app works using real browsers. But if you wish to test more of your UI layer, without increasing your budget too much, this method may be a handy tool in your testing setup. You can test a bit more than just the UI logic when using MVP pattern and you are not tied to certain patterns in your UI code.

Matti Tahvonen
Matti Tahvonen
Matti Tahvonen has a long history in Vaadin R&D: developing the core framework from the dark ages of pure JS client side to the GWT era and creating number of official and unofficial Vaadin add-ons. His current responsibility is to keep you up to date with latest and greatest Vaadin related technologies. You can follow him on Twitter – @MattiTahvonen
Other posts by Matti Tahvonen