Testing Applications with Playwright

How to set up end-to-end testing of a Vaadin application with Playwright.

Set Up Playwright
First Playwright Test
Running the Test
More about Playwright

Playwright is an open source end-to-end testing library developed by Microsoft. Similarly to Selenium, it provides Java API to simulate user interactions via browsers. Compared to TestBench it lacks high level API for Vaadin components and it doesn’t automatically wait for the client-server communication to stabilize after user interaction.

Still, this page describes how to set up Playwright for a Spring Boot based Vaadin application. It also shows how to create a simple Playwright test, and the basics of finding and interacting with components in a Playwright test.

Although these instructions are Spring Boot specific, the same principles can be adapted to other architectures. The primary difference is that you need to control the starting and stopping of the server in a different manner. For example, you can use the pre-integration-test and post-integration-test phases — as shown in Getting Started with End-to-End Testing. You can also adopt the @SpringBootTest based approach described here for Testing with Selenium.

Set Up Playwright

To use Playwright in a Vaadin application, you need to perform a few steps. First, add the Playwright dependency to your project’s pom.xml file like so:

<dependency>
    <groupId>com.microsoft.playwright</groupId>
    <artifactId>playwright</artifactId>
    <version>1.39.0</version> <!-- Check the latest version via https://playwright.dev -->
    <scope>test</scope>
</dependency>

Next, make sure you have the spring-boot-starter-test module in your project. Spring Boot’s test starter module brings in, for example, JUnit 5 and Spring’s own tools that you may use to control starting the server for tests. Below is an example of how you would add this dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

You might already be enabling this dependency, though. Don’t duplicate it.

First Playwright Test

To create your first Playwright test, you’ll first need a view to test. Add or modify your main view to be like this:

@Route
public class MainView extends VerticalLayout {

    public MainView() {
        var msg = new Paragraph("");
        // Add id to make testing easier
        msg.setId("msg");

        Button button = new Button("Click me", e -> {
            msg.setText("Clicked!");
        });

        add(button, msg);
    }
}

The above view is the same as generated by vaadin-archetype-spring-application. However, it also includes an identifier for the added paragraph. Adding identifiers to components generally helps with writing and maintaining end-to-end tests.

Next, add a Java class to your test directory (e.g., srs/test/java/org/example/) that tests the MainView works properly. You would do that like this:

package org.example;

import com.microsoft.playwright.Browser;
import com.microsoft.playwright.Page;
import com.microsoft.playwright.Playwright;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.server.LocalServerPort;

import static com.microsoft.playwright.assertions.PlaywrightAssertions.assertThat;

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT) // (1)
@Tag("playwright") // (2)
public class TrivialPlaywrightTest {

    // This will be injected with the random free port
    // number that was allocated
    @LocalServerPort // (3)
    private int port;

    static Playwright playwright = Playwright.create(); // (4)

    @Test
    public void testClicking() {
        Browser browser = playwright.chromium().launch(); // (5)
        Page page = browser.newPage();
        page.navigate("http://localhost:" + port + "/");

        assertThat(page.getByText("Click Me")).isVisible(); // (6)

        page.locator("//vaadin-button[contains(text(),'Click Me')]").click(); // (7)

        assertThat(page.locator("#msg")).containsText("Clicked!"); // (8)

    }
}

Instructs Spring Boot to instantiate a full context with the web server before running this test. A random port for the web server is used so that you can keep a development server running and so it doesn’t disturb the executing of end-to-end tests.
End-to-end tests may be slow to execute. It’s a good idea to tag them with standard JUnit 5 annotation so that you can easily include or exclude them when you want.
Inject the random port to a variable so that it’s available for tests.
Playwright environment initialization only needs to be executed once.
Launch a new Chromium browser, open a page and navigate to the local port where Spring Boot started the application.
Playwright’s assert methods: do "implicit wait", automatically. When Playwright gets to the Click Me text locator, the element probably won’t be there yet as loading of the single-page Vaadin web application takes some milliseconds. The assertion still passes as Playwright will wait a while for the text to be present.
Use XPath selector to check for the element name and that it contains the text, Click Me. On the locator, simulate user action with the click method.
Asserts that there is an element with id msg in the page that contains the text, Clicked!. If you get instead the text using page.locator("#msg").textContent() and assert using standard JUnit API, it might fail as the server round-trip response might not yet be completed. Again, using the assertion method from Playwright helpers gives a bit of time for a single-page web application to render the response. Alternatively, you could add, for example, a page.getByText("Clicked!").waitFor(); line before the assertion to ensure the server round-trip has been completed.

Running the Test

As the test is annotated with the JUnit 5 @Test annotation, the most natural way to run it is via an IDE. Also, the test is picked up by convention if you call it like so:

mvn test

If you had previously written some unit tests for your project, you probably noticed that execution time increased by a couple of seconds. This is natural as a full server is started and Playwright launches a browser to execute the test.

You can use standard JUnit 5 and Maven features to include or exclude tests. As there is the playwright tag in the test, your can execute only the fast unit tests by executing the following:

mvn test -DexcludedGroups="playwright"

More about Playwright

For more information about using Playwright, check out these pages:

A8496E86-4D72-11EE-BE56-0242AC120002

Documentation