I’ve started to create a library to ease the test of a Vaadin application with Playwright.
The focus has been on each component to help how to assert or get information without knowing how they are structured internally.
For example if you want to test if a textfield is invalid and has an error message you can do this:
// get the textfield by accessible name
TextFieldElement textfield = TextFieldElement.getByLabel(page, "Validated Textfield");
textfield.setValue("1");
textfield.assertInvalid();
textfield.assertErrorMessage("Minimum length is 6 characters");
The API is different than TestBench since the project is relying on Playwright assertions and is trying to follow Playwright’s best practices: Best Practices | Playwright.
So, internally, to get the error message of the textfield I will build a errorMessageLocator then use Playwright assertions to run the tests.
The library has not been published yet in the directory and if you have any feedback on the API that would definitely help.
There is also the mopo library, we will very likely merge the 2 projects since it doesn’t make a lot of sense to have 2 Playwright wrapper for Vaadin.
In theory, it’s not Flow related but since it’s written in Java I guess a Hilla user will write the tests in Typescript. I don’t think there is an easy way to combine it in one library (make a playwright Typescript/Java library, but all the suggestions are welcomed).
I haven’t tried to migrate from Mopo.
But for the grid you should replace new GridPw(gridLocator) to new GridElement(gridLocator)
The grid is quite specific since there is no label, test, placeholder, alt text, title and most of the components should use the accessibleName.
I’m not sure what would be the recommended way for a Grid (In my opinion we should have an accessible name for our Grid).
Currently I would probably use the GridElement get(Page page) = get the first grid in the page or GridElement get(Locator parent) = get the first grid in this part of the page.
Playwright is recommending to make tests on something visual/meaningful.
Of course of your application is in multiple languages you should know in which language your tests is running.
I can also understand that you don’t want to change your tests if only a label has been updated.
Dramafinder is built based on this recommendation, but it’s fairly easy to use a id or testid.
Fine and good when your development and test teams are separate.
But I have spent many hours trying to figure out why my playwright code isn’t finding that “visual/meaningful” element. I’d bet anyone working in this area has the same experience. And that playwright “pick locator” is often useless when working with vaadin windows.
If the same developer/team is building both the application and the playwright test code, using IDs can save a lot of hassle. It is the least fragile option.
We’re not going to solve best practices here. Just useful to share experiences.
I’m curious, is the issue with windows something you’ve experienced also with Vaadin 25? Given that Dialogs and other overlays now use the browser-native popover feature, searching for contents inside could become easier; I haven’t had the time to test it yet myself.
Things get challenging, e.g., when you have a Dialog opened from another Dialog and have to restrict the search to the one with the “opened” attribute. In the past, I preferred to use “Locator inside a Locator” paths, but that does not play well with Dialog and Shadow DOM. I will not be surprised if some of my “make it work somehow” Locators break in Vaadin 25. A good thing, since it means better stability in the long run.
I only recently learned about the dramafinder project, and it has some approaches that are better than mine. Better to work on team code, rather than solo stuff, so I will move to that project over time.
Normally the changes in the overlays ( dialog, combo box,…) should make things easier but will break your tests. The content of the overlay is a slot so in light dom.
Dramafinder is quite recent. The first version was our after Vaadin 25.
Feel free to use it and create some bug or feature requests. I’m pretty sure they are flaky test methods or issues. But it’s hopefully much easier than using plain Playwright.
I am looking forward to using it and have “someone to share the burden with”.
The project looks very nice, although possibly lacking the inevitable dirty-hacks required for overcoming the brutal reality of UI testing:
For example, I used to rely on vaadin-menu-bar-item isEnabled() and then I couldn’t.
And today I might be discovering that vaadin-confirm-dialog between version 24 and 25 no longer responds “correctly/intuitively” to isVisible() and I might have to switch my code to one of isExists()/isEnabled()/attribute(“opened”). But only half a day lost to that, so only suspicions and not able to confirm. C’est la guerre.
