Test failures in flow repo

I tried to run “mvn clean test” in the root folder of the flow repo for a project but that results in test failures in flow-client and most following tests. When running just the “mvn clean test -pl flow-client” like the docu says it works fine but some tests in other parts still fail when run that way.

Is that a configuration issue on my end, an issue with the repo or intended behaviour?

If you want to run the whole test suite, use mvn verify

If I do that it fails at flow-polymer-template with some license issue

[main] INFO com.vaadin.pro.licensechecker.LicenseChecker - Opening system browser to validate license. If the browser is not opened, please open https://vaadin.com/pro/validate-license?connect=34049176-0e32-4828-8a6d-842f27162d64&productName=flow-polymer-template&productVersion=24.2-SNAPSHOT manually [main] INFO com.vaadin.pro.licensechecker.LicenseChecker - For CI/CD build servers, you need to download a server license key, which can only be used for production builds. You can download a server license key from https://vaadin.com/myaccount/licenses.

Do I need to have a commercial lincense to run the tests?

flow-polymer-template is a commercial component, so you need a license. But you can skip the module adding -pl \!:flow-polymer-template to the command line

Thanks for the quick responses. Very helpful ^^

Now I get failures in Vaadin Development Mode Server:

[ERROR] Failures: [ERROR] OpenInCurrentIdeTest.ideaOnMacDetected:47 expected:<...ij980041274492389461[\MacOS\]idea> but was:<...ij980041274492389461[/MacOS/]idea> [ERROR] ThemeModifierTest.multipleRulesAdded_singleImportPresent:94 [ERROR] ThemeModifierTest.noImport_propertyAdded_importPresent:79 [ERROR] Errors: [ERROR] OpenInCurrentIdeTest.ideaOnLinuxCustomJavaAgentDetected:154 » NoSuchElement No value present [ERROR] OpenInCurrentIdeTest.ideaOnLinuxDebugModeDetected:118 » NoSuchElement No value present [ERROR] OpenInCurrentIdeTest.ideaOnLinuxDetected:80 » NoSuchElement No value present

I’m using IDEA on windows so I guess that makes sense? I guess I have to skip that as well.

for OpenInCurrentIdeTest it doesn’t matter your environment since test values are mocked, not real

I wonder why it is failing for you

Ah, ok. It is a file separator issues
In OpenInCurrentIdeTest, can you please try to replace all occurences of String baseDir = baseDirectory.getAbsolutePath(); lines with String baseDir = baseDirectory.getAbsolutePath().replace(File.separatorChar, '/'); and run the tests again?

If it works, the test can be fixed. You can even open a PR on github if you like :wink:

That seems to result in in a slightly different error

[ERROR] Failures: [ERROR] OpenInCurrentIdeTest.ideaOnMacDetected:47 expected:<C:[\Users\Leonard\AppData\Local\Temp\testIntellij14546433079516880941\MacOS\]idea> but was:<C:[/Users/Leonard/AppData/Local/Temp/testIntellij14546433079516880941/MacOS/]idea> [ERROR] ThemeModifierTest.multipleRulesAdded_singleImportPresent:94 [ERROR] ThemeModifierTest.noImport_propertyAdded_importPresent:79 [ERROR] Errors: [ERROR] OpenInCurrentIdeTest.ideaOnLinuxCustomJavaAgentDetected:154 » NoSuchElement No value present [ERROR] OpenInCurrentIdeTest.ideaOnLinuxDebugModeDetected:118 » NoSuchElement No value present [ERROR] OpenInCurrentIdeTest.ideaOnLinuxDetected:80 » NoSuchElement No value present

If I change the separatorChar back to ‘\’ I got the error messages from before.
I think I might use -fae or -fn to get through all the tests.

When I try to skip failing tests (which were just that one and all the ones that require a commercial license) it still gets stuck at
‘com.vaadin:flow-test-live-reload-multimodule-ui-hotdeploy’ at ‘prepare-frontend’. Might be npm related, will try it with ‘mvn clean verify’ again later.

I tried the test on Windows, but I didn’t get the errors, neither with original code :thinking:

I wonder what may be different in your environment

The error didn’t appear on my desktop and after changing the location of the repo it resolved most errors. I still have a some left though, could you help with that?

This error appeared in several modules, later als with status 400.
com.vaadin.flow.devbuild.ReusingThemeIT.reusableTheme_stylesAppliedFromReusableTheme_devBundleCreated[any_Chrome_] -- Time elapsed: 2.548 s <<< FAILURE! java.lang.AssertionError: Error message in browser log: [2023-09-05T13:27:08.344Z] [SEVERE] http://localhost:8888/view/themes/reusable-theme/fortawesome/icons/snowflake.svg - Failed to load resource: the server responded with a status of 500 (Server Error)

And in test-spring-security-flow:
org.openqa.selenium.TimeoutException:
Expected condition failed: waiting for condition to not be valid: com.vaadin.flow.spring.flowsecurity.AbstractIT$$Lambda$3435/0x000000080150d0d8@78a78257 (tried for 10 second(s) with 500 milliseconds interval)