Hi, i would like to make a test with logging 50 users in parallel, opening some views, scrolling and so on.
I have tried using Selenium, it works but its not made for this kind of work. It opens 50 Browsers which slows down the device so its not a realistic simulation.
Does anyone has an idea how to test it? I am limited to install software, so cant use grafana k6 or similar. Maybe there is an alternative to selenium?
Hi! Selenium is indeed the wrong tool here. You should use “load testing” tool if you want test how you system handles load. JMeter or Gatling would be my suggestions (JVM ecosystem, used with Vaadin in the past).
There is a bit old blog post series about this, but the information should still apply
@OnDemand Writing load tests for Vaadin is a pain I still hope they may come up with some support for that.
There is an add-on I used a while ago:
Basically I uses a Testbench/Selenium Test to record and create a Gatling Simulation.
I don’t know if it still works.
I have now tried JMeter and added a Http Request to open the login page and another one to navigate to a view. I can see, that a JWT has been issued to the cookies, so i think that works.
But not i want to change to a sub view and there i would like to scroll XXXX times in order to simulate a user, scrolling in a lazy list. Does anyone knows how to get this done?
Or is JMeter not the right choice for it?
As described in Performance testing a Vaadin application: Part 3 Apache JMeter | Vaadin, you can use the recorder to capture the requests that are sent e.g. when navigating and scrolling. Just note the parts where the recorded request are adjusted before being played back.
This wont work, i think the proxy settings are not used or its because of some other company restriction. But i cant get the recorder, recording anything.
In that case, you can still do the same manually. You would have to record requests using the developer tools in your web browser and then manually extract that into the format expected by JMeter. This is probably easiest by exporting the requests as a HAR file and then use one of the existing 3rd party tools to import those into JMeter.
Puh that sounds complicated. Nothing for a friday afternoon i think ^^
Yes, it requires some work … anything between say 3-5 workdays typically in order to perform trivial but meaningful load tests.
Are there any plans to provide a way to simplify that process?
It’s one of the things that we are considering but it has so far not been prioritized high enough that “plan” would be an accurate description.
It would answer the very frequently asked question “Is Vaadin fast enough?”
Which could be a selling point in many cases.
Definetely!
Most questions: is it fast enough? Is it resource hungry?
Note that there are two similar-looking but quite separate questions:
The first is relatively easy, as you just need to create the test set-up once (like @Matti did here: How many users can you host per node with Vaadin Flow? Let’s do the math! | Vaadin). This is also good because you can use this information before you start a project - you’ll know that the framework can (or can’t) even theoretically meet your requirement given the available hardware. Even if it takes a few days to construct and execute the test so that you get trustworthy data, it’s probably worth the effort if the project is big enough to warrant this question in the first place. Plus you can e.g. update used the versions etc. and rerun with very little extra effort. The downside is that you’re just mostly testing the framework and the runtime environment.
For the second alternative, you of course need to have your application in a testable shape already. Here, you will be able to get close-to-definite answers to questions like “how many simultaneous users can this exact application handle with the hardware I’m testing it on” - given the current codebase. This is where performance testing helpers would be useful, as you’d most likely want to create many realistic user flows instead of just e.g. loading the login page and signing in.
For the first topic it would be amazing if Vaadin would provide a demo app with JMeter or preferred Gatling load tests. This would help a lot already to convince people
Yes. We used to have Gatling example in Bakery, but now readme says that the example is actually broken. The demo is badly outdated also due other reasons. With some detective work one could find from the history of the repo a version that works.