Blog

Introduction to load testing with Apache JMeter

By  
Alejandro Duarte
Alejandro Duarte
·
On Feb 4, 2021 8:43:16 PM
·

Load testing allows you to simulate multiple users interacting with your application. This allows you to experiment with a real-world load on your application and understand its behavior under different levels of stress.

In this article, I'll show you, step by step, how to run load tests on a simple Vaadin Flow Java web application using an open-source tool called Apache JMeter. You can download the example application at https://github.com/vaadin/skeleton-starter-flow/archive/v14.zip.

You can also watch a detailed video tutorial on this topic on YouTube.

Building an application for load testing

You should run load tests against an application that is as close as possible to your production build. Vaadin Flow applications can be built in two different ways—development or production. For load testing, you want to build the application in production mode.

To build and run the application in production mode, open a terminal window, change to the directory in which you extracted the zip file, and run the following command:

cd skeleton-starter-flow-14
mvn package jetty:run -Pproduction -DargLine="-Xmx8m"

 

The argLine option allows you to configure the JVM. In the example above, we are setting a max heap size of 8MB just to demonstrate more easily how to perform stress testing later in this article. In real-life projects, you should configure the JVM to mimic the production environment.

Check that you can use the application in your browser at http://localhost:8080:

figure01

Creating a new test plan in JMeter

Download, install, and run JMeter. You can find the binaries and instructions at https://jmeter.apache.org.

In the JMeter menu, select File > Templates…. Next, select Recording in the combo box and click Create:

figure02

Change the following configuration details and click Create:

  • hostToRecord: localhost
  • schemeToRecord: http

You should see a test plan with preconfigured elements for cookies management, a thread group, and a script recorder, among others. A test plan is equivalent to a project in an IDE. It contains all the configuration required to run the load tests. A thread group contains the sequence of requests you want to perform against your application. Each sequence is run in a separate thread.

Recording a test plan in a web browser

JMeter includes a proxy that allows you to record the interactions with a web application using a web browser. The proxy listens to port 8888. Configure a manual proxy using the network settings of your browser. For example, in Firefox the configuration dialog looks like this:

figure04

Before starting the script recording, check your local IP address. For example, in Linux systems run:

ifconfig | grep inet

 

You'll find a line with your local IP similar to this:

inet 192.168.1.35 netmask 0xffffff00 broadcast 192.168.1.255

 

Use the reported IP address in your browser (Firefox in this example) to request the application, for example: http://192.168.1.35:8080. You'll get a proxy error and won't see the application at this point.

In JMeter, select HTTP(S) Test Script Recorder in the left view and click the start button:

figure05

You'll see a message about an SSL certificate. Click OK. Enter a Transaction name, for example, "home" since we are testing the home page of the web application:

figure06

Go to the browser (Firefox) and request the application again. This time you should see the app. Enter a name and click Say hello in the webapp.

In the Recorder: Transactions Control popup window, click Stop to stop recording. In JMeter, you should see the results of the interaction when you select View Results Tree under HTTP(S) Test Script Recorder:

figure07

You might have a different number of requests (and generated names), depending on exactly how you interacted with the application. For example, if you hit ENTER or clicked Say hello before a change event was fired on the text field or closed the browser before stopping recording, you might end up with different requests.

You should also be able to see the recorded requests under Test Plan > Thread Group > Recording Controller:

figure08

These are the requests that are part of the script to run later.

Extracting data from HTTP responses

To be able to run the test, you have to modify the recorded requests to use the CSRF token that Vaadin Flow uses as a security mechanism. This token is returned to the client in the body of the response to the first request to the application (home-12 in the example) in a JSON object that you can inspect in the results tree. Vaadin Flow returns a different token every time the app is requested for the first time. This token is included on later AJAX calls to the server, for example, when you click a button. You can see the value used during the recording in the results tree by selecting the first request and searching for Vaadin-Security-Key:

figure09

To extract the value of the token, right-click the first request in the thread group and select Add > Post Processors > Regular Expression Extractor:

figure10

Fill in the following information:

  • Name of the created variable: csrfToken
  • Regular Expression: "Vaadin-Security-Key":"(.{36})"
  • Template: $1$

This creates a variable with the name csrfToken that you can later use with the expression ${csrfToken}. The variable will contain the value of the first capturing group on the regular expression, that is, the 36-character string that contains the value of the CSRF token.

If you click the second request (home-17 in the example), you'll see that the request includes the value of the CSRF token in a JSON object:

figure11

You have to replace this "hard-coded" value in all subsequent requests with the expression that "injects" the value of the csrfToken variable extracted by the regular expression from the first request. You can edit the Body Data section directly to replace the content of the string with ${csrfToken} or you can use JMeter’s search and replace feature to do it all at once. For example:

figure12

Alternatively, with search and replace:

figure13

Running load tests

With the configuration in place, you can now run the test. Select View Results Tree in the thread group and click the start button in the toolbar:

figure14

You should see successful requests in the view:

figure15

Try simulating many concurrent users by adjusting the number of threads in the Thread Group as follows:

  • Number of threads (users): 1000
  • Ramp-up period (seconds): 5
  • Deselect the Same user on each iteration checkbox

Before running the experiment, right-click Test Plan and select Add > Listener > Summary Report:

figure16

Run the test again by clicking the start button on the toolbar. On my machine, the results were a success (0% errors):

figure17

This could be different in your case, depending on the resources of your machine. A conclusion of this experiment is that the application can successfully serve 1000 users arriving concurrently (within a time window of 5 seconds) with as little Java heap memory as 8MB.

Running stress tests

Stress testing pushes the limits of your application to see what happens when it is put under pressure. It's a way to test the robustness of your application. You can also use it to find the spot at which you will need to increase computing resources, such as memory, processing units, database pool connections, etc.

Try modifying the number of concurrent users (threads) to a much higher level, for example to 3000, and run the tests again. Remember to clear the results and summaries before running the tests. You can do so by selecting Run > Clear All in the main menu (or use the keyboard shortcut shown in the menu).

On my machine, 3000 users were a little too many and I started to get errors in the responses at some point:

figure18

This is most likely a combination of not enough computing power and memory. Let's change the memory parameter (since it's the easiest thing to do in this example), stop the application and run it again with more heap memory:

mvn jetty:run -Pproduction -DargLine="-Xmx256m"

 

Clear the results and rerun the test. On my machine, that change did the trick:

figure19

Again, keep in mind that you might get very different results, depending on the hardware you use to run and test the application and the other programs you have running at the same time. In real-life scenarios, you should have separate environments for running the app and running the tests. For example, you can have a test server where you deploy and run your test application and use your development machine to run JMeter and execute the tests.

JMeter includes many more features than those we explored here. I encourage you to take a look at their documentation to learn more about it.

Alejandro Duarte
Alejandro Duarte
Software Engineer and Developer Relations Manager at Vaadin Ltd. Author of Practical Vaadin (Apress), Data-Centric Applications with Vaadin 8 (Packt), and Vaadin 7 UI Design by Example (Packt). Passionate about software development with Java and open-source technologies. Contact him on Twitter @alejandro_du or through his personal blog at www.programmingbrain.com.
Other posts by Alejandro Duarte