we’re running load tests on our application and are seeing the exception below. It happens intermittently: for a set of 4000 tests we have 200 errors, approximately. It seems to be a problem in Vaadin.
Server is a Tomcat, Java is 8.
Any ideas anyone?
Thanks in advance
SCHWERWIEGEND: Servlet.service() for servlet [de.abc.zyx.AppUI$Servlet]
in context with path
[/app-context] threw exception [com.vaadin.server.ServiceException: java.util.ConcurrentModificationException]
with root cause
java.util.ConcurrentModificationException
at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:711)
at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:734)
at java.util.AbstractCollection.toArray(AbstractCollection.java:141)
at com.vaadin.event.EventRouter.fireEvent(EventRouter.java:188)
at com.vaadin.server.VaadinService.onVaadinSessionStarted(VaadinService.java:814)
at com.vaadin.server.VaadinService.createAndRegisterSession(VaadinService.java:771)
at com.vaadin.server.VaadinService.doFindOrCreateVaadinSession(VaadinService.java:724)
at com.vaadin.server.VaadinService.findOrCreateVaadinSession(VaadinService.java:665)
at com.vaadin.server.VaadinService.findVaadinSession(VaadinService.java:524)
at com.vaadin.server.VaadinService.handleRequest(VaadinService.java:1402)
at com.vaadin.server.VaadinServlet.service(VaadinServlet.java:350)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
SCHWERWIEGEND: Servlet.service() for servlet [de.abc.zyx.AppUI$Servlet]
in context with path
[/app-context] threw exception [com.vaadin.server.ServiceException: java.lang.NullPointerException]
with root cause
java.lang.NullPointerException
at de.abc.zyx.AppUI.prepareSubmenu(AppUI.java:87)
at de.abc.zyx.AppUI.init(AppUI.java:69)
at com.vaadin.ui.UI.doInit(UI.java:675)
at com.vaadin.server.communication.UIInitHandler.getBrowserDetailsUI(UIInitHandler.java:214)
at com.vaadin.server.communication.UIInitHandler.synchronizedHandleRequest(UIInitHandler.java:74)
at com.vaadin.server.SynchronizedRequestHandler.handleRequest(SynchronizedRequestHandler.java:41)
at com.vaadin.server.VaadinService.handleRequest(VaadinService.java:1408)
at com.vaadin.server.VaadinServlet.service(VaadinServlet.java:350)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
The CME looks like something to worry about. I’m not deeply familiar with the workings of
VaadinService , but I suppose there could be a problem with synchronization in the
EventRouter when firing those SessionInitEvents. If they are fired simultaneously from multiple threads during load testing, I suppose they could be a problem. Requests are otherwise synchronized on the sessions, and they are also locked on a new session in findOrCreateVaadinSession() that calls the methods eventually firing the events… The session handling looks a bit complex, so I hope there’s no problem with how that works… I’d find it suprising if there’s such a basic synchronization malfunctioning.
The latter exception is from your code, so maybe it’s unrelated?
This Bug was already found. Have a look at EventRouter, Method fireEvent
public void fireEvent(EventObject event, ErrorHandler errorHandler) {
// It is not necessary to send any events if there are no listeners
if (listenerList != null) {
// Make a copy of the listener list to allow listeners to be added
// inside listener methods. Fixes #3605.
Ticket 3605: Patch to avoid concurrent modification exceptions in EventRouter
Hmm. If it was fixed already in 6.2, how are you still experiencing the problem… The ticket seems to deal with a typical cause of CMEs, when you add or delete to a collection while iterating over it. That’s maybe different from your case. It’s quite possible that load testing would find a locking problem that causes CMEs because of modifications from simultaneously running threads.
In any case, I recommend using the -ea parameter and see if you get more detailed information.
I tried running the load test and the problem occurs to me as well. The -ea parameter didn’t seem to have any effect (if I used it correctly). It’s hard to say anything without close debugging, which can be difficult to do for this kind of a problem. I hope the problem is not caused by JMeter testing, but I guess not.
I really suspect that there’s some problem with the session locking, which seems to use a bit unusual pattern, and if so, the requests served in different threads could go through unlocked to the EventRouter, where they cause the CMEs.
I’m currently working on a patch and making good progress, since I’ve fixed the issue in my case. The problem is, that the fix provided in #3605 only makes the ConcurrentModificationException more unlikely to occur but doesn’t prevent ist completely. When the listenerList is copied by .toArray() a iterator object is still used. Adding a new Listener at the wrong time might cause a CME
So your fix uses a thread-safe set class… I’d think that the real problem is that the thread lock is leaking, so using a thread-safe set is just fixing the symptom. Well, whoever looks into the issue probably considers it, so it’s good to have your patch there as one solution or workaround anyhow.
Note that your patch contains a some unnecessary changes; I hope that you are using the correct code autoformatting rules for the Vaadin project. Oh, looks like the development instructions do not even mention that anywhere anymore…
OK, looks like we haven’t mentioned the coding conventions info in the GitHub repository info nor in the Contributing Code page. You can find the old and partially outdated version at
https://dev.vaadin.com/wiki/CodingConventions . The needed VaadinCleanup.xml and VaadinJavaConventions.xml files can be found under the
eclipse folder in the Vaadin Framework repository.
Unfortunately, after doing the step-by-step instructions eclipse keeps formatting lines I didn’t touch. Since the patch won’t find the way into any official release, I guess it’s applicable to extract relevant lines as long as there is no offical fix available. However I’m curious about an offical fix for this issue
