We are getting random session timeouts. Sometimes you can be in the app for three hours with no problems and sometimes it times out after five minutes. It seems to correspondwith the heartbeat. You can see successful heartbeats and then sometimes it dies when the heartbeat is sent. Sometimes it is during interactions sometimes it is just sitting. Any ideas would be greatly appreciated.
Tomcat expire sessions with idle > 30 minutes
Tomcat has 20 gig of memory, uses average 3 gig
From the Catalina log - At the heartbeat time we get thousands of rows like this - then the destruction of the sessioin.
Apr 21, 2014 9:21:29 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered Label (27) that was already unregistered.
Apr 21, 2014 9:21:29 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered Label (28) that was already unregistered.
Apr 21, 2014 9:21:29 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered UserPanelMenuBar (29) that was already unregistered.
Apr 21, 2014 9:22:35 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered HorizontalSplitPanel (3990) that was already unregistered.
Apr 21, 2014 9:22:35 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered VerticalSplitPanel (3863) that was already unregistered.
Apr 21, 2014 9:22:35 AM com.vaadin.ui.ConnectorTracker unregisterConnector
WARNING: Unregistered MainView (3862) that was already unregistered.
Apr 21, 2014 9:22:35 AM com.vaadin.server.DefaultErrorHandler doDefault
SEVERE:
com.vaadin.event.ListenerMethod$MethodException: Invocation of method sessionDestroy in ifp.Common.IFPServlet failed.
at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:528)
at com.vaadin.event.EventRouter.fireEvent(EventRouter.java:193)
at com.vaadin.server.VaadinService$1.run(VaadinService.java:464)
at com.vaadin.server.VaadinSession.accessSynchronously(VaadinSession.java:1251)
at com.vaadin.server.VaadinService.fireSessionDestroy(VaadinService.java:440)
at com.vaadin.server.VaadinService.cleanupSession(VaadinService.java:1141)
at com.vaadin.server.VaadinService$4.run(VaadinService.java:1324)
at com.vaadin.server.VaadinSession.accessSynchronously(VaadinSession.java:1251)
at com.vaadin.server.VaadinService.requestEnd(VaadinService.java:1321)
at com.vaadin.server.VaadinService.handleRequest(VaadinService.java:1401)
at com.vaadin.server.VaadinServlet.service(VaadinServlet.java:238)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at com.vaadin.data.util.sqlcontainer.query.TableQuery.fetchMetaData(TableQuery.java:659)
at com.vaadin.data.util.sqlcontainer.query.TableQuery.(TableQuery.java:189)
at com.vaadin.data.util.sqlcontainer.query.TableQuery.(TableQuery.java:127)
at com.vaadin.data.util.sqlcontainer.query.TableQuery.(TableQuery.java:105)
at com.vaadin.data.util.sqlcontainer.query.TableQuery.(TableQuery.java:147)
at ifp.DAL.Base.DatabaseBase.getTableQuery(DatabaseBase.java:68)
at ifp.DAL.Base.DatabaseBase.getSQLContainer(DatabaseBase.java:78)
at ifp.DAL.Base.DatabaseBase.getSQLContainerByFieldName(DatabaseBase.java:105)
at ifp.DAL.Base.DatabaseBase.getSQLContainerByPrimaryId(DatabaseBase.java:98)
at ifp.DAL.DataAccess.UserHistoryData.AddLogoutTime(UserHistoryData.java:30)
at ifp.Common.UserAuthenticate.logout(UserAuthenticate.scala:127)
at ifp.Common.IFPServlet.sessionDestroy(IFPServlet.java:72)
at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:508)
… 30 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1304)
at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1296)
at com.mysql.jdbc.ConnectionImpl.getMetaData(ConnectionImpl.java:3239)
at com.mysql.jdbc.ConnectionImpl.getMetaData(ConnectionImpl.java:3234)
at com.vaadin.data.util.sqlcontainer.query.TableQuery.fetchMetaData(TableQuery.java:609)
… 45 more
Secondly, I see a different cause in your stacktrace -
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed
Did you check this out? It would be great if you can share your stripped down version of your sources so that either me or some one from the vaadin team can check.
Just in case, if you are using older version of Vaadin, upgrade the same to the latest 7.1.x
We have looked at that. We don’t think our MySQL connections are the issue. We are’nt seeing them drop.
wait_timeout = 1860
interactive_timeout = 1860
we are using connection string with
autoReconnect=true&validationQuery=SELECT 1&testOnBorrow=true
There is way to much code to strip down and have it still work or make sense.
Yes, I have seen and solved that problem. First a newbie alert - I lack deep insight here so my theories might be off the mark - a little or a lot. But the solution did fix it for me.
Theory:
It seems that the first vaadin 7.1 UI starts a session that never drops. At least when push is enabled.
That session opens a JDBC connection pool in my app as in yours.
Other connections pools may be opened by other UIs and work normally. But if the first session becomes inactive, the JDBC connection associated with it goes inactive. After 8 hours the initial conneciton pool times out. But the session remains alive. The first user who tries to use a UI associated with that session and its connection pool after the timeout gets your “no operations…” message. The app is trying to use a conneciton pool that’s timed out and gone away.
The solution for me was to find the point in the code where an access to the database is first made when a browser is refreshed. There, instead of trying to use the connection pool, assume the connection may have expired. Test for the connection pool by doing an arbitrary access ( such as “int size = planeContainer.size();” ) on any SQL container based on the database. In the try/catch block, catch any exception and use that exception block to close, then re-open the connection pool, and re-initialize all associated containers. If the pool is OK, use it, otherwise re-establish it before continuing.
Doesn’t sound too elegant, does it? I’m hoping for better session lifecycle handling in Vaadin 7.2 and push (#12139, 12843?), then will reassess all this to find a real solution. For now, it works around the problem. I tried to use session detach listeners to better handle the connection pool but when the session doesn’t go away, the listeners hear only silence.
We do have serveral divs and frames. They basically define an area that holds SVG graphics.
We are not calling invalidate() anywhere.
We are not using refresher, push, or UI poll. We may need to look into these technologies.
No, It is not reproducable. Sometimes it happens, sometimes it doesn’t. It does not happen while using a certain function and there is no pattern to it.
I took that out of the connection string and we now don’t get the error you noted before.
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
We don’t get any tomcat error logged now when it happens.
Watching the connections, they look okay and don’t seem to have any relation to the session timeout.
I think I am going to look at doing something like you suggested testing the connection pool but here is the current condition that I described to Krishna.
I tracked down some information on connecting to the MySQL DB with “autoReconnect=true”. http://stackoverflow.com/questions/667289/why-does-autoreconnect-true-not-seem-to-work
I took that out of the connection string and we now don’t get the error you noted before.
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
We don’t get any tomcat error logged now when the session timeout happens.
I just ran across a link to this and thought i should update it with the solution. We were seeing many timeout at random times. The timeout was at an hour and the heartbeat 5 min. Sometime it would work and sometimes it didn’t. Sometimes you would be in for 1 min and you would time out. Sometimes 23 min.
We finaly noticing severe errors in the java logs. We fixed the null pointers and other issues. It seemed to get a bit better.
Then we addressed all the ambiguous warnings that look like this.
Sep 09, 2014 8:20:07 AM com.vaadin.data.util.sqlcontainer.SQLContainer containsId
WARNING: containsId query failed
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column ‘SiteGroupId’ in where clause is ambiguous
at sun.reflect.GeneratedConstructorAccessor82.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
and addressed all the toString warnings that look like this. As it says, you have to turn on fine logging to get it the source.
Jun 27, 2014 8:52:28 AM com.vaadin.data.util.LegacyPropertyHelper maybeLogLegacyPropertyToStringWarning
WARNING: You are using toString() instead of getValue() to get the value for a Property of type com.vaadin.ui.TextField. This is strongly discouraged and only provided for backwards compatibility with Vaadin 6. To disable this warning message and retain the behavior, set the init parameter “legacyPropertyToString” to “true”. To disable the legacy functionality, set “legacyPropertyToString” to false. (Note that your debugger might call toString() and trigger this message). To find out who is calling toString(), enable FINE level logging.
Jun 27, 2014 8:52:28 AM com.vaadin.data.util.LegacyPropertyHelper maybeLogLegacyPropertyToStringWarning
FINE: Strack trace for legacy toString to ease debugging
java.lang.Throwable
at com.vaadin.data.util.LegacyPropertyHelper.maybeLogLegacyPropertyToStringWarning(LegacyPropertyHelper.java:64)
at com.vaadin.data.util.LegacyPropertyHelper.legacyPropertyToString(LegacyPropertyHelper.java:47)
at com.vaadin.ui.AbstractField.toString(AbstractField.java:400)
And the session timeouts went away. Odd that the
warnings seem to be the issue.
Aminul Karim:
did somebody solved the problem , I’m having a similar issue , we have server push enabled
Same issue for our application using Vaadin 8. We set the session timeout to 200 mins. But it randomly gets session time out for sometime. From my understanding if the heartbeat duration(default to 5 mins) less than the session timeout duration(200 mins for our app) and closeIdleSessions=false, it should never get session timeout, right?
Will we set the heartbeat duration to 1 min reduce this occurrance?
