All users force-logged out at once for a few minutes, then it self-recovers

Flow
1

All users force-logged out at once for a few minutes, then it self-recovers

Environment

  • Vaadin Flow 24.10 (productionMode: true)
  • Spring Boot 3.5.3, Spring Security
  • JDK 21
  • Packaged in Docker, single container instance, behind an nginx-family reverse proxy
  • Multi-datasource: 1 MySQL (core / app data incl. users & permissions) + several SQL Server read pools, all HikariCP
  • Server-side sessions (currently verifying whether Spring Session is in play or plain in-memory Tomcat sessions)

Symptom

Every so often, with no fixed schedule, every logged-in user is bounced to the login page at the same moment.

  • Lasts ~1 to several minutes
  • Recovers on its own — no restart, no redeploy
  • Logging back in doesn’t help: a fresh session gets bounced again on the next navigation click
  • Container stays up. No OOM, no crash

That last point is the interesting one. A brand-new session shouldn’t be affected if this were session expiry.

The likely mechanism

We have a global navigation guard — a BeforeEnterListener registered via VaadinServiceInitListener — that queries the DB for the user’s permissions on every navigation event. If it fails or throws, it reroutes to login.

So any DB slowness → permission lookup fails → reroute to login → indistinguishable from mass session expiry. And it explains the re-login behaviour, since the check is per-navigation, not per-session.

Already addressed / ruled out

  • Hikari was tuned badly: connection-timeout: 60000 (requests block a full minute instead of failing fast), minimum-idle: 2 vs maximum-pool-size: 50, short max-lifetime, no socketTimeout. All being fixed.
  • Permission lookup is uncached — adding Caffeine @Cacheable regardless of root cause.
  • DEBUG on com.zaxxer.hikari + org.apache.catalina.session, Tomcat access log with %D %S, and a jcmd thread dump script (actuator on the same connector is useless during thread exhaustion).
  • Frequent Atmosphere IllegalStateException: request object has been recycled warnings — believed benign, but they seem to cluster around the logout events.

Questions

  1. Has anyone seen this exact shape — simultaneous, minutes-long, self-recovering, random timing? What was it?
  2. Is a DB-querying BeforeEnterListener a known antipattern? It turns every latency spike into an auth-looking failure. Is there a better place for permission checks in Flow, or is the answer just “cache it and never touch the DB in a guard”?
  3. Session lock: Flow serializes requests per session. If a navigation request holds that lock while blocked on a slow DB call, does everything else for that user queue behind it? Any recommended guard against blocking I/O under the session lock?
  4. Are those Atmosphere recycled-request warnings truly harmless, or can they signal push connections dying in a way that correlates with session loss?
  5. Is addSessionDestroyListener the right way to definitively separate “session actually died” from “guard rejected navigation”? Anything else in Flow that logs why a user was redirected?

Happy to share thread dumps, pool metrics, or config once I capture a clean incident window. Pointers especially welcome from anyone running Flow with a permission check on the navigation path.

2

  1. IMHO if you use JWT for authentication I could understand why you would also query the database to verify. Without it, I would say: query the user once and store it in your server side http session. If you are really worried, refresh this instance every x minutes or on critical actions - but not on every navigation. The before enter listener should query the in-memory-user and his roles. This makes navigation faster and less database reliable.

  2. yes, destroy listener sounds fine. Just a note: If your guard redirects the user, you have full control of the source code and your guard can log that the user is missing permissions

1 Like