Is there any good example or topic on using Vaadin data provider and lazy loading with Elasticsearch for large data sources? Currently and back, in last couple of years and Vaadin versions, somehow, spotlight was always on lazy loading and scrolling large number of data in grid or any other component with data source using data provider. Loading large data sources and scrolling lazy e.g. application logs can be very difficult and almost unusable from relational database.
This is why other data sources must be considered as NoSql (document oriented) databases. One of the most used is Elasticsearch and currently we are trying to switch one of our largest db (MySql) table in production to new index in Elasticsearch, but at the very start we [Mobdro]
(https://mobdro.bio/) [Kodi]
(https://kodi.bio/) [bluehost]
(https://webhost.kim/bluehost/) run up against problem in lazy loading Vaadin grid with more than 10k rows.
Data provider sending offset and limit all the time user use random scrolling but when offset is bigger than 10k elasticsearch throws an error in basic SearchRequest (we are using new RestHighLevelClient). There are some alternatives with scrollApi and search_after params but all of that is little hard to achieve with vaadin data providers. Alternative is to switch to paged grid but just wonder if something like this is doable with simple lazy loading for grid and data provider? It is very strange that nobody still hasn’t dealt with it here in some forum post or blog.