!!! Problem in Vaadin 8 Grid + Lazy Loading + Big Data

Vaadin 6 / 7 / 8
1

Now explore the opportunity to upgrade the project to vaadin 8.
Faced with this problem:
A large selection of data(over million records), the task is to display it in grid using lazy loading with the scrolling.
Source data set setDataProvider(FetchItemsCallback, SerializableSupplier)
When loaded, the Grid does not display rows farther than 883050(± 30 depending on table height) engine scrolling reaches the end of the table and the remaining rows are not visible.
That is, the parameters (offset + limit) for a query is not calculated over this number. I think, that doesn’t depend on the size of the selection if it exceeds this limit (tried with 1.5 million and 4 million records).
If the sample is less than this limit, everything works fine.
As the second parameter to the setDataProvider use count of the selection. Can’t understand where it ‘breaks down’ for large selections.

2

Hallo Sergey,

I use Spring Data.
The
springframework.data.domain.
Page

 of the Repository of Spring Data has the count of all Elements.

Example:

Page<AnyObject> page = _repository.findAll(new PageRequest(page, limit, null));
long count = page.getTotalElements();

You can use this count. But you need two JPA requests…

This is my method to build a CallbackDataProvider

public static <T> CallbackDataProvider<T, Integer> build(PageCommand<T> pageCommand) {
    //First Request with page and limit = 1 
    long totalElements = pageCommand.findPages(1, 1).getTotalElements();
    return new CallbackDataProvider<T, Integer>(new CallbackDataProvider.FetchCallback<T, Integer>() {     
         @Override
        public Stream<T> fetch(Query<T, Integer> query) {
            // Seccond Request with page and limit = query.getOffset(), query.getLimit()
            Page<T> page = pageCommand.findPages(query.getOffset(), query.getLimit());
            return page.getContent().stream();
        }
    },
    new CallbackDataProvider.CountCallback<T, Integer>() {
        @Override
        public int count(Query<T, Integer> query) {
          return (int) totalElements;
        }
    });
}

public interface PageCommand<T> {
  Page<T> findPages(int offset, int limit);
}
3

Hi, Samuel!
Thanks for the reply!
Attached Your method to my grid :

tempGrid.setDataProvider(build((offset, limit) ->
personRepository.findAll(new PageRequest(offset / limit, limit))))

Unfortunately, nothing has changed. Can’t scroll through the table on 883056 line when the value of totalElements is 1.800.000 rows.
Tried to work with Vaadin versions 8.0.5, 8.1.0alpha3

4

I Use Vaadin 8.0.5, Spring-Boot 1.5.2.RELEASE, Spring Data and MSSQL… but my Page.totalElements has max 200.000 rows…

My example above comes from this URL:

https://github.com/alejandro-du/lazy-loading-spring-demo

perhaps you may find some more help there…

5

I also have it works fine if less then 883.000 lines
I talked with support and they said that probably this is a bug of the framework.

thanks

6

For the record, this is a limitation in how big elements different browsers support. IE is the most restrictive browser by only allowing about 40 000 rows with the default row height, whereas other browsers have other limits.

This limitation can be worked around by internally changing the way Grid handles scrollbars, but it’s not exactly trivial.

The issue has originally been reported in
https://github.com/vaadin/framework/issues/6290
.