Limit for CallbackDataProvider bigger than pageSize of Grid

Flow
1

Hello,

considering the following example code:

Grid<Integer> grid = new Grid<>();
grid.setPageSize(20);
grid.addColumn(i -> i).setHeader("value");

CallbackDataProvider<Integer, Boolean> dataProvider = DataProvider.<Integer, Boolean>fromFilteringCallbacks(
        query -> {
            AtomicInteger value = new AtomicInteger(query.getOffset());
            System.out.printf("offset: %s, limit: %s%n", query.getOffset(), query.getLimit());
            return Stream.generate(value::getAndIncrement).limit(query.getLimit());
        },
        query -> 10_000);
grid.setDataProvider(dataProvider);

I am getting the following output when scrolling fast in the grid:

offset: 0, limit: 20
offset: 20, limit: 40
offset: 660, limit: 80
offset: 1840, limit: 80
offset: 2520, limit: 100
offset: 5120, limit: 100
offset: 5260, limit: 100
offset: 5360, limit: 20

My expectation was, that limit is ALWAYS as high as determined via setPageSize-Method of the grid (so 20 in this example). But on fast scrolling the limit is bigger. Vaadin also seems to not accept less elements (fixing the limit of my stream to 20 will result in IndexOutOfBoundsException)

Considering I have an API which allows max. 20 elements to be fetched I would need to combine multiple api-calls for a single fetching call.

My assumption seems to be confirmed by documentation of [Grid.setPageSize]
(https://vaadin.com/api/platform/14.1.27/com/vaadin/flow/component/grid/Grid.html#setPageSize-int-) and note in this [example]
(https://vaadin.com/docs/flow/binding-data/tutorial-flow-data-provider.html):

The number of items that need to be fetched, query.getLimit(), is set by the component that uses the DataProvider. For example, in Grid component the default number is 50. This number can be changed via its constructor, like Grid grid = new Grid<>(20);, or via its setPageSize method, like grid.setPageSize(20);.

Could anybody please enlighten me if this is a bug or if simply my assumption is wrong?

Best Regards
Olli

2

Hi Olli,

I had this same issue and noticed the same thing. See my comments in this [post]
(https://vaadin.com/forum/thread/17843501/grid-grid-pro-indexoutofboundsexception).

In my experience if you set the size to 20 you will get multiples of 20. As I am using Spring data jpa which works on a page and offset basis I had to call it several times (e.g. 5 for a 100) so that it satisfied the request. I made myself a generic class to do this which I am willing to share.

Mark.

3

Hi Mark,

thanks for your reply. Its a bit annoying that the dataprovider isn’t using the pageSize as documented but ok, I will work around that.

I am not using Spring JPA but Spring resttemplate. But for others who might stumble across this thread it would be very nice if you could share your generic class :slight_smile:

Thanks.
Olli

4

I wrote some methods to help cope with the difference between Spring Data JPA and the provider for a Vaadin Grid. The class is extended from Grid:

//--------------------------------------------------------------------------
public void fromCallbacks(FetchCallback<T, Void> fetchCallback, CountCallback<T, Void> countCallback) {
	setPageSize(MyConstants.DB_PAGE_SIZE);
	setDataProvider(DataProvider.fromCallbacks(
			// First callback fetches items based on a query
			query -> {
				int offset = query.getOffset(); // The index of the first item to load
				int limit = query.getLimit(); // The number of items to load
				int limitRemaining = limit;
				List<T> lst = new ArrayList<>();
				while (limitRemaining + MyConstants.DB_PAGE_SIZE > MyConstants.DB_PAGE_SIZE) {
					Stream<T> stm = fetchCallback.fetch(query);
					List<T> fcs = stm.collect(Collectors.toList());
					lst.addAll(fcs);
					limitRemaining -= MyConstants.DB_PAGE_SIZE;
					offset += MyConstants.DB_PAGE_SIZE;
					Query<T, Void> query2 = new Query<>(offset, limit, query.getSortOrders(), query.getInMemorySorting(), null);
					query = query2;
				}
				while (lst.size() > limit) {
					lst.remove(lst.size() - 1); // remove records if we have too many
				}
				log.info("Record list has {} entries, offset was {}, limit was {}", lst.size(), offset, limit);
				return (lst.stream());
			}, query -> {
				int count = countCallback.count(query);	// number of items for a query
				log.info("Record count is {}", count);
				return (count);
			}));
}

//--------------------------------------------------------------------------
private static PageRequest makePageRequest(List<QuerySortOrder> sortOrders, int queryOffset) {
	List<Order> orders = new ArrayList<>();
	sortOrders.forEach(qso -> {
		String fieldName = qso.getSorted();
		if (qso.getDirection() == SortDirection.ASCENDING) {
			Order order = Order.asc(fieldName);
			orders.add(order);
		}
		else if (qso.getDirection() == SortDirection.DESCENDING) {
			Order order = Order.desc(fieldName);
			orders.add(order);
		}
	});
	log.info("Number of order fields is {}", orders.size());
	PageRequest pr = PageRequest.of(queryOffset / MyConstants.DB_PAGE_SIZE, MyConstants.DB_PAGE_SIZE, orders.isEmpty() ? Sort.unsorted() : Sort.by(orders));
	return(pr);
}

//--------------------------------------------------------------------------
public static <T> PageRequest makePageRequest(Query<T, Void> query) {
	PageRequest pr = makePageRequest(query.getSortOrders(), query.getOffset());
	return(pr);
}

//--------------------------------------------------------------------------
public static <T, F> PageRequest makePageRequestFiltering(Query<T, F> query) {
	PageRequest pr = makePageRequest(query.getSortOrders(), query.getOffset());
	return(pr);
}

And I use it by, for example:

mMyGrid.fromCallbacks(
		query -> mKeyRepository.findByKey(key1, !caseSensitive, GridProvider.makePageRequest(query)).stream(),
		query -> mKeyRepository.countByKey(key1, !caseSensitive));

This method relies on the (observed) behaviour that if you set the page size to (say) 50, then Vaadin will always ask for records in multiples (sometimes 1) of 50, on 50 boundaries. Vaadin might ask for 150 records from offset 350. The above code translates that into a Spring Data JPA request for page 7, and then loops so it asks for a further 2 pages (to give the 150 requested). The code will trim off records if more are returned than required but I’m not sure now if this code will ever be needed.

If the observed behaviour changed then that would be a problem.

It would be simpler if the Vaadin code could be set to say ‘strict’ mode where it only ever asks for a single page of the given size. Then, none of the above code would be required. Unless of course I am missing something and making everything more complex than it needs to be.

Mark.