DataProvider.fromCallbacks / infinite loop / uncertain number of items

Vaadin 6 / 7 / 8 Framework Data Binding
1

So, I’ve been working on converting a reasonable large Vaadin 7 app to Vaadin 8, which means retiring Tables, various Containers, and the like, in favor of Grid and DataProvider.

One particular case seems like a great fit for lazy-loading Grid + a callback-based DataProvider. However:

  1. It isn’t feasible to count the number of items in the backend, as there are many thousands of them, and each is protected by a per-object ACL which might or might not allow the current user to view. It’s no problem to grab a page of them based on offset and count, but enumerating all just to get a count is really undesirable. Any way to provide a rough estimate, or “-1” to signal an uncountable iterable?
  2. If I provide a larger number of items than are present in the count, the Grid+DataProvider dive into an infinite loop of fetching data, again and again. This is obviously really undesirable, as the count and fetch operations are nonatomic, setting aside other concerns.
2

What you describe seems to match this feature request. At the moment

https://github.com/vaadin/framework/issues/8492

Lack of this feature has also raised other issues like this one

https://github.com/vaadin/framework/issues/10104

In most common cases the count of items is usually known. But there are natural use cases, where the count is changing rapidly (lot of concurrent insert/delete activity in database), and thus this becomes difficult.

One way to workaround this is to implement way to cache data, and validate count against cached data, and updating counts if they have been changed (potential risk of infinite loops is here too), etc. Application specific knowledge may help to find correct strategy.