Data binding to Grid gets easier and more efficient

About a year ago, I discussed two common problems when doing lazy data binding to the Grid: the need for a count query and difficulties integrating with “paging backends”. That generated a lot of public and private discussion. The solutions are now available for testing. And while we were on the topic of data binding, we’ve added a few other enhancements as well. 

The changes are not small, but we tried to keep their impact to a minimum for most developers. Let me introduce what is now available in Vaadin 17 pre-releases and discuss the consequences a bit. There are three great enhancements, but we did lose some hair trying to keep everything backward compatible—I'll explain why.

1. Count query is now optional

The primary goal of the project was implemented pretty much as it was in the prototypes. If you don’t provide the size in your lazy data binding, the Grid polls your backend for more when the user scrolls to the end of the table. The old style provides the best UX, but the new style is easier for you and lessens the strain on your backend.

With the new API, you can also provide an “estimated size” and configure how many rows are added to the estimation when the end is not found where expected. With these options, you can drill down into the UX issues, if you can’t or don’t want to provide the exact count of your items. 

var dataView = grid.setItems(q -> service.fetchPersons(q.getOffset(), q.getLimit()));
        // Optional, you may give a rough estimate of the item count,
        // good compromise for db performance and UX
        dataView.setItemCountEstimate(10000);

2. Bind easily to Spring Data repositories

Doing a lazy binding to Spring Data-based repositories, or to any other backend that uses “paged access” is now a lot easier. The setPageSize method is now guaranteed to also affect the queries made to your data binding callbacks. We also provide helpers to get the page index, instead of forcing you to calculate it yourself.

// the old way to hook Grid to Spring Data Repository
grid.setDataProvider(DataProvider.fromCallbacks(
    q -> {
      final int pageSize = q.getLimit();
      final int startPage = (int) Math.floor((double) q.getOffset() / pageSize);
      final int endPage = (int) Math.floor((double) (q.getOffset() + q.getLimit() - 1) / pageSize);
      Sort sort = Sort.by(q.getSortOrders().stream()
          .map(
              so -> so.getDirection() == SortDirection.ASCENDING
              ? Sort.Order.asc(so.getSorted())
              : Sort.Order.desc(so.getSorted()))
          .collect(Collectors.toList()));
      if (startPage != endPage) {
        List<Person> page0 = repo.findAllBy(PageRequest.of(startPage, pageSize, sort));
        page0 = page0.subList(q.getOffset() % pageSize, page0.size());
        List<Person> page1 = repo.findAllBy(PageRequest.of(endPage, pageSize, sort));
        page1 = page1.subList(0, pageSize - page0.size());
        List<Person> result = new ArrayList<>(page0);
        result.addAll(page1);
        return result.stream();
      } else {
        return repo.findAllBy(
            PageRequest.of(endPage, pageSize, sort))
            .stream();
      }
    }, q ->  service.countPersons()
));

// the new way
grid.setItems(q -> repo.findAll(PageRequest.of(q.getPage(), q.getPageSize(),
      VaadinSpringDataHelpers.toSpringDataSort(q))).stream());

3. DataView - Easily mutate, read and observe your data

DataView is a new concept that we introduced along with the other changes. The type depends on the component and the nature of the data binding. The first example above, shows you how to fine-tune the UX via the GridLazyDataView instance when doing a lazy data binding. The new data view concept also allowed us to expose some often-requested APIs for common in-memory data binding. For example, it is now easier to mutate an existing in-memory data set or to find the index of a specified item in the Grid.

The code snippet below shows examples of how you can use the new API. It includes the following use cases:

• Adding and removing options dynamically.
• Changing the selection programmatically to the next item.
• Notifying the user of item additions and removals.
• Passing a stream of currently-displayed items to a method that exports the data.

Select<String> select = new Select<>();
ArrayList<String> options = new ArrayList<>(Arrays.asList("foo", "bar"));

select.setItems(options);

// The data view is returned by the setItems method, but
// can also be retrieved using getListDataView and 
// getGenericDataView methods. Those methods can be 
// handy in for example various add-ons
SelectListDataView<String> dataView = select.getListDataView();

TextField newItemField = new TextField("Add new item");
Button addNewItem = new Button("Add", e-> {
    // this adds new item to the backing list and
    // automatically refreshes the component accordingly
    dataView.addItem(newItemField.getValue());
});

Button remove = new Button("Remove selected", e-> {
    // remove item without resetting the options
    dataView.removeItem(select.getValue());
});

Button selectNextItem = new Button("Select next", e-> {
    // Get current value, detect and select next
    String currentValue = select.getValue();
    dataView.getNextItem(currentValue)
        .ifPresent(next -> select.setValue(next));
});

// Subscribe to an event to be notified when the amount
// of items in the component changes
dataView.addItemCountChangeListener(e -> {
    Notification.show(" " + e.getItemCount() + " items available");
});

 Stream<String> currentlyShownItems = dataView.getItems();
 exportItems(currentlyShownItems);

The bad parts: backward (in)compatibility

The backward-incompatible part relates to the addition of a return value to the setItems methods. It is just a matter of recompilation for many apps, but we expect you to have some backward-compatibility issues with these changes, if you:

• Directly use the HasItems, HasItemsAndComponents HasDataProvider interfaces.
• Override some of the setItems methods.
• Either of the above exist in an add-on or library that you use.

We always aim for the best backward compatibility, and currently think this is the best way to make these changes. A lot of the old API was bolted to interfaces that were implemented by a number of different components—that you probably extend. Also, all the best method names were already taken, and we wanted to change their signature.

We had two bad options: come up with a completely new API next to the old one, or slightly break the backward compatibility. In other words: either break backward compatibility in your minds or for the compiler (add-ons and potentially in some application code). We decided to try the "human-backward compatibility" approach first, and return to the subject if we get a lot of issues during the pre-release or non-LTS major release phases.

If all goes well, you could see the changes in the V14 LTS series too, but it is also possible that these features will never make the V14 branch. We’ll decide this based on how many Vaadin users are affected by the change. Thus, we are more than eager to hear your feedback about the breaking changes, both good and bad.

Some of the old API is now deprecated in favor of the new, but there is no rush to upgrade. 

Try it and let us know what you think

We’d love to hear your feedback about the changes to help us iron out the rough edges. Check out the latest pre-release of V17 (17.0.0.alpha7 when publishing this article) and try it out. For insights, use GitHub, the comment section below, or approach me directly. Note that most of the team will be enjoying the rather snowless Finnish summer days for the upcoming weeks, so please bear with us if we don’t answer as soon as we normally do. Remember to enjoy your holidays as well!