Paginate a grid without Query limit and offset

Hi. I had asked the question here - https://stackoverflow.com/questions/76268708/vaadin-grid-pagination-using-the-token-of-the-last-returned-record

I was hoping to talk to someone here.

I am using Vaadin 24. I’d like to paginate rows on a Grid. The way it works is, I call a rest endpoint and it returns a 100 records. I need to hold onto the last row’s token. When the user scrolls down I need to call the rest api again with the last row’s token and get new records. How do I implement this with Vaadin Grid?

The token is a string. I have seen the docs here https://vaadin.com/docs/latest/components/grid/#lazy-loading. It talks about using a data provider and using the offset to fetch new data. I don’t have an offset. All I have the last row’s token returned to me via my api. I also don’t know how many records could be there. I am hoping to build something simple, where I call the endpoint and it returns data. I show the first set of data on the screen. When the user scrolls down, I call the endpoint again, and add the response to the existing list and show that up on the screen.
The data set will not be very huge 500 - 1000 rows at most.

Depending on the rest service performance, I would either go and load everything or you have to go with a custom lazy loading where you have to cache the latest returned token within the lazy loading method of the data provider. It has also an estimate / guess method for the count, so that’s technically not required.