displaying huge content through Vaadin tables

Hi,

I have a requirement where I need to display data in a table which is fetched from a file. Now this file can be huge in size, exceeding 2GB as well. The problem is trying to display such a huge content in a browser throws “out of memory” exceptions. If I had to display the same content reading from a database, I could have used pagination and handled the scenario. Can anyone please guide me how this can be implemented when reading from a file in a Vaadin table?

Thanks!

The table is lazy on the client-side (browser), meaning, it only loads to the browser whatever you need in order to render the screen. When you scroll down, it automatically fetches more data. You problem is on the server-side. If you read the whole file into memory and put it in a container, then I don’t doubt at all that you are out of memory.

What you need to do, is make the server lazy as well. First of all, you are going to need a container that is lazy (doens’t require everything to be read into memory up-front). For this you can use, for example, the
LazyQueryContainer
. It’s a third party add-on, but it allows you to define yourself how you read the data from the actual data source. What you’ll then need to do, is implement the approriate adapters for LQC so that you can read the content of the file lazily. You’ll probably need to use something like java.io.RandomAccessFile to implement the adapter.

Thanks for the input Kim.

I will try your suggestion and report back.

I I suggest to look at MTable or LazyList with ListContainer that it uses behind the scenes in
Viritin
. It has bit similar lazy loading helpers as LazyQueryContainer, but it might be bit easier to use as it pretty much hides all the Container-Item-Property stuff (just implement two interfaces, with e.g. Java 8 lambdas). Also the memory usage is highly optimised in it.

BTW. I also suggest Kim to try it someday :wink:

cheers,
matti