Apologies if its not the right forum to post.
I have a table which gets its data from the service layer (implemented in spring) I have to lazy load the data because the result can be huge.
I would like to pass an index to the service layer which denotes the row from which the data has to be sent back to the UI layer.
I could not find anything in the docs that mentions how to do it. Can someone please explain what has to be done here ?
~tia
Hi!
Don’t know a lot about Spring, but I know something about Table in Vaadin and its lazy loading. I hope this helps.
The lazy loading must be implemented at container level. Table doesn’t paginate its content or provide any api for it - it just knows the Container interface. And to make it really work efficiently, you gotta implement the Container.Indexed sub interface.
I think the best reference implementation of lazy-loading-capable container is HbnContainer that can be found from our incubator. It is still under development, but it has been tested to work efficiently with large datasets.
In case you come up with a something that might help other Spring and Vaadin users, please contribute.
cheers,
matti
Is there a way we can track when the table’s scrollbar has reached the end of the current set of data and needs to load further data to scroll more ?
If so can we pass in the ItemId which can be the surrogate key of the corresponding entity in the database, & get next set of data and redraw the table with the new data. ?
Finally a few suggestions - I mean no offence here.
When scrolling beyond the client-side buffer (which covers a few rows outside of the visible window), the table UI widget requests a consecutive set of rows with a start index and a “page length” from the server side component. The server side Table component then makes a series of requests to the container to fetch that data, effectively requesting consecutive items in order.
If your data source does not support efficient random access, you could implement an indexed container that internally fetches data in consecutive “pages” from a back-end data source and presents it as a random access data table/list with some caching. Item identifiers can be e.g. running numbers, in which case a read-only container just needs to be able to tell the data set size and fetch items (rows) from it by index (and implement a few methods on top of that functionality). A writable container needs to provide a few more operations. This should eliminate the need for paging on the table level in most cases.
In theory, you could also use a surrogate ID to trigger fetching of additional data if you want to force the browsing order or if that is required by the back-end data source. The ID may get fetched already when approaching the end of the table due to the small client-side buffer/cache, though, and there may be other complications.
As for Spring, Vaadin itself is a back-end agnostic framework. Many of the Spring frameworks can be and are being used with Vaadin, although the documentation is still rather thin on that (
Spring Integration
).
Tutorials and other documentation might be needed more than actual code changes to support Spring, which is very modular and designed so that its higher layers can be replaced easily.
For data access, so far Hibernate is the best supported data source, but different projects have their own container implementations for various back-end frameworks. A JPA container is being planned.
For the leakage concern, I assume you think about accidental exposure of internal information to the end user of a system.
Vaadin being a server-side framework, DAOs are commonly used in the server-side part of the UI, but only the information actually shown in the components gets transmitted to the web browser of a user. This makes avoiding exposing internal data much easier than in client-side frameworks - most of the time, nothing needs to be done.
If you are overly concerned about this, or the lower layers do not trust the implementation of the UI layer, you could always create another service layer (and related containers etc.) between a Vaadin UI and the underlying layers.
hi there
i’m new to vaadin (since this morning actually !), and i’m very impressed by your work… but i’m getting the same pb as Victor ; in fact several widgets in vaadin are supposed to support lazyy-loading, but i found that they all refer to a Container instance, which contains all the data ; in fact ui widget only display a part of the data of their Container, but this can obviously not be called “lazy-loading”, because the whole data has been loaded from underlying datasource (database) at Container instantiation…
real lazy loading would be a typical LazyContainer interface, which defines some methods like “getItemIds(int firstResult, int maxResult)” instead of “getItemIds()”, and perhaps “boolean isFullyLoaded()”. that way, at instantiation time, the container instance could contain only 10 items when the database contains 100. and the data would be silently added to the container on getItemIds(…, …) calls…
is there anything like that in your enhancements catalog ?
is there any other way to avoid data to be fully loaded in the container instance at instantiation time ? (i’ve got over 10million rows to load from database…)
thanks a lot
There are two parts to the misunderstanding here:
Most of the basic Container implementations do not lazy-load from some datasource simply because they are generic and do not know about any datasource. The API supports lazy-loading, though, and Containers made for a specific datasource can lazy-load (e.g hbncontainer).
That is: the components have a ‘default’ Container for basic use (‘manually feeding’ the content to the component), but
you’re supposed to change it to a more suitable Container when appropriate.
Even if you use a Container that does not lazy-load from the datasource, the components still lazy-load between the client (browser) and the server. Note that this is every bit as important, and quite complicated to do from scratch, in a browser-based application.
I.e. if you use a lazy-loading container with Table, you’ll get lazy-loading all the way between the browser and the datasource.
Having said all that, I have to mention that the current Container API (in combination with how for instance Table loads data) does not make it particularly easy to make a custom lazy-loading Container, so something like the API you suggested might be a good idea… I think there are some tickets about improving the Container API.
Hope this clarifies the lazy-loading at least a little…?
Best Regards,
Marc
I think this an important note. The way things currently are done in Vaadin, creating your own Container is an option you really should consider if you want to use large data sets. It may seem like I daunting task at first, but if look at HbContainer and QueryContainer for pointers, it should not be hard.
For what it’s worth, I have contributed the basics/beginnings of such a container to the incubator
http://dev.vaadin.com/browser/incubator/PagingComponent
See com.vaadin.incubator.pagingcomponent.demo.HiddenPagingContainer (where the loading of pages is completely transparent to the user) and com.vaadin.incubator.pagingcomponent.demo.ExplicitPagingContainer (where the user uses next/previous/first/last buttons to explicitly load pages)
They are far from clean and finished - it is the incubator after all! - but they work, and show how such a container can be built.
Best Wishes,
Charles.
hi there
thanks for your quick replies ! i think you’re right, ‘full’ lazy-loading might be implemented through some custom Container, i’ll check NbnContainer and PagingComponent soon ; actually my server layer is implemented with spring and hibernate, so i might implement my own lazyloadingcontainer based on yours…
regards
FLE
Any updates on JPA container?
According to the roadmap, it should be included in 6.2, which is due in five weeks. Although, I’m not sure if the release date is final.
Thanks, great news.
The way we use lazy loading on all layers (client/browser-server) with jpa is by specifying pages to retrieve on the server side through jpa and keep track on the paging, by using a variable. That variable along with the vaadin container (bean container) containing the data of the page can be displayed on to the vaading client. So that the user knows exactly which page is being viewed.
This way lazy loading happens on the client between the browser and the server via the vaadin container (while scrolling) and there is complete management of data retrieval on the server side via jpa. It’s quite straight forward actually providing you with full control of the data retrieved & displayed. 
There will not be a JPA container in the core Vaadin product.
Such a container will probably be implemented and released as a separate add-on instead, but this will not happen before the 6.2 release.
There might be some minor improvements to HbnContainer before that for those using it.
Hi there,
Is there a new roadmap for the JPAContainer add-on?
Can we already test the JPAContainer?
Thanks
Hi all, the work is still in early phases, but we expect the first implementation to be testable by the end of the year and guesstimate an “early beta” implementation to be available by the end of February.
The work is done in
http://dev.vaadin.com/svn/addons/JPAContainer/trunk/
and you can follow the progress there.
Requirements for the JPAContainer 1.0 are summed up in
http://dev.vaadin.com/svn/addons/JPAContainer/trunk/docs/REQUIREMENTS.txt
.
API draft can be found in:
All comments are welcome!
All the links provided above requires a username / passwords to be viewed. Is it sensitive data or should the implementation be moved to a place which does not ask for credentials?
The files are the same repository as Vaadin with the same permissions so you should be able to checkout the sources without username and password. For some reason the browser sometimes asks for login/password but at least if you use an svn client you can checkout them without problems.
All right, yeah, now it works for me too. Shouldn’t this be investigated a little more and try to fix it the rights checking system instead of having a “sometimes it works, sometimes it doesn’t” -system 
Edit: Noticed a fun little detail. It seems to let me through without asking anything when I’m NOT logged in dev.vaadin.com, but starts asking for credentials after I log on. My trac credentials are not sufficient to let me into the svn after login.
Looking. Starting with Requirements.txt… You may want to think about having a JPAContainer that requires JPA 2.0. It’s hard to get much useful out of JPA without the new Metamodel stuff in JPA 2.0 and you’ll want the new Criteria interface for the filtering stuff.
Looking at the code some more…
Yeah, you’ve had to re-invent your own meta data set of classes. Better to move forward with JPA 2.0 I think. Either that, or you really need to divide this into two implementations, a core implementation, and a JPA 2.0.
Reviewing demo app…
You know, I wish you were a bit farther along with this, because I’m exploring using Vaadin for our next generation of server side stuff, which is all JEE6. (We’ve started calling it “come to jeesix” around the office). It’s funnier if you’re in America and can imitate that Southern Christian Evangelist thing.
Nice use of Generics. Hmmmm… Seems like your ProviderBeans would make more sense as EJBs instead of Spring stuff. Why assume JPA+Spring if you can just assume JEE6?
So… I think you need a JEE6Container that takes this code and assumes EJBs and JPA 2.0. Just my $.02.