Google App Engine Container available in the repository

Hi everyone,

I have now uploaded a version of the GAEContainer that implements:

-Most of the functionality in the Container.Indexed and Container.Sortable interfaces.
-A custom interface for doing queries with various filters supported by the Datastore
-Optimistic locking and manual commits for items
-Multivalued properties
-A cache hierarchy with a local in memory cache and memcache that can cache items, indexes and sizes. They are highly customizable with remove strategy, life time and more.

Instructions on usage can be found in the documentation and trough the javadoc.
Everything under:
http://dev.vaadin.com/browser/incubator/GAEContainer/GAEContainer

Special thanks to everyone that tries this out and gives feedback, questions, suggestions or errors reports.

Disclaimer: This has so far only been tested by myself so don’t expect any production use yet. I’m sure there is many bugs to be found.

Johan

EDIT:
The GAEContainer is now also available as a .jar under http://vaadin.com/directory#addon/gaecontainer.

Hey Johan,

The gaecontainer looks very useful.

I’m having a little trouble getting it to work with an existing entity. I keep getting nullpointer exceptions when trying to obtain items from the container.

I’ve debugged the code and I think I found the problem: The nullpointer is on line 252 of the GAEContainer. When constructing a GAEproperty entity.getProperty(“version”) return null and is then unboxed to a long to call the GAEproperty constructor. Unboxing the null to long results in a NullPointerException

Am I correct in assuming that entities need to always have a version property? I’ve tried turning of versioning, but that doesn’t seem to do much (as far as loading the entities is concerned)

Currently I only want to display entities and have no intention of updating them. Is there a way around this? Or should I just go ahead and add a version property that’s set to 0?

BTW I’m using version 0.5.0. If you’d like I can cook up some code samples.

Kind regards,
Peter

Hi Peter,

First of, sorry for the delay in my answer and thank you for trying out the container.

Yes, it sounds very likely that you would get a exception if you try to use the container with entities that does not have a version field as the container expects the field. All items created with the container should have a version field. If you are manually inserting entities you should as you say create a ‘version’ property. I will have a look if the container could be modified so that it would still work without the version field. But for now, include it.

EDIT
This should be fixed in version 0.6.1. Entities in the datastore no longer needs to be equipped with a “version” property. The container does neither add a “version” property if optimistic locking is disabled.

Johan - I just independently released a fairly related GAE-based add-on (see http://vaadin.com/forum/-/message_boards/message/31262) called GAEDatastoreContainers. I was unaware of your project. Henri Sara suggested I contact you and discuss possibly merging the projects. That would mean taking the code in my project and merging it into yours. Yours is the more advanced and has the nicer name. Please take a look and see if this is something you would like to do.

Regards,

Jonathan

Hi Jonathan,

I had a look at your project. I haven’t played around with JDO in App Engine myself. What are your thoughts about the performance?

As for the merging part. The GAEContainer does not use the low level API for all interaction with the Datastore. The cache is also designed in a quite specific and low level way. I cant at least at first glance see how to combine the projects. Do you have something in mind?

Johan

Can’t comment on JDO performance. I’m not at a point where I’d notice any difference. An important concern, though, is not being completely dependent on the Google datastore and having other databases as an option. Sticking to a standard like JDO or JPA helps with that.

I took a look at your project and didn’t see an obvious way either to integrate them. Perhaps Henri could offer some comments here?

Just posted a significant update to my add-on which allows the data in the GAETable to be edited and persisted back to the datastore. In the first version, the GAETable just displayed all instances of a persisted class by essentially just specifying the class. This current version enables editing.

The add-on now has the following external library dependencies:

mcvaadin (http://code.google.com/p/mcvaadin/)
commons-lang 2.5 (http://commons.apache.org/lang/)
commons-beanutils 1.8.3 (http://commons.apache.org/beanutils/)

There’s one known bug I’m still trying to track down. When you make a change to a table row and then double click on another table row (indicating you want to edit that other table row), the pop-up window asking if you’re sure you want to do that, triggers a NotSerializableException.

See
this message
concerning the NotSerializableException.

It would be much easier for others to use this if mcvaadin were in the Directory - I hope Sami will publish it there someday.

Just started using Vaadin and GAEContainer and first of all thanks Johan for this with well commented code :slight_smile:

But as I see in sources, it keeps entity count for each kind in in “_size” store. But it relies on that once it is created in datastore and makes trouble when real entity count is different from this number.
I started my app as JDO, so in other parts i just create and persist entities this way and its not really working together.

Hi,

Thanks for having a look at the container.

The for storing entity count as metadata (_size) is that it is very time consuming to get the count for a large number of entities. If the _size does not exist the container counts the entities in the datastore and creates it. The _size entities are transactionally updated on adds and removes by the container. However if you manually add entities after the _size entity has been created this approach will not work.

I haven’t put so much thought into using the container with JDO so there could be other problems as well. As for the _size thing, I can make it customizable so that counting will not use metadata if that helps.

Well customizable or some refresh on demand feature. Other thing is when i remove some items from it, it leaves the row in table, but just makes it smaller.
I’m gonna checkout your sources and do some tweaks myself. I’m really new to all that and that “everithing has to be serializable” thing in vaadin & gae makes it even more confusing to start with it

Other thing is that I cant use it for standard fileds, as you do not support getItemIds(). i know its because of high volumes of data in large datasets, but it would be wery useful for short list (Cities, Lists of other entities for creating relations in forms). It woul mean to load the whole list to memory, but tha would not be that bas, as lot of forms would use this for list tha would have likely tens of items
It would be fine to have that defauld behavior as it is, but availability to configure it for some other use.
I must be pain in the ass, but I prefer perfecting already existing tool, than creating something else.

Posted a significant update to the add-on, which has evolved into something a bit more substantial. I am shooting for a fairly full-featured framework for bridging the gap between Vaadin and Google App Engine (GAE) applications which use the GAE datastore. The standard relied upon is JDO.

The aim is to enable Vaadin-based GAE applications where the main work in building the application is defining the datastore Pojos (classes extending AbstractGAEPojo from the add-on) used by the application.

There is an included demo which allows adding, deleting, and editing a GAEPojo called SimpleBeanableClass. The demo can be viewed online at: http://gaedatastorecontainers.appspot.com/.

The external libraries now used are commons-lang 2.5 (http://commons.apache.org/lang/) and PasswordField-0.2.jar.

Jonathan

Looking forward to test your add-on in motion, as it does exactly what i look for (vaadin - gae datastore JDO).

Hi Jonathan,

Thats really great! I won’t bring the GAEContainer closer to JDO if your container supports it. It seems that your container is much more mature now.

JDO is a tough standard to work with. At a certain point I came across Objectify (https://code.google.com/p/objectify-appengine/) which seems really good. The reasons I didn’t end up using it were 1) wanted apps to have the ability to potentially migrate away from GAE, 2) it’s a bit early to see if the Objectify project has staying power, and 3) was too far down the road with JDO already.

However, the introduction to Objectify has this funny line:

“After you’ve banged your head against JDO and screamed “Why, Google, why??” enough times, read the Concepts and then the IntroductionToObjectify.”

I can attest that my head now has lots of bruising.

Thats the exact reason I want to go with JDO. I try not to lock the project on the platform itself. And Johans add-on blew my mind how easy was to get the values to show up in the GUI. With JPA Container it was too much messing around for fast prototyping. Seems like i just found way around my many-to-many relations problem, so looking forward testing it.

Any feedback would be appreciated.

Jonathan

For Container, i makes me do a lot of changes in my model so far. On the other hand it looks good. The password filed dependency is unnecessary, especialy if its tied to exact field name.
but as it makes me do lot of changes to my code, its exactly other approach to what I prefer. I makes my model coupled with one special container and would be lot of unnecessary work when migrating just to another component.

Hi,
Is it possible with gaedatastorecontainers to have a form where one of the values to be recorded would have list of values from where the user should choose the one to record?