News: Java-based persistence and the Google App Engine datastore
App Engine for Java seeks to take the worry out of writing a persistence layer for scalable Web applications, but how well does it achieve that aim? In this article, Richard Hightower concludes his introduction to App Engine for Java with an overview of its persistence framework, which is based on Java Data Objects (JDO) and Java Persistence API (JPA). While initially promising, App Engine's Java-based persistence currently has some serious drawbacks, which he explains and demonstrates. Learn the nuts and bolts of why Java persistence in the current preview release isn't quite ready for prime time, while also getting a working demonstration of what you can do to persist data in the Google App Engine for Java applications.
- Posted by: Frank Charles
- Posted on: August 26 2009 06:53 EDT
- Too green by E. Conde on August 26 2009 13:41 EDT
- Re: Java-based persistence and the Google App Engine datastore by Jürgen Lind on August 26 2009 14:59 EDT
- Re: Java-based persistence and the Google App Engine datastore by James Perry on August 27 2009 07:11 EDT
- Re: Java-based persistence and the Google App Engine datastore by Guillaume Bort on August 27 2009 08:17 EDT
- Re: Java-based persistence and the Google App Engine datastore by Erik Bengtson on August 27 2009 09:15 EDT
It is to be expected that this framework would be rather immature, it is still disappointing to see how immature it is. The fact that using this framework fully ties you application to it for good, and that the documentation is very poor just makes it usable for serious development (but probably fun to experiment with). Yet, the scalability associated to this type of cloud architecture makes it worth to keep an eye on it. Great article by the way. E. Conde Lead Developer jBilling Open Source Billing
I think the above summary is a bit misleading as one might expect to read something about the problems of using Java Persistence in GAE. Unfortunately, the author of the article does not go into any details of this subject. He rather rants about the "leaky abstractions" without giving any deeper details on what this leaks might be. His major point seems to be that one might not be able to just switch an existing application to GAE - which might not be the most common case . The largest part of the article shows how to implement a simple DAO in JDO on top of BigTable
One of the cons of using GAE/J that the author didn't talk about, is that you never know whats happening with your app. We were tempted by GAE/J and switched to it, to do it we migrated from beautiful GORM to GAE's implementation of JDO. Not a big deal, but Grails loose many points without GORM :( Well... it was all ok, until our app started to fail, randomly. Grails could not be the problem, the app -locally- was doing it really fine. But in GAE, our app was trying to create more than one JDO Persistence Manager Factory (and thats a really big problem on GAE). So we analyzed grails base code, we searched the mail lists (gae's ones, jdo's ones, grails ones, etc.) and many users reported to have the same kind of problem (not only in JDO, byt in JPA too), but allways with Grails, over and over with grails... The only -remarkable- difference between the Grails GAE plugin and the traditional way of creating Persistence Manager Factories is that grails use a Spring bean for represent the Singlenton Holding the PMF, and GAE/J examples defines te PMF singleton using static class constant. We changed the grails plugin, making it use a singleton class and... it stops breaking our deploy. So, after a couple of days of researching and interpreting the problem we did concluded that GAE is re-starting the app or starting more than 1 instance in the same JVM, then the singleton class works, but not the singleton spring bean. For me this problem *smells*, my coworker says its only a little problem and that is easy to avoid, so he is in peace :P. And as our app is doing it very well at development, we are still at GAE. I hope its the only point where the google boys make something un-expected (and un-adviced) in his cloud, but i doubt it :(
I do not believe it's a big drawback as if one develops with GAE/J then you have to take off your relational normalization hat off as BigTable isn't just about structured data - like what you see in a RDBMS. It also enables to persist unstructured data or a combination of both. I created http://www.talktosven.com using GAE/J with GWT which persists conversation every time the user submits a reply and, so far, I haven't noticed any performance issues. :-D
Seriously you should take a look at the siena library (http://www.sienaproject.com/). It is a simple port of the python GAE API to Java. I think that it is a better match for the Google Datastore than JPA. I see two reasons you would like use JPA on Google App Engine : - To be able to transparently move your application in/out of GAE - Because you already know the JPA API very well, and you want reuse this knowledge. But these 2 reasons fail. The JPA support is so different in GAE that you just can't use it in on transparent way. It will just work for a single simple entity without relations. But even with a simple relation, you will have to rely on special google classes. And because the JPA support is so different you can't generally do things the same way you are used to do them. When I have added GAE support for the play framework (http://www.playframework.org), I thought that adding JPA support would be a good choice. But after my first application I definitively switched to siena as preferred persistence engine for GAE.
I'm fiddling with a play-test project on GAE and tried to integrate Siena, after reading your recommendation. At first look, Siena looks like a nice and useful DB wrapping layer. The Siena site has downloads of some jars from earlier this year, but urges users to build their own more up-to-date release using Maven. I spent an evening trying to get version numbers to match, and eventually gave up. I'm sure my problems were caused at least as much by Maven (and my ineptitude with it) as by Siena, but I think Siena could achieve better acceptance if it was distributed in a more approachable way.
Hi, I'm the lead developer at siena. Play! comes with a siena module that comes with a bundled siena jar file. So you don't need to create a jar file if you are using play framework with its siena module. Siena was first a very simple ORM inspired on the Google App Engine Python Datastore API. When Google added support for Java in GAE I added support for it on siena. So you can use siena in a relational database or in GAE. I am also working on support for Amazon's SimpleDB and HBase. Other implementations will come in a future. Hope you like it!