Ebean ORM 0.9.7 released - ORM with automatic query tuning

Discussions

News: Ebean ORM 0.9.7 released - ORM with automatic query tuning

  1. Ebean ORM 0.9.7 has been released. This is an ORM for Java that looks similar to JPA with some differences. The main feature for this release is built-in support for "AutoFetch" - which is a mechanism where the ORM queries are automatically tuned for optimal performance based on profiling information. With AutoFetch, the queries are tuned by specifying joins and properties to include in the query and thereby reducing lazy loading. The queries also only include the properties that are actually used. The automatic tuning is based on profiling information that is gathered by previous executions of the same query. There's a lot of interesting stuff here. It uses many of the JPA mechanisms while not actually using JPA itself, and can generate a map of columns itself, avoiding the use of code for the entity beans; the API does seem to be very light. Cool stuff.

    Threaded Messages (14)

  2. Even if JDO is cited, the author('s) doesn't seem to be acquainted with. JDO doesn't have all the downside listed. In fact, there is: 1. non need for a container 2. no need for a 'version' column for optimistic verification And doesn't have all the limitations listed. In fact it: 1. works with J2SE 1.4 2. has a Java-like query language 3. does not use reflection 4. has multiple inheritance mapping strategies 5. works with different datastore technologies So, the question is: why ? NIH syndrome ? NWU syndrome ? (NWU: Not Well Understood) Guido
  3. The session management issue...[ Go to top ]

    Thanks for the comments Guido. What you say is true of JDO and JPA. However, what I have possibly not made clear is that both JDO and JPA have a "session" object. For JDO this is the PersistenceManager and for JPA this is the EntityManager - Ebean has no equivalent object. So the main differences I see are... - Ebean has no "session" object (no PersistenceManager or EntityManager) - Hence Ebean has no concept of attached/detached beans or merge() flush() type operations. These are replaced by save() and delete(). I believe this makes life much simpler which is the goal. Yes, you can use both JPA/JDO outside a container (I never said you can't) but then you have to manage the session objects yourself with the associated work and issues that has. With Ebean there are no sessions to manage. Do you think I have got this wrong (PersistenceManager is not a session object)? Perhaps you could clarify what I have misunderstood? There are a number of other things Ebean has... such as support for Generics, partial objects and "AutoFetch" automatic query tuning, Background fetching and per row PersistenceContext for large queries. It would be nice to see other ORMs support this if and when they can. Cheers, Rob.
  4. Also to be clear... AutoFetch was invented by Ali Ibrahim and William Cook at the University of Texas. You can visit Ali's AutoFetch home page here. Ali built an implementation for Hibernate, which is on sourceforge - and I built AutoFetch support into Ebean 0.9.7. Even if you don't like the idea of Ebean (or sessionless ORM) you should have a good look at AutoFetch - the idea has a lot of very interesting implications in terms of how ORM queries are built and can be executed for optimal performance. Something interesting for every ORM vendor I would think.
  5. For JDO this is the PersistenceManager and for JPA this is the EntityManager - Ebean has no equivalent object.
    Well it does actually. You have a class with static methods Ebean. This is the equivalent of a singleton PM or EM ... so JDO and JPA have the same with a single line of code - so hardly an "advantage". Ebean cant cope with multiple datastores within an app then ? JDO and JPA can, and its a common enough requirement. I don't get the "associated work and issues" managing a PM or EM. How hard is it ? Having PMF and PM, and EMF and EM provides flexibility. Ebean doesn't have that. PS. JDO has no "merge" either, that is a JPA concept. JDO and JPA support generics, but then you don't say what that means. JDO has advanced configurability over fetching. Implementations allow more control than that. Implementations allow large result set support.
  6. Singleton PM or EM...[ Go to top ]

    This is the equivalent of a singleton PM or EM ...
    Interesting point because you can *NOT* have a singleton JPA EntityManager (Just like you can't have a Singleton Hibernate Session - its effectively a per user session object). Q:Are you sure you are allowed a singleton PM? If so then that is very interesting and a fundamental difference between PM and EM (that I was not aware of). Is it not the case that a given object with a given Id for a give data store must be unique for a given PersistenceManager... that doesn't sound singleton friendly to me? Would pm.flush() flush all new/dirty/deleted objects for all users globally? Now, if you can have a singleton PM. This then does become effectively the same as Ebean - correct, and JDO and Ebean are much closer in design than I thought - which is great!! Ebean can cope with multiple DataSources (see Ebean.getServer()). The static methods on Ebean use the 'default/primary' DataSource so are a convenience feature for those with only one DataSource. So you can say there is one EbeanServer per DataSource (One singleton PM per DataSource if you like). The work and issue related to managing an EM would not apply to PM if it can be a singleton (good news for JDO). An EM is a per user session object and will fill up with 'persistence' beans until it is closed or cleared. If you use JPA Extended Persistence Context then you need per user session management of the EM, and at some point you will clear the EM or perhaps run out of memory. Often a session bean or spring (or Seam) will manage the EM lifecyle for you (but it will still fill up with beans over time). If you choose to create and close an EM per request/transaction then you have to use a version column otherwise you lose optimistic concurrency checking (not too bad for most). By generics I mean List, Set and Map... By large results I means that for a JPA query all results are loaded into the PersistenceContext so you run out of memory :( - Ebean and Ibatis have a query listener/Callback feature. I didn't see that in JDO but yeah, I'll look again.
  7. I am a little scared for the lack of a session object. In fact, how would you manage multi-threading access (well, JDO has a multithread option to set) ? I don't know JPA and I learn all the lifecycle issues from your post. In JDO, once you get a PersistenceManager you can do whatever you want with it. No limits. More, a session object, besides being the obvious mechanism to get access to the framework API, provides a cache of the loaded object. Guido
  8. The sessionless approach[ Go to top ]

    I'm getting the impression you think PM is a session object? The question for PM is not whether it is multi-thread safe (I'd expect it to be) but whether it is safe for use by multiple users. It seems to me that 2 users sharing the same PM trying to use the same object will clobber each other (so you can't use it as a singleton). Can we confirm if a single PM instance is safe for multiple users to use (ala as a singleton) as has been suggested? One way of looking at the Ebean "sessionless" approach is that it has split the functions of the JPA Persistence Context (JDO StateManager?) and moved them to other places. The "session less" technique that Ebean uses is based on the 15+ years ago techniques of client server tools such as Oracle Forms and Powerbuilder etc. People from that era may recognise that Ebean uses the same optimistic concurrency checking techniques of those tools with some persistence by reachability, relationship management and "transaction scoped persistence context" thrown in. Yes, I accept that for people who haven't thought/seen a sessionless approach before, it might sound scary. I'm probably doing a poor job of explaining it ... but discussions like this help so thanks for that. Cheers, Rob.
  9. Re: The sessionless approach[ Go to top ]

    I'm getting the impression you think PM is a session object?

    The question for PM is not whether it is multi-thread safe (I'd expect it to be) but whether it is safe for use by multiple users. It seems to me that 2 users sharing the same PM trying to use the same object will clobber each other (so you can't use it as a singleton).
    Here I don't understand what a user is. How he comes into play. A PM can be set to be thread-safe in its internals. However, a persistent object thread-safety is up to you. And isn't trivial. So, if different threads wants to change an order resulting in the following scenario: 1. Thread 1 opens a transaction 2. Thread 2 opens a transaction 3. Thread 1 read order X into O1 instance If Thread 2 read order X, in which Java instance is the DB row resumed ? Into O1 ? Into a new O2 ? If it is resumed into the exactly the same instance used by thread 1 you have to handle multi-threading access to O1 instance, while if you have a per-transaction view of the object you are simply moving the L1 cache of loaded objects managed by the "session" into the transaction context. With a drawback: when the transaction completes the cache should be cleared, while in a session the objects are still there until it is closed.


    Can we confirm if a single PM instance is safe for multiple users to use (ala as a singleton) as has been suggested?

    One way of looking at the Ebean "sessionless" approach is that it has split the functions of the JPA Persistence Context (JDO StateManager?) and moved them to other places.

    The "session less" technique that Ebean uses is based on the 15+ years ago techniques of client server tools such as Oracle Forms and Powerbuilder etc. People from that era may recognise that Ebean uses the same optimistic concurrency checking techniques of those tools with some persistence by reachability, relationship management and "transaction scoped persistence context" thrown in.
    Not very familiar with those prods, but AFAICR Oracle Forms is the client (single-user) opening a "connection" with the server. How much does this "connection" differ from a session ?


    Yes, I accept that for people who haven't thought/seen a sessionless approach before, it might sound scary. I'm probably doing a poor job of explaining it ... but discussions like this help so thanks for that.

    Cheers, Rob.
    Guido
  10. multi-threaded[ Go to top ]

    what a user is. How he comes into play.
    Just a normal User. The point being that we need to clear up the claim that you can have a singleton PM. You still haven't stated where you are on that, So I'll ask again. Q: Do you believe PM can be a singleton or not? There are two quite different multi-threaded issues as I see it. Simple/common thread per user case. I'm reasonably confident you are not talking about this case (too easy)? Sounds like you want to talk about the case where a "user" needs to spawn multiple threads with their own transactions for some purpose. In this case you have the options of using either one or mulitple PM's? This is indeed a very interesting case and definately non-trivial. Leads to the question of the read consistency required by the app (aka what is the purpose that the multiple threads are working to) which would control the transaction demarcation and isolation levels. Can you explain the requirements of the real use-case in question? The independent threads seems too simple so I suspect your multiple threads are working to a common purpose? What is that purpose/requirements? Requires read consistency etc? In regards the last point... objects still in cache. The PM/EM holds the objects in cache for a purpose. So you just have to ask what the purpose is, and how does Ebean support the same purpose?
  11. Re: multi-threaded[ Go to top ]

    > what a user is. How he comes into play.
    Just a normal User. The point being that we need to clear up the claim that you can have a singleton PM.

    You still haven't stated where you are on that, So I'll ask again.

    Q: Do you believe PM can be a singleton or not?
    Absolutely yes. But you have to face with the multi-threading issues I pointed out.


    There are two quite different multi-threaded issues as I see it. Simple/common thread per user case. I'm reasonably confident you are not talking about this case (too easy)?

    Sounds like you want to talk about the case where a "user" needs to spawn multiple threads with their own transactions for some purpose.
    Again, what a "user" is in a Java application ? There is no "user" concept in JDO/JPA/Hibernate/JDBC etc. Is there such a concept in Ebean ?
    In this case you have the options of using either one or mulitple PM's? This is indeed a very interesting case and definately non-trivial. Leads to the question of the read consistency required by the app (aka what is the purpose that the multiple threads are working to) which would control the transaction demarcation and isolation levels.

    Can you explain the requirements of the real use-case in question? The independent threads seems too simple so I suspect your multiple threads are working to a common purpose? What is that purpose/requirements? Requires read consistency etc?

    Please clarify what happens in Ebean in my scenario with 2 "user" that are concurrently modifying the same order in a web application.
    In regards the last point... objects still in cache. The PM/EM holds the objects in cache for a purpose. So you just have to ask what the purpose is, and how does Ebean support the same purpose?
    The purpose is to ensure the uniqueness of the in-memory image of a persistent state in a given context (the session). Guido
  12. Re: multi-threaded[ Go to top ]

    ... 2 "user" that are concurrently modifying
    By default in Ebean you get 2 different object graphs in the 2 different transactions. But there are 2 other scenarios where this can be different. A big point here though is that you mention the magic word which is "modify" impling you don't want immutable objects. You also mention these are 2 different concurrent users (as opposed to a single user parallel fetch type scenario). more on this but first...
    what is a user
    JPA does use the concept of user (sometimes). That is, with extended persistence context often the EM is stored in a HttpSession or EJB Session Bean. So, if we talk about a web application and you and I are users, and we use JPA extended persistence context... then your EM is stored in your HttpSession and my EM is stored in my HttpSession. So generally, in this long lived EM scenario, each user has their own EM, stored in a HttpSession or Session Bean, and the servlet requests use the EM's for their user.
    ... 2 "user" that are concurrently modifying
    ... so when you have two different EMs you are generally going to get separate object graphs and O1 and O2 in your scenario are different instances. ... however, they COULD be the same instance if you choose to use query hints to get a read-only + from-cache type instance. In which case they could be the same instance. Ebean gives you this option and most JPA providers I'd expect do to. ... however, there is also another scenario... that is where (back in our webapp here) you can a servlet request and that is serviced by a single thread. At this point you could decide to do something like populate the 'persistence context' in a parallel fashion - sharing a persistence context across multiple threads which are each populating generally a different part of the object graph. Now Ebean has not exposed the API to enable a user to do this. I haven't yet had anyone requesting this but it is something that can happen behind the scenes. Apologies because I thought you might be headed towards this scenario which is a fair bit more complicated.
    The purpose is to ensure the uniqueness of the in-memory image of a persistent state in a given context (the session).
    Yup, thats one way of looking at it... I'm thinking more along the lines that there are 2 main purposes/functions/benefits of the JPA Persistence Context. - One is to provide 'consistent object graph construction'. - The second is to support optimistic concurrency checking. Another way of saying this is that in JPA if you change the scope of the Persistence Context you can modify the 'consistency of the object graphs constructed' and you can modify the optimistic concurrency checking - hence the decision making behind extended persistence context is based on these two issues (the way I see it anyway).
  13. Re: multi-threaded[ Go to top ]

    There is no "user" concept in JDO/JPA/Hibernate/JDBC etc.
    So to hopefully be clear... - The concept of a "user" is important in JPA/Hibernate when you have a long lived EM/Hibernate Session (ala extended persistence context). - There is no "user" concept required for JDBC, agreed. - There appears to be no "user" concept with Ebean because it currently does not exposed its Persistence Context. Unlike JPA it does need to expose its Persistenc context for optimistic concurrency checking on beans without a version property. However, one day Ebean may expose its persistence context... if that was the case and it was long lived (stored in HttpSession etc) then hopefully developers will understand the 'read consistency' issues related to that.
  14. Re: multi-threaded[ Go to top ]

    JPA does use the concept of user (sometimes). That is, with extended persistence context often the EM is stored in a HttpSession or EJB Session Bean.
    Actually JPA does not use any such concept; please read the JPA spec if you don't believe me. A PersistenceContext is a context. Whether your application decides to associate that to a user is your decision. It is not a feature of the specification. A web application may have requests and responses. These may or may not be associated to a user, or a PM/EM per request. The specification of the persistence technology provides the structure to cope with such situations, hence why people prefer specified persistence solutions. If you have something that you feel you can add to the current specifications then why not actually feed it back to the specification groups ?
  15. Re: multi-threaded[ Go to top ]

    I must admit to being very familiar with the EJB3 Persistence spec already but thanks for the hint. The EJB3.0 spec states ... A container-managed extended persistence context can only be initiated within the scope of a stateful session bean. The user is implied as soon as you have a stateful session bean. You may disagree, lets just move on.
    These may or may not be associated to a user, or a PM/EM per request
    I agree... depends on your need/use of extended persistence context but I have already said that.
    why not actually feed it back to the specification groups ?
    I have actually, but that doesn't mean they need to take heed of any of my suggestions or ideas or even acknowledge them. There are many reasons why they would not wish to do so and there is plenty of "grey" areas in ORM in general. We also have to remember that the JPA spec will not be able to change too radically now. It will have to support backward compatibility (even back to EJB2 in terms of the query language). It's focus will probably always be towards running inside a container (everyone in the JPA group has a container they want people to use). From what I've heard JPA 2.0 seems a disappointment to me but we will have to wait and see. Its not the end of the world that I have some issues with the JPA Spec. In the end I'm only one person with an opinion... last time I checked I was free to express that opinion, some will agree and some not... that's life. Cheers, Rob.