SolarMetric Kodo JDO 2.3 Released

Discussions

News: SolarMetric Kodo JDO 2.3 Released

  1. SolarMetric Kodo JDO 2.3 Released (21 messages)

    SolarMetric has released Kodo JDO 2.3, its implementation of the Java Data Objects (JDO) 1.0 standard. The latest version improves its distributed caching, schema generation, JDOQL extensions, smart proxies for set and map fields, etc. JDO is a spec designed to allow plain java object models to be persisted transparently.

    Checkout Kodo JDO 2.3.3.

    Read the Press Release.

    Threaded Messages (21)

  2. This isn't a question specific to Kodo JDO, but I'll ask it anyway: Why do persistent classes have to implement the PersistenceCapable interface in JDO, be it via sourcecode or bytecode manipulation?

    I've gone through the spec and various JDO resources more than once but I can't figure it out. The role of PersistenceCapable is described as the interface that JDO implementations use to talk to persistent objects. JDO objects seem to keep persistence state like "new", "dirty", "transactional", etc inside of them.

    O/R toolkits like CocoBase, Castor, Hibernate, and Jakarta OJB do not need manipulation of persistent classes. I assume that they set and get fields via reflection and keep state outside of the persistent objects themselves. Optionally, they provide a callback interface for persistent objects that want to be notified of persistence events.

    So is there any real advantage of PersistenceCapable over the plain approach? Using the latter, the above mentioned toolkits provide all of JDO's functionality and more. I don't consider them miracles, so I guess their approach is valid. Why hasn't JDO adopted it?

    Juergen
  3. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Juergen --

    The PersistenceCapable interface allows for very efficient management of persistent state because it bypasses reflection and the associated primitive boxing/unboxing. Combined with enhancement, you get persistence that is not only efficient, but totally transparent well.

    In JDO, you don't need to violate encapsulation by making your persistent fields public or always providing set/get methods. Not only that, but JDO can perform lazy reads of persistent data because it 'knows' whenever you attempt to access a persistent field. Unlike reflection-based systems, this lazy reading can apply to any field, not just Collections/Maps/special proxy types.

    Finally, JDO does provide the InstanceCallbacks interface if you'd like to receive object lifecycle callbacks. This interface is *not* magically added by enhancement; you implement it explicitly if you need lifecycle event callbacks.
  4. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    It's not "totally transparent" if you have to pollute your business logic with JDO-specific interfaces.
  5. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    I wonder why people always get so polemic when it comes to JDO. Since when is defining an interface a case of "pollution"? And, if I understand this correctly, this is done by the enhancer anyway without affecting your code - unless you want to use the callback interface, which makes perfect sense.

    The next thing is that enhancement will be called "mutilation", as some do. What I cannot believe is that this comes from many of the people who at the same time swallow EJB/CMP without a word.

    peace,
    Christian
  6. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    My problem is that JDO proponents claim that JDO is transparent, when, compared to certain non-standard O/R tools, it's nothing of the sort.

    JDO has many benefits, but I wouldn't say transparency is one of them.

    And for the record, I don't much like CMP either :-)
  7. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    none of the other O/R frameworks can do miracles. If they provide automatic in-memory transactions, then they will have to intercept all accesses to object state somehow. To me enhancement is still more transparent than source-level manipulation/geneneration.

    So I'd still contend that JDO a pretty transparent way to go. Now I better duck before Ward Mullins from CocoBase drops in here...

    my2Cts
  8. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Hi Christian,

    Thanks for the intro :) So just because I disagree with you, I'm not welcome to comment, that doesn't seem very democratic or fair does it?

    Actually bytecode manipulation is completely unnecessary, and CocoBase provides complete object graph & state detection for both local and instance graph copies across j2ee. And it does it without any source or binary code manipulation...

    I disagree that bytecode manipulation is a good way to go, it's actually an unnecessary complication, support and runtime debugging issue. It also complicates doing instance reconciliation, and a variety of other tasks... I keep hearing engineering talk about no free rides, but generally speaking CocoBase performs on par, and often faster than hand coded JDBC, so this is simply not true. I'm seeing performance numbers published on this site where the open source guys run almost twice as slow as hand coded JDBC, and they're proud of this fact. If my engineers were proud of performing twice as slow as hand coded JDBC, they'd be working on an open source product instead of CocoBase too :)

    A well written O/R system should be similar in performance to hand coded jdbc applications, and in fact a generalized library offers opportunity for optimizations and management that a hand coded application won't generally benefit from... But of course we charge real money for CocoBase, because it's a real product, and a real O/R mapping layer and not just a hobby project.

    We've been fighting the ODMG/JDO bigots for years on this issue, and they keep claiming that performance is the reason to use this approach, but we keep outperforming their systems :) The JDO approach is overly complex. The API is signicantly more complex and frankly less useable in my opinion than our Shared Source Code CBFacade implementation. With CocoBase a developer can go from a standalone application, to one where the persistence is running in a J2EE app server (we support virtually every app server in the market), with startup connection parameter changes and no recompile!

    But then in my experience, facts never convince religious zealots... They only entrench them...

    An O/R layer isn't just about a particular spec, it's about a comprehensive set of tools, integrations and features that developers can use to actually build and deploy enterprise level projects... CocoBase has been shipping for 6 years, and benefits from worldwide usage and deployment. Some of our customers have had projects deployed for much of that 6 years - something few java vendors can actually claim... And we do it in a TRULY transparent way, with no source code, byte code, or application object changes required!

    Just my $.02

    Ward Mullins
    CTO
    THOUGHT Inc.
    http://www.thoughtinc.com
  9. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Hi Ward,

    Do you think OR mapping needs a standardized performance test like the ECPERF?

    "I'm seeing performance numbers published on this site where the open source guys run almost twice as slow as hand coded JDBC". Which one?
  10. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Matthew, that's a great question...

    Generally speaking we always compare against jdbc and not other vendors. Most vendors (including us) have license restrictions that limit folks from publishing numbers because it sooo easy for customers/developers to skew tests with apples to orange comparisons.

    For example a simple JDBC test where you know the predictable outcome and SQL is very different from a test where the JDBC reflects the typical real world need of changing bind parameters, potentially non-updated fields, object graph change detection, etc.

    When we build performance tests, we try to use baseline JDBC as the basis point, and make sure we issue identical SQL, and if possible do exactly the same operation conceptually. A good example of this is that we have a base runtime that lets you issue conceptual operations such as select, insert, update and delete, and these are equivalent to how most customers would write a raw JDBC simple test. But we also have our higher level object graph transparent persistence layer which simplifies graph management, but has more overhead than the runtime because of the higher level graph reconciliation logic that gets applied. If you use this higher level API to manage only root node instances and not graphs, it has more overhead than the runtime, but then the tests aren't equivalent because it is the wrong API to be functionally equivalent to the baseline test... What the TP facade buys the developer, is that it will actually make complex development easier, and potentially increase performance in complex tests & environments, but you could easily skew an apples to orange test with this API if you were disreputable - and that's why we don't allow anyone to publish numbers other than ourselves. However we ship the source code to our benchmark tests, so customers can see for themselves EXACTLY what's happening...

    Not to mention that amount of memory, thread usage, etc. can all play significantly into testing, as can the database used, JVM used, JIT used, OS of platform, record contention, etc., etc., etc. If a vendor has a single high benchmark number on one platform, it doesn't mean that the numbers will hold in another environment. This is why you get hardware and database vendors always slinging mud at each other...

    It's really easy to inadvertantly or on purpose skew any benchmark. That's why we have always worked directly with customers to make sure our system is optimized for their usage, and that they're using the appropriate APIs for the task at hand... After all a sledgehammer might kill a gnat, but it might also dent the furniture :)

    I used to do benchmarking as part of my responsibilities in an earlier life, so I have a lot of skeptic responses to an industry wide tests... I think a benchmark framework like we've developed that customers can use & extend with custom tests is actually the best approach, but that's just my opinion...

    Thanks for the posting, great question!

    Ward Mullins
    CTO
    THOUGHT Inc.
    http://www.thoughtinc.com
  11. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Hey Ward,

    Good thing you aren't a "religious zealot" like those "ODMG/JDO bigots".

    Dan
  12. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    All,

    As Ward mentioned, object/relational mapping tools have an incredible opportunity to do a good deal of analysis at transaction commit time, potentially drastically outperforming hand-coded SQL.

    With pure-JDBC or low-budget in-house solutions, SQL typically gets executed more or less in the same manner as the application developer performs business operations. Kodo JDO (and I'd assume other O/R mapping tools as well) can delay execution of SQL until the application developer commits the transaction, and then do some dynamic processing to optimize the commit process.

    Additionally, Kodo JDO's various caches -- both the intra-PersistenceManager and the PersistenceManagerFactory caches -- can result in quite significant speedups on the read side of the picture, as database roundtrips can be reduced in situations that repeatedly read the same data from the database.

    -Patrick
     
    --
    Patrick Linskey
    VP, Engineering
    SolarMetric, Inc.
    http://www.solarmetric.com
  13. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    Mr. Mullin

    Let's suppose I'm an Oracle + TopLink user.
    Would you to tell us in your opinion, what would be the benefit(s) to migrate to Cocobase ?

    * they are on the market for longer than Thought Inc.
    * they have more references
    * they now belongs to Oracle (this is our DB)
    * it is now for free, according to what Oracle says
    * is there any benefits in using Cocobase instead of TopLink (performance, scalability, flexibility, reliability, other 'ities') ?

    We have spent a lot of time and energy to learn the product, the APIs, the tools (GUI and other), the way to define mappings, the limits...
    Are we supposed to spend all this energy twice to migrate to Cocobase ?

    cheers, marc
  14. SolarMetric Kodo JDO 2.3 Released[ Go to top ]

    <quote>
    In JDO, you don't need to violate encapsulation by making your persistent fields public or always providing set/get methods. Not only that, but JDO can perform lazy reads of persistent data because it 'knows' whenever you attempt to access a persistent field. Unlike reflection-based systems, this lazy reading can apply to any field, not just Collections/Maps/special proxy types.
    </quote>

    So there seem to be three approaches to handling persistent classes:
    1. using reflection on unprepared classes (CocoBase, Castor, Jakarta OJB, Hibernate, etc)
    2. special preparation of persistent classes (JDO)
    3. method call interception on managed objects (EJB Entity Beans)

    I wonder how the first one recognizes dirtiness - maybe by keeping a copy of the loaded state and comparing it to the current state? Not very efficient, I guess, or do I miss something here?

    An interesting issue is the amount of toolkit-specific code you need to use in your application classes. Modifying your entities simply corresponds to calling some of their methods. The only O/R-specific code that is universally necessary is:
    - store commands (for both create and update);
    - delete commands;
    - lookup commands (for loading by id);
    - query commands (returning standard Collections);
    - transaction demarcation.

    Isolating that kind of code should not be too hard in a typical application, to be able to switch your O/R toolkit eventually, even to a different API. The hardest part would probably be queries, as you have to deal with converting between the various query languages. You could even let your entities implement JDO's PersistenceCapable interface but eventually use a non-JDO toolkit, omitting the JDO enhancer in this case...

    I would really like to read other opinions, especially from developers with experience in more than one toolkit!

    Juergen
  15. Juergen,

    <quote>
    You could even let your entities implement JDO's PersistenceCapable interface but eventually use a non-JDO toolkit, omitting the JDO enhancer in this case...
    </quote>

    Actually, with JDO, a developer never needs to explicitly implement the javax.jdo.spi.PersistenceCapable interface. It is the enhancer's job to perform the work necessary (either in Java using a source code enhancer, or in Java bytecode using a bytecode enhancer) to make your class implement that interface.

    The only interface that your domain objects might want to implement is the javax.jdo.InstanceCallbacks interface, which allows the object to act on certain lifecycle events.

    So, as you said, you could potentially write a class that is used in an entity bean, can be enhanced, and has all the appropriate public accessors / public fields to be used by a reflection-based persistence system.

    I haven't really thought about this too hard, but I think you'd run into issues with EJB2.0 CMP entity beans -- the EntityBean implementation in CMP2 has to be an abstract class, with abstract accessor methods for the container-managed fields.

    -Patrick

    --
    Patrick Linskey
    VP, Engineering
    SolarMetric, Inc.
    http://www.solarmetric.com
  16. Hi Juergen,

    I would just like to add the following.

    One of the pillars on which open based standards like JDO, Entity CMP stand on is the ease with which you can migrate from one implementation to another (We are not yet totally there but we are much, much closer that anybody else).

    If you have a JDO based application using some JDO implementation (Kodo or FrontierSuite for JDO, for example), if you want to migrate from one implemntation to another, there hould ideally be no need to modify any of your application code, the API you were talking about. Since all the implementations support the same API, you application is implementation independent - the only changes that can be visualized are the deployment level activities. (And for this JDO people are called 'bigots'. Really diificult to visualise somebody who wants to have a neutral discussion and wants to call people names at the same time. But that was on the side.)

    But as Patrick says, using a JDO class as a n Entity bean may potentially result in some problems, at the very least, you are going to end up with redundant code and a dual implemntation support.

    But there is one possible integration and this is because of the fact that both JDo and CMP are basically specs/frameworks for storing data and they are not very different. Maybe we will see Application Servers that use a base persistence engine that provides persistence to both CMP as well as JDO (Our persistence engine does that actually). It is only the API level differences that need to be incorporated. And maybe, JDO can be integrated with CMP (Who knows, this also may happen).

    regards

    S Rajesh Babu
    ObjectFrontier Inc
    www.ObjectFrontier.com
  17. <quote>
    Actually, with JDO, a developer never needs to explicitly implement the javax.jdo.spi.PersistenceCapable interface. It is the enhancer's job to perform the work necessary (...) to make your class implement that interface.
    </quote>

    Oops, an error in reasoning on my side. Somehow I considered PersistenceCapable a marker interface that the enhancer recognizes. But of course the enhancer makes the class implement the interface. So it should be possible to use the same entity classes for both JDO and reflection-based O/R toolkits. The only difference is that in case of JDO the classes must get enhanced in the build process.

    Concerning switching between toolkits: I am aware of the possibility of switching between various JDO implementations. And I don't care much about EJB CMP: Entity Beans have a quite different programming model, not really suitable for lightweight persistence of fine-grained entities.

    I am interested in switching between JDO and relection-based O/R toolkits, though. This should be possible via isolating the application-specific persistence commands (store, delete, lookup, query) in Data Access Objects, with DAO implementations for JDO, Castor, Hibernate, OJB, etc. You will probably need transaction demarcation across multiple DAO calls, but all toolkits seem to support

  18. [continued, hit wrong key ;-) ]

    You will probably need transaction demarcation across multiple DAO calls: Fortunately, all toolkits seem to support JTA in J2EE environments.

    Juergen

  19. Hi Juergen,

    <Quote>
    I am interested in switching between JDO and relection-based O/R toolkits, though. This should be possible via isolating the application-specific persistence commands (store, delete, lookup, query) in Data Access Objects, with DAO implementations for JDO, Castor, Hibernate, OJB, etc. You will probably need transaction demarcation across multiple DAO calls, but all toolkits seem to support.
    <Quote>

    Theoretically what you have visualized may be possible. But some of the potential issues I see are:

          a. How are the different frameworks going to interpret the same classes (assuming that should be the same).
          b. What about the deployment procedures ?

    Transaction demarcation across 'DAO's are possible in almost all. But what interests me is the need for migrating from a JDO implementation to a reflection based O/R mapping framework ? Is it because of any perceived advantages ? Any shortcomings in your opinion in JDO (There are many, but work is on on them and some of the O/R implementations cover this lacunea with their extensions).

    S Rajesh Babu
    ObjectFrontier Inc
    www.ObjectFrontier.com
  20. <quote>
    a. How are the different frameworks going to interpret the same classes (assuming that should be the same).
    b. What about the deployment procedures ?
    </quote>

    Of course it will be necessary to write deployment descriptors for each toolkit supported, and according deployment scripts, including enhancer invocation in case of JDO. With analogous mapping configuration, the toolkits should work similarly: in what they do, not in how they achieve it. As I understand the issue, O/R mapping configurations of JDO toolkits are not portable, so JDO does not provide any benefits in this respect - but neither does EJB CMP, by the way.

    The main reason for using a reflection-based O/R toolkit is that there are numerous open source ones available, e.g. Castor, Hibernate, Jakarta OJB. The latter is the only one of those that has JDO support, currently in beta. An interesting notion is that OJB has a reflection-based persistence broker kernel and builds a high-level JDO API on it...

    From the specification perspective, I don't really understand why JDO enforces the enhancer approach. It could easily allow for both ways: enhancement and reflection. Why doesn't it stick to specifying the API, leaving all further internal choices to the implementation? I wonder if a JDO implementation still can choose to use reflection, like OJB seems to do: by not caring about enhanced or not enhanced classes but just analyzing given entity classes via reflection.

    Juergen
  21. Juergen,

    One of the reasons that the specification defines an enhancement methodology is to permit bytecode compatibility across vendor implementations. That is, because the contract between PersistenceCapable and StateManager is defined in the specification, it is possible to use any PersistenceCapable class with any JDO implementation.

    This has a number of ramifications. The most obvious one is probably the ability to swap implementations if one vendor raises pricing or stops innovating. However, there are many other benefits to this bytecode compatibility. For example, you could write an application targeted for relational databases using Kodo JDO, but then sell it to someone who has an in-house JDO implementation for their proprietary data store needs. Provided that their JDO implementation was specification-compliant, they could run your application with no problems.

    If the JDO API did not either define a PersistenceCapable interface or define some rigid reflection requirements, this would not be possible.

    Obviously, JDO could have gone with a reflection-based persistence mechanism instead of an enhancement-based mechanism. This has become something of a philosophical debate that we really don't need to repeat, but here are some of the issues surrounding enhancement:

    - imposes less constraints on your object model. Fields in your persistent class can be private, protected, public, or default visibility, and there are no requirements that field accessor methods be defined.

    - adds an extra step to the compilation process, either before java compilation in the case of source code enhancers, or after compilation in the case of bytecode enhancers. On a related note, source code enhancers are in general a bit inconvenient, since they cause line number mismatches in the resultant bytecode, which makes debugging a bit awkward, and since they present people with the temptation to treat the generated source as something that should be added to a project's version control system, rather than treating them as intermediate files. Bytecode enhancement does not have either of these issues, since it performs its work on the Java bytecode created by the compilation process.

    - provides the opportunity for the JDO implementation to intercept all field accesses, so that fields can be lazily loaded in a completely transparent fashion, and changed fields can be marked as dirty upon change rather than by comparing field values to a copy made when the object is first loaded. What I mean by 'completely transparent' is that, because the enhancement mediates persistent and transactional field accesses through the StateManager, the application code need not make any extra calls before accessing any field. The JDO implementation is responsible for ensuring that any unloaded values are materialized upon demand. This is nice because it means that you can use persistent objects in code that was not written with any knowledge of JDO, or even persistence. So, you can invoke a 'doSomeBusinessStuff()' method on an object that was obtained from a database via JDO with no preparation of the object and with no requirements that the doSomeBusinessStuff() method know anything at all about persistence.

    - bytecode enhancement is a fairly unfamiliar technology to many people, so is often viewed as 'magic.' Personally, I've been using bytecode enhancers for a couple years now, and various different compilers for longer than that, so I don't share this view, but I hear questions about bytecode enhancement quite a lot. I think that these questions stem from the fact that many people do not realize that Java bytecode is a proper language specification in its own right, and bytecode enhancers are merely tools that parse class files into syntax trees and make modifications to the instructions in those files. This is analogous to the more familiar java compilation process, except that it modifies existing class files rather than starting with a human-readable source file and generating the corresponding bytecode in a class file.

    - enhancement makes modifications to the bytecode that make it possible for third-party classes to access the private persistent state of the classes, potentially allowing malicious code to access private state. This is true, but is not really a valid point in a debate between enhancement and reflection-based systems -- reflection-based systems have exactly the same requirement.

    - enhancement (source or bytecode) results in faster execution than reflection-based systems. This one is really not something that you should base a decision about enhancement v. reflection on, since invoking setters and getters is by no means a performance bottleneck in a persistence infrastructure -- instead, efficient database access methodologies and smart resource allocation tend to be overwhelmingly more important. A couple nanoseconds here and there are nothing compared to the cost of inefficient handling of large result sets or lack of (transparent) support for lazy loading.


    One of the big goals of the JDO standard is to provide persistence services in a way as closely aligned to the Java language specification as possible. So, in light of this goal, enhancement won as the mechanism for accessing fields, largely because of the point above about transparency. Because of the field visibility constraints possible in the Java language, there is no way to access all the data in a typical Java object -- any fields that are either not public or do not have public accessor methods cannot be accessed by a third-party class. So, you have three choices: provide a modified JVM (not usually a desirable alternative), require that persistent fields have some means of public access (essentially delegating the enhancement to the application developer, where the enhancement contract is that fields must have some means of public access), or perform bytecode enhancement.

    -Patrick

    --
    Patrick Linskey
    VP, Engineering
    SolarMetric, Inc.
    http://www.solarmetric.com
  22. <quote>
    From the specification perspective, I don't really understand why JDO enforces the enhancer approach. It could easily allow for both ways: enhancement and reflection. </quote>

    I thought it was you who (correctly) said that reflection-based systems cant really offer transparent persistence management, as they cannot intercept object write accesses? I think this is one of the greater benefits of JDO - once you have moved your object to the correct state, all modifications are automatically registered and written to the database at transaction commit. There are no "o.makeDirty()", "transaction.register(o)", or "o.save()" calls.

    What I also like about JDO, is the fat that you can move your object to the "transient" state, where it will behave like any object in memory without regard to persistence. In CMP, you would have to copy the object (and update all references, too).

    regards,
    Christian