JDO 2.0 (JSR-243) Early Draft Specification Published

Discussions

News: JDO 2.0 (JSR-243) Early Draft Specification Published

  1. The JDO 2.0 Expert Group is pleased to announce the publication of the Early Draft of the JDO 2.0 specification. This document is available for the public to download and study. An official review period has started and will close at the end of July. It is hoped that the public will take this opportunity to examine the work which the Expert Group has done and provide feedback to the comments alias (jsr-243-comments at jcp dot org).
     
    Download the JDO 2.0 Early Draft Specification.

    Visit the JSR-243 Home Page.
     
    The Early Draft is the fruit of many months of work, which commenced in August 2003 when the then "JDO 1.0.1 Maintenance Group" met in Washington DC to define the scope of JDO 2.0. The specification brings technical detail to bear on the JSR's high-level objectives. Of particular interest to the community will be several important areas of new functionality:
     
    • Object/Relational Mapping metadata specification - see Chapter 15
    • SQL Query support - see Chapter 14
    • JDOQL Improvements including aggregation, projection, query paging, and deletion by query - see Chapter 14
    • Detachment with long-lived optimistic transaction semantics - see Chapter 12
    • Fetch Plans for performance tuning and detachment demarcation - see Chapter 12
     
    There is also a section dedicated to specific requests for public feedback - see Chapter 27. Please note that we are planning a few additional items, in view of which the Early Draft should not be considered to be feature-complete.

    JDO is the JCP standard for transparent object persistence in J2SE, a role it has held since the JDO 1.0 standard was approved over two years ago. The new release is based on feedback from the community and from the 20 commercial and open-source providers of JDO implementations. With the Early Draft's standardization of object/relational mapping metadata and SQL queries, JDO's support of relational databases is clear. With the relaxation of the PersistenceCapable implementation strategy requirement (it is no longer mandated), the Expert Group have opened up the JDO market to established commercial and open-source providers wishing to exploit reflective, proxy-based and other implementation strategies. Furthermore, JDO's datastore neutrality facilitates the development of applications which are independent of the underlying datastore paradigm and are portable across different flavours of SQL, legacy repositories, object databases, and various file-based persistent storage solutions.
     
    The full benefits of JDO - arising from persistence which is datastore agnostic and transparent to the Java object model - can be realized inside a J2EE container with the same developer productivity that is witnessed in a J2SE context. Within a J2EE container persistent objects (which are POJOs) are managed by Session Bean, MessageDriven Bean, Servlet or JSP components through the same PersistenceManager API. This seamless pluggability of programming logic and developer skill-set illustrates how we have endeavored to make JDO the ideal persistence choice for projects of all sizes.
     
    We look forward to receiving your feedback on the Early Draft specification.
     
    Robin M. Roos
    Ogilvie Partners Ltd

    On behalf of the JDO 2.0 (JSR-243) Expert Group

    Threaded Messages (77)

  2. Cool![ Go to top ]

    Way-to-go Robin, this should move things along a little in the JDO world.

    Downloading...

    -John-
    CTO C24
  3. Very glad to hear it, congratulations! I look forward to digging into the spec and the benefits it will provide to Java developers.
  4. Congratulations[ Go to top ]

    Congratulations and well done to the Expert Group.
    Thanks for all your hard work.

    Jaco Uys
    http://www.jdogenie.com
  5. JSR-175 Metadata[ Go to top ]

    Robin,
       I was wondering whether the expert group has discussed the possibility of specifying the O/R Mapping metadata in JSR-175 annotations. If so, what were the pros and cons?
  6. JSR-175 Metadata[ Go to top ]

    Robin,   I was wondering whether the expert group has discussed the possibility of specifying the O/R Mapping metadata in JSR-175 annotations. If so, what were the pros and cons?
    From chapter 18: XML Metadata

    NOTE: J2SE introduced standard elements for annotating classes and defining the types of collections and maps. Because of these features, programs compiled with suitable metadata annotations and type information might not need a separate file to describe persistence information. This early draft does not include proposed JDO standard annotations but the intent is to do so in a future draft specification.

    [bold in original text]

    Cheers,

    Oliver Kamps
    www.fiftybar.com
  7. JSR-175 Metadata[ Go to top ]

    Hi Rob

    The JDO Expert Group does intend to support JSR-175 Annotations. Indeed, when I circulated the wording of this announcement to the expert group one of the experts reminded me of the lack of Annotation support in the Early Draft, and that in turn led to my comment about the draft not yet being "feature complete" (although there are a few other items outstanding as well).

    I believe that Annotations are a useful addition to the Java language, but that they should not be over-used. Where a capability has the same life-cycle as the source code itself, annotations are an appropriate markup mechanism for that capability. However, where the lifecycle of the capability is different to the lifecycle of the Java source code, Annotations are probably an inappropriate tool.

    Two possible cases might be:

    Persistence Modifier of a Field

    The persistence modifier allows developers to override the default nature of a field. By default, fields which are static, final or transient are not persistent. The persistence-modifier XML tag allows this default to be changed. It will usually be appropriate for this to be set in the source code through a corresponding annotation.

    Column Name of a Field

    The column name could vary for a number of reasons, whether you choose to map to a different schema, or change the name of a schema you have generated. These changes are independent of any other source-level changes. In this regard, marking up column names in the Java source code with Annotations may be less than ideal.

    Conclusion

    By the time the Proposed Final Draft of JDO 2.0 is ready there will be comprehensive support for Annotations. This will cover all of the JDO metadata and will probably extend to the ORM metadata also. However the spec will carry warnings that over-exploiting Annotations is not necessarily good for non-trivial applications.

    Kind regards, Robin.
  8. JSR-175 Metadata[ Go to top ]

    For my money I'd like to see all configuration/mapping done three ways, via annotation (similar to how many of us use xdoclet right now), via (xml) configuration file and programatically (AllKnowingEntity.addMapping("foo", "com.company.Bar") etc), with those three options being listed in lowest to highest precedence (ie, a mapping done in code overrides something done in a config file which would override an annotation). I guess there could always be debate over the precedence, but the important thing is to retain flexibility. Some people are excited about annotations, some of us really like xml configuration and there is always a need for programatic configuration of a framework.
  9. JSR-175 Metadata[ Go to top ]

    Hi Donald

    This is a great ideal. JDO 2.0 will provide the XML and the Annotation possibilities. I'm also not sure about the precedence.

    However, much of the JDO metadata required as part of the .jdo or .orm files cannot be done programatically. It's feasible to construct a fetch plan programatically, or a query, but it is infeasible to declare which classes are persistent, or to which tables they map, in a programatic manner which would preclude this information from being available at startup.

    Hopefully we will strike the correct balance in JDO 2.0.
  10. That's great![ Go to top ]

    That's great!I'm looking forward to the JDO 's implementation products.
  11. That's great![ Go to top ]

    There is already implementations avalible that gives you a preview of the jdo 2.0 features. I know jdogenie just rls'ed a beta.
    But yes It's really great and i hope jdo will be more accepted now with the new features.
    It's been really nice to develop with jdo and this will only make it better :)
  12. JDO Mature Enough[ Go to top ]

    Wao ! JDO2.0 looks mature enough. We had adapted JDO 1.0 for an application development but had tough time in creating/supporting complex queries.

    Looking ahead for a happy JDO way :)
    -SPanda
  13. Readers might like to know that the new features have been added to JDO in the 2.0 Early Draft whilst retaining complete backward-compatibility with JDO 1.0.1, which itself was backward-compatible with JDO 1.0. Of course this might change before the Proposed Final Draft.

    Given the added scope of JDO 2.0 over its preceding versions, and the lack of any necessity for deprecation in order to make way for technically detailed support of this scope, the resulting backward compatibility is a strong testimony to the essential correctness of JDO 1.0's architecture, APIs and semantics.
  14. SQL Query[ Go to top ]

    "The JDO implementation must not make any changes to the query string. The tokens “?” must be used to identify parameters in the SQL query string."

    Why it was decided to specify how implementation must process query string.
    I see no problems if implementation does not alter query execution plan ( for performance tunning ), load fields I do not select or transforms it to multiple queries. It will mean "bad tool", but I see no reasons to say it
    in specification.
  15. SQL Query[ Go to top ]

    Hi Juozas

    If the user is coding a SQL Query then they have specific knowledge of the schema. They are using SQL for a reason, and one of those reasons might be that for some reason they do not trust the SQL generated by the tool. For this reason the JDO specification requires that SQL Queries be unaltered by the tool. In this way we provide the safeguard that, in the minority of cases where the SQL generated by the tool is deemed inappropriate, you can take complete control.

    I believe the combination of SQL Queries and Named Queries will be very powerful. By externalizing JDOQL into Names Queries, the application merely invokes the query by name. Later changing the query to be SQL that is hand-crafted by the DBA requires only a change to the named query definition (which resides in an external XML file). No changes are required to the application code which invokes the query and uses the results.

    This gives users a clear opportunity to work with JDOQL (which is easy to write and requires no schema-specifics), in the knowledge that they can later convert certain queries to SQL on a case-by-case basis with little or no application impact. Usually a SQL query would be targeted as a performance optimization, and developers should not spend time pre-optimizing their code.

    Also, SQL queries live in ORM files and there can be separate ORM files for each database mapping. So you can exploit SQL even if you map the application to more than one schema.

    Cheers, Robin.
  16. JDO at JavaOne[ Go to top ]

    I will be attending JavaOne this year. I expect to attend relatively few sessions, but instead spend most of my time in the vicinity of one or other of the JDO vendor stands, or the Sun "JDO Pod", in the Pavillion. I look forward to meeting as many members of the TSS community as possible and discussing the merits and positioning of JDO.

    See you all there?
    Robin.
  17. JDO at JavaOne[ Go to top ]

    See you there! :)
  18. Quick Tutorial?[ Go to top ]

    Instead of going through the spec, is there a quick example/tutorial on how dealing with a persistent class would look in the new JDO? Just to get the feel....
  19. Re: Quick Tutorial?[ Go to top ]

    Instead of going through the spec, is there a quick
    > example/tutorial [...]?
    Robin Roos' book is a good tutorial on JDO (http://www.amazon.co.uk/exec/obidos/ASIN/0321123808/ref=sr_aps_books_1_1/026-9206648-9401202) alternatively, if you want something really snappy, there are a number of very short articles referenced on Sun's JDO page: http://access1.sun.com/jdo/

    Eoin.
  20. Re: Quick Tutorial?[ Go to top ]

    Thanks for recommending my book, Eoin.

    As far as the "new" JDO is concerned, it is just like to "old" JDO. Remember we've had no cause to sacrifice backward compatibility for the JDO 2.0 Early Draft. The management of persistent objects is the same except that there is more stuff you can now do:

    - detach a persistent object (or object graph) from the PersistenceManager and later reattach it to a different (but equivalent) PersistenceManager. In the interm the object (graph) might have been altered, and it might have been involved in a round-trip between the tier that has the PM and some other tier which has no PM and no database connection.

    - use SQL to express queries, which return persistent instances or which return projections of selected field expressions, which might involve aggregation etc. Direct SQL usage can compromise portability, but if the SQL is externalised as named queries in the ORM file then you're reasonably well off, since there are different ORM files for each schema to which you map.

    - use JDOQL's new aggregation/projection/paged query/delete-by-query capabilities, so you almost never have to use SQL if you don't want to. This gives you the best level of portability, since JDOQL must be supported by every JDO implementation regardless of the underlying datastore

    - execute queries in a paged fasion, retrieving pages of rows at a time.

    - define named Queries, in both JDOQL and SQL, so that query definitions do not "pollute" application logic.

    - define fetch plans so that the exact sets of fields and related objects which YOU need for YOUR PARTICULAR use-case are fetched from the database in the most efficient way, to prevent unnecessary subsequent hits on the database. In this usage fetch plans are non-functional, which means performance tuning can be done without affecting functional application behaviour.

    - use Simple Identity, which is a special case of Application Identity that caters for the trivial case whete there is only one primary key field and it is of a well-known type: with Simple Identity you do NOT have to provide a separate Object ID class for each persistent class.

    - exploit a standard metadata representation of object-relational mapping. This standard strikes an excellent balance between being comprehensive for the complex case, and yet being very concise for the common case. Chapter 15 is full of examples. If you stick to the standard O/R Mapping metadata, then the application you write is portable across other JDO Implementations presuming (a) they support the JDOR feature, and (b) the schema is the same in each case.

    So, with JDO 2.0 you can do more. But the fundamentals are the same as they were for previous JDO releases. Any of the available books on JDO are a great start - you could even try the Chinese edition of my book which is getting great reviews.

    Finally it is my hope that various members of the JDO Expert Group will shortly publish some articlaes illustrating the use of JDO 2.0's new features, although I don't think this will happen until after JavaOne.

    Kind regards, Robin.
  21. Re: Quick Tutorial?[ Go to top ]


    detach a persistent object (or object graph) from the PersistenceManager and later reattach it to a different (but equivalent) PersistenceManager.

    I'm really new to JDO. So please bear with me if this really dumb question. I want to know what you mean by equivalent. If for some reason and application uses 2 different JDO providers can the object/graph detached from PersistenceManager of one be attached to PersistenceManager of another?
  22. Re: Quick Tutorial?[ Go to top ]

    If for some reason and application uses 2 different JDO providers can the object/graph detached from PersistenceManager of one be attached to PersistenceManager of another?
    No, you can't use detach()/attach() for that. Instead you'd have to use makeTransient()/makePersistent().

    Detach/attach employs long-lived optimistic transactions, and the instances must be attached to the same datastore managed by the same JDO implementation. I use the phrase "equivalent PM" because - and this is very important - it doesn't have to be physically the same PM instance. This makes detach/attach useful in cases like web-requests where the PM is closed (ie. returned to the pool) after the objects are detached. Later another equivalent PM is obtained from an equivalent PMF (not necessarily the same PMF, but a PMF of the same vendors implementation and for the same datastore) and the objects can be reattached. If the reattached objects are seen to have been altered by another transaction in the interim you get a JDOOptimisticVerificaionException.

    On the other hand, with makeTransient()/makePersistent() you can move instance of a common object model from one JDO implementation and datastore to another JDO implementation and datastore.

    Does that help?

    Keep simple questions coming by all means, but for reference a lot simple JDO questions are answered on the community forums at www.jdocentral.com .

    Kind regards, Robin.
  23. Re: Quick Tutorial?[ Go to top ]

    If for some reason and application uses 2 different JDO providers can the object/graph detached from PersistenceManager of one be attached to PersistenceManager of another?
    No, you can't use detach()/attach() for that. Instead you'd have to use makeTransient()/makePersistent().

    Detach/attach employs long-lived optimistic transactions, and the instances must be attached to the same datastore managed by the same JDO implementation. I use the phrase "equivalent PM" because - and this is very important - it doesn't have to be physically the same PM instance. This makes detach/attach useful in cases like web-requests where the PM is closed (ie. returned to the pool) after the objects are detached. Later another equivalent PM is obtained from an equivalent PMF (not necessarily the same PMF, but a PMF of the same vendors implementation and for the same datastore) and the objects can be reattached. If the reattached objects are seen to have been altered by another transaction in the interim you get a JDOOptimisticVerificaionException.

    On the other hand, with makeTransient()/makePersistent() you can move instance of a common object model from one JDO implementation and datastore to another JDO implementation and datastore.

    Does that help?

    Keep simple questions coming by all means, but for reference a lot simple JDO questions are answered on the community forums at www.jdocentral.com .

    Kind regards, Robin.

    PM=PersistenceManager, PMF=PersistenceManagerFactory
  24. Re: Quick Tutorial?[ Go to top ]

    "detach()/attach()" and "optimistic transaction" is not so simple and it very confused me some time ago. It is not transaction by formal definition, it is not atomic, not isolated, it can not be recovered. Optimistic transaction or "optimistic certifier" is something else in concurency control theory.
  25. Re: Quick Tutorial?[ Go to top ]

    Hi,
    "detach()/attach()" and "optimistic transaction" is not so simple and it very confused me some time ago. It is not transaction by formal definition, it is not atomic, not isolated, it can not be recovered. Optimistic transaction or "optimistic certifier" is something else in concurency control theory.
    I am unsure which bit of Robin's post you are referring to, but could you please explain why optimistic transactions do not have transactional properties from the client's point of view?

    I think that, from a client's point of view, they atomic, consistent, isolated, and durable.

    Kind regards,

    Oliver Kamps
    http://www.fiftybar.com
  26. Re: Quick Tutorial?[ Go to top ]

    I am not a good speaker, I have tried to explain it once in forum, but probably nobody understand it. First you need to demarcate "action" before to talk about transaction (there is nothing to recovery if you do not have "begin"). Concurency control theory is based on serialization theory, it is a special kind of graph theory in math. It is possible to demonstrate "cascading abort" phenomen (one of flaws in "optimistic transaction"). It is not nesary "bad" in theory and it can work in practice, but it is not a transaction.
  27. Re: Quick Tutorial?[ Go to top ]

    I am not a good speaker, I have tried to explain it once in forum, but probably nobody understand it. First you need to demarcate "action" before to talk about transaction (there is nothing to recovery if you do not have "begin"). Concurency control theory is based on serialization theory, it is a special kind of graph theory in math. It is possible to demonstrate "cascading abort" phenomen (one of flaws in "optimistic transaction"). It is not nesary "bad" in theory and it can work in practice, but it is not a transaction.
    Transactions are frequently defined as a set of operations exhibiting ACID properties (as described in my previous post). Which of these ACID properties do JDO optimistic transaction not exhibit?

    Thanks,
    Oliver

    --
    Oliver Kamps
    http://www.fiftybar.com
  28. Re: Quick Tutorial?[ Go to top ]

    I hope you do not provoke me :), we will write a book about concurency control,recovery and transactions if we will start to prove it and to add "preasumptions" to this algorythm.
    I am not a good speaker, I have tried to explain it once in forum, but probably nobody understand it. First you need to demarcate "action" before to talk about transaction (there is nothing to recovery if you do not have "begin"). Concurency control theory is based on serialization theory, it is a special kind of graph theory in math. It is possible to demonstrate "cascading abort" phenomen (one of flaws in "optimistic transaction"). It is not nesary "bad" in theory and it can work in practice, but it is not a transaction.
    Transactions are frequently defined as a set of operations exhibiting ACID properties (as described in my previous post). Which of these ACID properties do JDO optimistic transaction not exhibit?Thanks,Oliver--Oliver Kampshttp://www.fiftybar.com
  29. Re: Quick Tutorial?[ Go to top ]

    I have asked a polite question. I do not believe this qualifies as provocation.
  30. Re: Quick Tutorial?[ Go to top ]

    I have asked a polite question. I do not believe this qualifies as provocation.
    It is polite, no problems, we just can not prove any of this stuff in forum.
  31. Re: Quick Tutorial?[ Go to top ]

    Hi Juozas

    I'm sorry to hear that you are confused. I'll do my best to help.

    JDO has always had "Optimistic Transactions". These are transactions in which the work done by the application occurs largely without a datastore connection. Optimistic transactions are underpinned by concurrency assumptions - it is assumed that no other transaction will update the same data whilst the optimistic transaction is underway. On commit() of the optimistic transaction a connection with the datastore is obtained, a transaction is commenced on that connection, and the concurrency assumptions are verified. If it transpires that some other transaction has updated any of the objects invoolved then the optimistic transaction fails with a JDOOptimisticVerificationException and the underlying connection transaction is rolled back. Otherwise the updates are flushed to the database and the underlying connection's transaction is committed.

    Optimistic transactions are great when there is user interaction between reading data and writing back changes, as they prevent locks from being held on the underlying database whilest the user "goes out to lunch", etc.

    In JDO 2.0, the semantics of optimistic transactions are provided for detach()/attach(). A detached instance knows its Object ID and its "Version". Upon attach(), consistency checks are made which validate that these objects have not been changed by any other transaction. If they have been changed, then the JDOOptimisticVerificationException is thrown. Otherwise the changes made in the detached state are synchronized into the PM cache of persistent instances and, on commit() of the JDO transaction within which the attach() took place, they are propagated durably to the database.

    There are several different ways of managing "versions" of instances; perhaps the most common way is to use an integral version number, but options for timestamp-based and state-comparison strategies are also available. Users choosing to use these strategies should understand that timestamp and floating point comparisons may not be precise.

    So, are optimistic transactions useful? Yes. Are optimistic concurrency semantics appropriate for detach/attach? W believe so, but look forward to community feedback as people read the Early Draft specification. Are optimistic transactions ACID? Only if the same connection is used to read all of the objects that form part of the transaction, in which case a read-only transaction on the connection can be used to guarantee an element of transaction isolation. Does that matter? Again, it's up to the community to give us feedback on these issues. I suspect some vendors will provide for retention of a connection by a PM to facilitate this, but it might be at the expense of resource utilization (less efficient connection pooling) and the specification does not mandate such a policy.

    However, I don't actually write JDO Implementations (I consult to users of JDO technology) and the Vendors might have more comments on this topic.

    Kind regards, Robin.
  32. Re: Quick Tutorial?[ Go to top ]

    Yes, understand it later myself, it is the same as CVS concurency control.
    I understand how versions are usefull in practice and I call it "version checking" not transactions with ACID properties.
     You well said "Only if the same connection is used to read all of the objects that form part of the transaction" and probably it must be added more "if". I am not not going to solve this problem on forum and to use math to prove "flaws" (I do not think I am expert too). "Ideal" concurency control algorythms are not used any way, I think nobody can use "predeclared write/read sets" in practice. RDBMS solves many concurency control problems (no tools can solve all concurency control problems) and if I can delegate it for RDBMS then I will do it. I can write applications without "attach" workaround and I think it is an antipattern to (It is not easy to prove), but I use versions to solve more "ugly" problems too.
  33. Re: Quick Tutorial?[ Go to top ]

    Ok Juozas, that's cool.

    I agree that few RDBMS products really handle isolation in a pure way (perhaps Firebird does?). JDO's attach/detach gives users the semantics which they expect, equivalent to the semantics of non-detached optimistic transactions against the same datastore. If you and I were to get into mathematical proofs of correctness we would soon loose the interest of the community!
    I can write applications without "attach" workaround and I think it is an antipattern [snip]
    The attach/detach feature fills a niche for users who want to
    • retain instances in a servlet session etc. without retaining database resources, such as connections
    • expose domain objects directly to the client tier
    I think the first usage is great. I think that in many applications there are other concerns which make decoupling the client from the domain object model beneficial. This is typical in enterprise applications (whatever enterprise means today!), and in such cases SDO (JSR-235) might mature to be the preferred detachment representation.

    Cheers, Robin.
  34. Correction[ Go to top ]

    Sorry, I meant to say: I agree that few RDBMS products rarely (i.e. infrequently) handle isolation in a pure way.
  35. Re: Quick Tutorial?[ Go to top ]

    I just think it is better not to fight with HTTP. I do not know any good use case for attach/detach feature (It does not implies version checking and version checking does not implies attach/detach), but probably somebody has a good use case for detached object graphs. My "persistent objects" just never live longer than transaction (I speak about transactions implemented by DB). attach/detach can cause scalability problems ( synchronized access, memory leaks ), I am sure it is possible to make it work in practice, but it is a workaround any way (It must be the last way to solve any problem).
  36. Detachment and HTTP[ Go to top ]

    Hi Juozas

    Detachment within the web tier has a very fundamental use case which involves the management of updates to persistent data over a number of successive HTTP Requests, with a desire only to commit changes at the end of the request series.

    By using detachment judiciously, the appearance of committed but incomplete data sets in the database can be avoided.

    Now this is a particular use-case. I acknowledge your "anti-pattern" comment, although you must acknowledge that other users of JDO technology have other complexities with which they deal, as well as differing perspectives on the applicability of techniques such as detachment to the problem domain in which they work.

    Kind regards, Robin.
  37. Detachment and HTTP[ Go to top ]

    if CGI can solve this problem then it is not a problem. CGI scripts use "hidden" form fields to store this sate (not the bad way too), we can store it HttpSession attributes. It is a user input in the most of cases, is not it ? I do not think it is a good idea to store this temprory data in database too, but I see no problems to recreate object per request. As I understand this feature implies local cache storage in httpSession, I think this is antipattern, not Detachment itself.
    Probably we will not agree on this too, many people want the tool to solve this problem, It is just not a problem for web (most of my applications are JAVA web applications or some kind of web/messaging frontends for "legacy" systems too).
  38. Detachment and HTTP[ Go to top ]

    There are many ways to skin the proverbial cat, so we probably have to agree that I will deliver this technology through JDO and you will choose not to use it, but others might.

    Below is an hypothetical example of a web interaction which spans several pages. I indicate what the user is doing, followed by what JDO might do. JDO actions are presumably initiated by code running in a servlet engine, although other architectures are equally plausible. What I show is one possible approach which exploites detach/attach; not necessarily the best approach.

    HTTP Request 1 - Choose menu item to alter a travel itinerary.

    JDO - No action.

    HTTP Request 2 - Select an existing travel itinerary.

    JDO - Get a PersistenceManager from the PersistenceManagerFactory. Query for the itinerary object, and detach it along with the small graph of reachable objects necessary to satisfy this use-case. Close the PersistenceManager (resources go back to the pool).

    HTTP Request 3 - Update passenger contact details.

    JDO - take form parameter data from request and update Address and Phone objects in the detachment. No database access and no connection.

    HTTP Request 4 - Update passenger meal preference.

    JDO - take form parameter data from request and update PassengerPreference objects in the detachment. No database access and no connection.

    HTTP Request 5 - Update passenger window/aisle seating preference.

    JDO - take form parameter data from request and update PassengerPreference objects in the detachment. No database access and no connection.

    HTTP Request 6 - Create a new comment requesting musician's allowance for the bagpipe as hand-luggage.

    JDO - create a new transient Comment object. Take form parameter data from request and populate the transient Comment object. No database access and no connection.

    HTTP Request 7 - Review details

    JDO - Data from the object graph (a combination of detached and transient instances) is used to populate the summary form. No database access and no connection.

    HTTP Request 8 - Save details

    JDO - Get a PersistenceManager from the PersistenceManagerFactory. Attach the object graph. "Detached" objects are updated in the database as long as they were not changed by another transaction. "Transient" objects are inserted into the database. Commit the transaction and close the PersistenceManager (resources go back to the pool). If commit is successful thank the user for using the site, otherwise tell them what failed.
  39. Detachment and HTTP[ Go to top ]

    If all resources are released on step 2 then I see no problems, but as I understand PM with local cache will be stored in httpsession (memory is an important resource too and I think cache per httpsession is not a good idea "antipattern" ). All objects stored in httpsession must be thread safe. Is PM thread safe or it depends on implementation ?
  40. Detachment and HTTP[ Go to top ]

    Juozas, have a good look at Chapter 12. The detached objects are independent of the PM. They require no supporting infrastructure. The PM is not held in session. So the difference here is whether you store the detached objects in session (the approach I outlined), or whether you store only accrued form data in the hopes that you can meaningfully apply it on "Save".

    In a web cluster the Session is serialized between machines, so obviously keeping Session memory footprint small is a good design goal in web applications. The question is, how small is small enough...?
  41. Detachment and HTTP[ Go to top ]

    BTW I glad about SQL support, "attach" is not important for me and as I understand it is more related to implemention than to specification.
  42. SQL Support[ Go to top ]

    I'm glad you're pleased with SQL Support. In what ways might you use it?
  43. SQL Support[ Go to top ]

    First I want to use it to work with trivial data structures (bean,map, list or array ). If more concreate http://voruta.sf.net, I am going to use more abstract "driver" and probably to add more AOP features. It is experiment at this time, I do not use solve "mission critical" problems (everybody thinks his problem is "mission critical" any way :), but I hope declaratyve programming will be "Next Big Thing" and this thing will be good after we will have new JAVA features. Probably I will find more usefull features in JDO later too, if it will help to use better model then I will be happy too (I do not drop idea to implement JDO for RDBMS myself).
  44. Use-case for attach/detach[ Go to top ]

    Hi Juozas
    I do not know any good use case for attach/detach feature
    It is very useful for thick client applications (e.g. Swing GUIs). They can get detached graphs from the server, modify them and send them back. Without attach/detach ypu have to write DTOs and so on to do this. For some apps the looser coupling provided by DTOs is worth the work but for many others attach/detach will save a lot of time.

    Cheers
    David
    www.jdogenie.com
  45. As Robin mentioned earlier, JDO2 is an evolution from the existing JDO specification. Existing JDO1 applications will not need to be recompiled to use them with JDO2, so the programming model will look similar to how things work in JDO1.

    Probably the most immediately visible change in JDO2 is the standardization of the O/R mapping metadata format. The spec includes several examples of the new mapping format in section 15 (page 149 in the EDR). (For simplicity, the mapping files in these examples are broken out into a standalone O/R mapping XML document (a .orm file). This is optional -- the O/R mapping info can be interspersed with the .jdo metadata file as well.)

    Another neat new XML addition is the JDO2 fetch group configuration. See section 12.7.2 / page 107 for some examples of this. This includes some additional methods in the PersistenceManager, Query, and Extent interfaces, and a new interface called FetchPlan which can be used to configure fetch groups.

    We've seen a lot of Kodo users taking advantage of Kodo's previews of the detachment APIs and the aggregate and projection support, so we anticipate that these will be among the more popular JDO2 features. Sadly, the JDO2 EDR doesn't contain any examples of the detachment APIs, but you can see some examples of the new query capabilities in section 14.10 / page 143.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  46. SQL Query[ Go to top ]

    No, named queries are not so powerfull as some people think (I thought this way before too) DBA can tune views, but I do not think some DBA modifies java or XML files, he will paste query from log and I will ask me to "fix" it. I see no problems to "grep" files, it can be XML or JAVA (query must be satatic). The best way to write query is "SELECT something FROM MY_VIEW" any way.
  47. SQL Query[ Go to top ]

    Cool - you can define a SQL Query in JDO 2.0 which selects from a view, so if that's the way you prefer to do it, no problem.
  48. SQL Query[ Go to top ]

    I see SQL query returns arrays or mapped objects. I need some way to select "structs" without fake class mapping, it must be trivial to map "better" name in view or with SQL alias, I do not need any state management for this kind of "objects" too.
  49. SQL Query[ Go to top ]

    I see SQL query returns arrays or mapped objects. I need some way to select "structs" without fake class mapping, it must be trivial to map "better" name in view or with SQL alias, I do not need any state management for this kind of "objects" too.
    You can have the JDO implementation automatically pack the column values from the result of a SQL query into a class of your choosing using Query.setResultClass(). JDO will match the column labels (as reported by the result set metadata) to public fields or setter methods in your result class, and instantiate one result object per row returned. This also works for JDOQL queries with projections, by the way (except instead of result set metadata, the property names are taken from aliases assigned in the Query.setResult() string).
  50. SQL Query[ Go to top ]

    I understand it is not supported for SQL queries after reading it first time. It is very usefull feature for me, I want to update,insert,delete data this way too.
     I think it is good way to implement scalable applications ( It is not any kind of recommendation, I can change my thinking about "architecture" at any time myself)
  51. Good progress[ Go to top ]

    It's nice to see this progressing; this draft makes me hopeful that we'll see a finished JDO 2.0 sooner rather than later. We're already using some of these features (supported by recent versions of Kodo), and I'm quite pleased with them. JDO is maturing nicely!
  52. Great job, team.[ Go to top ]

    This is a good example of the JCP at work: an expert group basing a JSR on understanding learned from real-world experience.

    Nice job.
  53. Play devil's advocate here:

    Form the responses in the thread it seems like that people are much warmer to JDO 2.0 that 1.0.x. However, having a kicking-ass spec is <20% of a success story. Today what really pushes the technology over the hump is an free or even open source RI. From my observation of the politics involved, don't expect Hibernate and TopLink to be JDO 2.0 compliant any time soon, and BEA and IBM will not likely to whole-heartedly support JDO. So it is up to the 20 or so current JDO vendors to give a feature complete but free RI, and then compete on added-features. In the JDO market as any others, economic laws only allow up to 3 commercial vendors, and a small numbers of open-source ones. So if you want to make a name for yourself for JDO product, give away your product and as it get populate negotiate to have it bought by IBM/ORCL/HPQ (don't sell it to SUNW please).
  54. Hi Eric

    There's no need for the "devils advocate" prelude to your post. The JDO community (EG members, Users and Vendors alike) are all aware that a strong open source implementation is a necessary prerequisite to world domination by JDO.

    The JDO 2.0 spec cannot be ratified by the JCP until we have a Reference Implementation. The provision of the RI has been a subject of recent discussion in the Expert Group, and some open source projects are being considered alongside non-open source candidates.

    Of course, just a few months ago we had hoped that Hibernate would be the JDO 2.0 RI.... ;-(

    Cheers, Robin.
  55. Robin:

    Good to hear that a JDO 2.0 RI is part of the spec. I, as with others here, am eagerly awaiting its official arrival. BTW, was there an RI for JDO 1.0?

    Also, please play nice to Rod Johnson and Juergen Hoeller so that Spring will include support for JDO 2.0, like it does for Hibernate and iBatis :-). That way we can all say sayonara to EJB 3.0, the JBoss-sponsored but ill-advised persistence spec IMHO.
  56. Hello again, Eric.

    Yes, there is an RI for JDO 1.0. Since this version did not specify anything perculiar to relational databases the RI did not have to use SQL, and Sun produced an RI which implemented JDO against a simple file store.

    In my mind that's what an RI is for - something which proves that the specification is implementable, but not necessarily something really useable for production applications. No-one in the JCP requires that an RI be efficient or stable - all that is required is that when it works it works correctly. Tomcat is the exception!

    Every JCP-ratified standard must have three components:
    • Specification, detailing the APIs and sematics of the standard
    • Reference Implementation, proving the standard to be implementable
    • TCK, a set of tests which validate an implementation's compliance to the spec
    If any one of these three components is missing then JCP cannot ratify the specification as a standard.

    JDO 2.0 standardizes O/R Mapping and supports SQL as a query language along-side JDOQL. So the JDO 2.0 RI must be capable of illustrating these features. This does not mean that the RI will necessarily be appropriate for production deployment, but it will be much closer to being useful in the real world than the JDO 1.0 RI was.

    And yes, I (and others in the JDO community) already play very nicely with Rod and Jurgen.

    Good night, Robin.
  57. Spring + JDO[ Go to top ]

    Eric,

    If you'll be at JavaOne next week, swing by the Pavilion Floor Theatre #1 on Tuesday at 2:00 -- Bruce Tate and I will be presenting about some work that we've been doing with the Spring folks to get Spring's JDO2 support up to speed.

    If not, then, well, send me an email next week (pcl at solarmetric dot com) and I'll send you an electronic copy.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  58. Spring + JDO[ Go to top ]

    Deal. I will stop by your booth next week.
  59. If you'll be at JavaOne next week, swing by the Pavilion Floor Theatre #1 on Tuesday at 2:00 -- Bruce Tate and I will be presenting about some work that we've been doing with the Spring folks to get Spring's JDO2 support up to speed.
    OK, I'm an idiot -- the talk is on Wednesday at the Pavilion Floor Theatre #1 at 2:00. Note the date change.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  60. identifying relationships[ Go to top ]

    Well, since we're all assembled here, I might as well share with y'all (right? I'm german) the following question, which I also sent to the jsr-interest address for comments:
    Hello,

    after skimming over the new spec draft, I did not find a reference to
    a feature I have been missing since the early days,
    namely "identifying relationships". By this I mean that a persistent
    entity may be identified by a relationship to another persistent
    entity. In relational terms, this would mean that the primary key of
    the parent entity would migrate as a foreign key into the primary key
    of the subordinate entity. Such relationships can be of 1-1 or 1-N
    cardinality, i.e. the foreign key can be the full primary key or part
    of it.

    To me, it seems completely arbitrary to limit the set of types
    allowable for identfication to primitive "value-" types. Identifying
    relationships as I describe them are a very common modeling construct
    in the relational/ER world, and they reflect a domain concept that
    cannot otherwise be expressed/maintained.

    thanks,
    Christian Sell
  61. identifying relationships[ Go to top ]

    Hi Christian

    Thanks for the comment. The specification lead will no doubt reply to your email formally during the review process. In the mean-time...

    JDO 2.0 introduces the concept of "dependent" relationships.

    Section 12.6.7 - Delete persistent instances

    Section 27.10 - Embedded, dependent and serialized values

    Section 18 - search for the "dependent", "dependent-key", "dependent-value" and "dependent-element" attributes which appear in various elements of the JDO metadata.

    JDO also supports the generation of foreign-key constrains with delete actions such as "cascade". This is relevant for users wishing to use JDO Implementations to generate schema.

    Section 15.5 - Foreign Keys

    Section 18.12 - ELEMENT foreign-key

    I hope that helps. Once you've gone through that material we can discuss whether what we have provided meets your requirement, or whether further capabilities are needed in JDO 2.0.

    Thanks for your input,
    Robin.
  62. identifying relationships[ Go to top ]

    Hello Robin,

    Maybe I was not clear in my question, so this is a good point to start clarifying. My point does not deal mainly with relational specifics (as I said, identifying relationships are a domain modeling contruct), and I dont see how the references you provide relate to it.

    What I would like to see is that the identity of a persistent class can be defined such that it is (partly) provided by a reference to another persistent class. In the case of application identity this would mean that the key class contains a persistent relationship to another persistent class (of course, IMO, the existence of the key class construct in itself makes this whole thing somewhat awkward).

    In terms of domain modeling, imagine a subordinate entity (object, class) that does not have identity of its own, but wholly depends on a related entity.

    Christian
  63. identifying relationships[ Go to top ]

    Hi Christian

    Sorry for sending you on a wild goose chase!

    If the child object has no identity of its own, then would the following suffice?

    1. Make the Child object have Datastore Identity
    2. Make the relationship from Parent to Child be dependent (or perhaps even embedded)?

    Now, if the child object does have an identity, but its identity is partly based on its parent's identity, things get awkward.

    If the "primary key" of the Order is {MonthNumber + OrderNumber}, and the "primary key" of the LineItem is {MonthNumber + OrderNumber + LineNumber}, and if both classes had Application Identity, then it would be great if the ObjectID class of the LineItem (call it LineItemPK) could have two fields:

    1. A field of type Order (or more likely OrderPK)
    2. A LineNumber field.

    This would be in contrast to the present definition of Application Identity which required LineItemPK to have all three simple fields (i.e. not persistent references).

    JDO 2.0 does NOT address this, but we are open to feedback and discussion concerning such a capability.

    Kind regards, Robin.
  64. identifying relationships[ Go to top ]

    child object has no identity of its own, then would the following suffice?
    1. Make the Child object have Datastore Identity
    2. Make the relationship from Parent to Child be dependent (or perhaps even embedded)?
    In any case, that would be a hack IMO. I am asking for first-class-support for the concept of identifying relationships.
    Now, if the child object does have an identity, but its identity is partly based on its parent's identity, things get awkward.If the "primary key" of the Order is {MonthNumber + OrderNumber}, and the "primary key" of the LineItem is {MonthNumber + OrderNumber + LineNumber}, and if both classes had Application Identity, then it would be great if the ObjectID class of the LineItem (call it LineItemPK) could have two fields:
    1. A field of type Order (or more likely OrderPK)
    2. A LineNumber field.
    This would be in contrast to the present definition of Application Identity which required LineItemPK to have all three simple fields (i.e. not persistent references).
    JDO 2.0 does NOT address this, but we are open to feedback and discussion concerning such a capability.Kind regards, Robin.
    Yes, that sounds like the type of support I am looking for. Personally, I dont like the idea of the key class in the first place (its a contruct that has to be provided only for the sake of persistence, which goes against the idea of transparency IMO), but I guess I'll have to live with that.

    BTW, I know of at least one JDO implementation (JDOGenie) that offers this capability as a custom extension. They require you to declare the key variables from the parent class in the subordinate (key) class also, and map them redundantly both as key fields and relationship fields.

    EJB has the same restrictions, and requires similar workarounds.

    Christian
  65. identifying relationships[ Go to top ]

    ... if both classes had Application Identity, then it would be great if the ObjectID class of the LineItem (call it LineItemPK) could have two fields:1. A field of type Order (or more likely OrderPK)2. A LineNumber field.This would be in contrast to the present definition of Application Identity which required LineItemPK to have all simple fields...
    Yes, that sounds like the type of support I am looking for.
    FYI, all JDO implementations should support the workaround of mapping the key columns as both simple value primary key fields and non-primary-key relationship fields. By implication, I think this is even required in JDO 2, which standardizes O/R mapping and never says the same column can't be used in multiple mappings.

    I believe the restrictions on primary key fields were put in place to make them compatible with EJB, which, as you mentioned, has the same restrictions.
  66. identifying relationships[ Go to top ]

    By implication, I think this is even required in JDO 2, which standardizes O/R mapping and never says the same column can't be used in multiple mappings.
    to conclude that because the spec does not mention redundant mappings, support for such mappings would be a requirement sounds a little far-fetched to me.
    I believe the restrictions on primary key fields were put in place to make them compatible with EJB, which, as you mentioned, has the same restrictions.
    I am glad that JDO hasnt inherited all other restrictions from EJB as well ;-)

    christian
  67. identifying relationships[ Go to top ]

    I am glad that JDO hasnt inherited all other restrictions from EJB as well ;-)
    FYI, that's pretty much the only one -- the only place where EJB restrictions sneak into JDO is in the application identity definition.

    Also, I agree with your sentiment -- I think that it should be possible to use a persistence-capable object as a primary key field. We'll hopefully be working on this in a near-future version of the spec.

    -Patrick

    --
    Patrick Linskey
    http://solarmetric.com
  68. identifying relationships[ Go to top ]

    Also, I agree with your sentiment -- I think that it should be possible to use a persistence-capable object as a primary key field.
    This is an interesting design issue. I agree with Christian Sell that the case he describes should have first class support in JDO, now that we are specifying object/relational mappings. It seems limited enough in scope to try to get it into 2.0, given how common this mapping is. I also agree with Patrick Linskey that JDO should allow a persistence-capable object as a primary key field to support this case.

    But originally this thread discussed the fields in the ObjectId class. I think Robin mentioned as a possibility (without actually advocating it) that the ObjectId class itself could have a persistence-capable object as a field. I would have some concerns about having such a "heavyweight" key. It would be nice to be able to construct a key without necessarily constructing or retrieving an object instance. What do people think about using the ObjectId class of the parent as a member of the ObjectId class of the child (while still allowing the type of child's key field to be the parent class)?

    Daniel O'Connor
    http://www.mvcsoft.com
  69. identifying relationships[ Go to top ]

    Hi Daniel
    ...the ObjectId class itself could have a persistence-capable object as a field. I would have some concerns about having such a "heavyweight" key. It would be nice to be able to construct a key without necessarily constructing or retrieving an object instance. What do people think about using the ObjectId class of the parent as a member of the ObjectId class of the child (while still allowing the type of child's key field to be the parent class)?
    Thats an excellent idea. Please post it to the expert group. We could even extend the idea a bit to allow the primitive "foreign key" field in the objectid-class and a normal reference with the same name in the class itself.

    Cheers
    David
    www.jdogenie.com
  70. identifying relationships[ Go to top ]

    It seems limited enough in scope to try to get it into 2.0, given how common this mapping is.
    let me emphasize again that the underlying issue is not about mapping, and does not arise from the context of relational databases per se (except that RDBMS are the only systems that explicitly model it). It is a conceptual decision whether you allow identifying members of a PC class to be references to another PC class.

    Other than that, I really like the suggestion of having the parent key class as a member of the subordinate key class, and the relationship mapped redundantly in the subordinate class. Sounds perfectly clean to me.

    Christian
  71. identifying relationships[ Go to top ]

    From my earlier post:
    1. A field of type Order (or more likely OrderPK)
    2. A LineNumber field.
    The easiest mechanism to conceptualise is the ChilePK having a reference of type ParentPK. Howevere this restricts the identifying capability to Application Identity - indeed BOTH classes must use Application Identity.

    So as long as we're cool with that, I see no reason not to promote this to the Expert Group.

    Cheers, Robin.

    P.S. I spoke with the JCP last night and download stats indicate the JDO 2.0 Early Draft spec if figuratively flying off the shelves!
  72. identifying relationships[ Go to top ]

    So, to sum up in an example, I understand the consensus to be:

    /** parent class */
    public class Order {
      public static class OrderKey {
         int number;
      }
      int number;
      List lineItems;
    }

    /** child class whose identity is partly defined by the parent */
    public class LineItem {
      public static class LineItemKey {
         OrderKey orderKey;
         int number;
      }
      Order order; //key field
      int number; // key field
    }

    I think limiting this to application identity is ok.

    cheers,
    Christian
  73. identifying relationships[ Go to top ]

    + 1 to the Christian Sell code.
  74. identifying relationships[ Go to top ]

    So an Object ID class for Application Identity can contain simple fields as before, and also single-valued references to instances of Object ID classes of other persistent classes which also have Applicatio Identity, and that such references must be non-cyclic.

    Christian's example shows the Child class having a field of type Parent, instead of the Child class enumerating the simple primary key fields of the ParentPK. Christian's solution is a good one from a transparency/usability perspective. We need to be sure that we can map the results correctly.

    Conceptually, viewing everything from the perspective of the persistent class instead of its Object ID class;

    For Application Identity: If a primary-key field of persistent class X is a single-valued field of type Y, where Y is a persistent class which is different from X, then the Object ID class of X must have a single-valued reference to the Object ID class of Y, which uses the same field name as the primary-key field.

    So I'd alter Christian's sample code as follows:

    /** child class whose identity is partly defined by the parent */
    public class LineItem {
      public static class LineItemKey {
         OrderKey order;
         int number;
      }
      Order order; //key field
      int number; // key field
    }
  75. identifying relationships[ Go to top ]

    From my earlier post:
    1. A field of type Order (or more likely OrderPK)2. A LineNumber field.
    The easiest mechanism to conceptualise is the ChilePK having a reference of type ParentPK. Howevere this restricts the identifying capability to Application Identity - indeed BOTH classes must use Application Identity.
    Robin, why do you say both classes have to use application identity? I think it would be perfectly valid that the child's application identity class has a field of type Object to hold the parent's datastore identity object. This means that an instance of that application identity class will no longer be portable among different vendors' PMFs at runtime, but I think that is unavoidable, and a small price to pay.
  76. identifying relationships[ Go to top ]

    Robin, why do you say both classes have to use application identity? I think it would be perfectly valid that the child's application identity class has a field of type Object to hold the parent's datastore identity object.
    The expert group actually discussed this quite a while ago (post-JDO 1.0, pre-JDO 2.0 kick off) and we had actually only considered this to be the case for application identity. Your point, Abe, is well taken.

    A dependent object id that uses another object id class of type Object or a custom class, is necessarily a form of application identity. However, the object id class being depended upon could be a datastore or application identity class.

    I seem to recall (but I could be wrong) that we discussed adding this in JDO 1.0.1, but concluded that JDO 1.0.1 was to be a small maintenance release to JDO 1.0 (whose difference was a mere 0.0.1 of a spec :) and that adding dependent application identity classes was too much change in a maintenance release. I guess we just forgot about revisiting that particular topic, given all of the other new features in JDO 2.0.

    Thanks for reminding us of this one, Christian!

    --matthew
  77. Review Period Extended[ Go to top ]

    It transpires that the JCP can accommodate a review period of exactly 30, 45, 60 or 90 days, but not the 38 days to 31 July which I had proposed. Therefore the review period for the JDO 2.0 Early Release has been extended by 7 days.

    The review period will now close on 7 August 2004.

    We've already received comments on the spec to the comments alias (jsr-243-comments at jcp dot org). Please keep these coming in. We will acknowledge each comment we receive, and endeavour to provide detailed responses in a timely manner.

    Kind regards, Robin.
  78. Congratulations Robin! I know this has been a lot of hard work with your other real-life responsibilities. I am sure that work has paid off with a valuable contribution to the Java community!