Ted Neward explains ORM as "The Viet Nam of Computer Science"

Discussions

News: Ted Neward explains ORM as "The Viet Nam of Computer Science"

  1. Ted Neward once described object/relational mapping as the "Viet Nam of Computer Science," which was an incredibly effective soundbite, if not completely clearly understood by all who read it or heard about it. In this long and exhaustive blog entry, Ted explains everything behind what he meant - and what's wrong with O/RM. He starts off with a fairly concise explanation of what exactly happened in Viet Nam, starting with events in World War II, and including the effect on American politics after the conflict ended (although his explanation of the victories and losses is a little simplistic, because the North Vietnamese successfully maneuvered both the French and the American forces into siege situations, and won - even at catastrophic costs to themselves, with incredible political ramifications.) Ted says that the US military in Viet Nam was "faced with a deadly form of the Law of Diminishing Returns. In the case of automated Object/Relational Mapping, it's the same concern--that early successes yield a commitment to use O/R-M in places where success becomes more elusive, and over time, isn't a success at all due to the overhead of time and energy required to support it through all possible use-cases." Not doing entire justice to what Ted wrote, a quick summary is that O/RM does a good job up to a point, and then efforts to manage the remaining object/relational mapping tasks takes more effort than they're worth - but, having committed to using an ORM tool, users and implementers feel like they have to go all the way rather than simply saying that there are situations in which the tools aren't appropriate. (Again, in all fairness, your Humble Editor has witnessed cases where advocates of O/RM have clearly and plainly said that their tools were not appropriate for some uses, which indicates that Ted is using generalizations covering implementers and users, not necessarily indicting all users of O/RM.) Ted also goes through a number of techniques in common use in O/RM, covering query-by-example (showing very simple requirements can hurt QBE), the object/relational mismatch, duplication of metadata, identity problems, and more. Ted finally builds a list of responses the community at large can make (with some edits for clarity):
    Several possible solutions present themselves to the O/R-M problem, some requiring some kind of "global" action by the community as a whole, some more approachable to development teams "in the trenches":
    • Abandonment. Developers simply give up on objects entirely, and return to a programming model that doesn't create the object/relational impedance mismatch. While distasteful, in certain scenarios an object-oriented approach creates more overhead than it saves, and the ROI simply isn't there to justify the cost of creating a rich domain model. This eliminates the problem quite neatly, because if there are no objects, there is no impedance mismatch.
    • Wholehearted acceptance. Developers simply give up on relational storage entirely, and use a storage model that fits the way their languages of choice look at the world. Object-storage systems, such as the db4o project, solve the problem neatly by storing objects directly to disk, eliminating many (but not all) of the aforementioned issues; there is no "second schema", for example, because the only schema used is that of the object definitions themselves. While many DBAs will faint dead away at the thought, in an increasingly service-oriented world, which eschews the idea of direct data access but instead requires all access go through the service gateway thus encapsulating the storage mechanism away from prying eyes, it becomes entirely feasible to imagine developers storing data in a form that's much easier for them to use, rather than DBAs.
    • Manual mapping. Developers simply accept that it's not such a hard problem to solve manually after all, and write straight relational-access code to return relations to the language, access the tuples, and populate objects as necessary. In many cases, this code might even be automatically generated by a tool examining database metadata, eliminating some of the principal criticism of this approach (that being, "It's too much code to write and maintain").
    • Acceptance of O/R-M limitations. Developers simply accept that there is no way to efficiently and easily close the loop on the O/R mismatch, and use an O/R-M to solve 80% (or 50% or 95%, or whatever percentage seems appropriate) of the problem and make use of SQL and relational-based access (such as "raw" JDBC or ADO.NET) to carry them past those areas where an O/R-M would create problems. Doing so carries its own fair share of risks, however, as developers using an O/R-M must be aware of any caching the O/R-M solution does within it, because the "raw" relational access will clearly not be able to take advantage of that caching layer.
    • Integration of relational concepts into the languages. Developers simply accept that this is a problem that should be solved by the language, not by a library or framework. For the last decade or more, the emphasis on solutions to the O/R problem have focused on trying to bring objects closer to the database, so that developers can focus exclusively on programming in a single paradigm (that paradigm being, of course, objects). Over the last several years, however, interest in "scripting" languages with far stronger set and list support, like Ruby, has sparked the idea that perhaps another solution is appropriate: bring relational concepts (which, at heart, are set-based) into mainstream programming languages, making it easier to bridge the gap between "sets" and "objects". Work in this space has thus far been limited, constrained mostly to research projects and/or "fringe" languages, but several interesting efforts are gaining visibility within the community, such as functional/object hybrid languages like Scala or F#, as well as direct integration into traditional O-O languages, such as the LINQ project from Microsoft for C# and Visual Basic. One such effort that failed, unfortunately, was the SQL/J strategy; even there, the approach was limited, not seeking to incorporate sets into Java, but simply allow for embedded SQL calls to be preprocessed and translated into JDBC code by a translator.
    • Integration of relational concepts into frameworks. Developers simply accept that this problem is solvable, but only with a change of perspective. Instead of relying on language or library designers to solve this problem, developers take a different view of "objects" that is more relational in nature, building domain frameworks that are more directly built around relational constructs. For example, instead of creating a Person class that holds its instance data directly in fields inside the object, developers create a Person class that holds its instance data in a RowSet (Java) or DataSet (C#) instance, which can be assembled with other RowSets/DataSets into an easy-to-ship block of data for update against the database, or unpacked from the database into the individual objects.
    What do you think? Was Ted right in that O/RM is a quagmire for computer science in the first place? Has he accurately summarized the problems and possible solutions?

    Threaded Messages (152)

  2. ...hot & beautiful.[ Go to top ]

    ORM, The Viet Nam of Computer Science: hot & beautiful?
  3. Re: ...hot & beautiful.[ Go to top ]

    ORM, The Viet Nam of Computer Science:
    hot & beautiful?
    Yeah, weird analogy. Wonder what the Iraq War of Computer Science is and what "Mission Accomplished" would represent? User interface technology? ______________ George Coller DevilElephant
  4. What do you think? Was Ted right in that O/RM is a quagmire for computer science in the first place? Has he accurately summarized the problems and possible solutions?
    I think Ted needs to turn off the computer tonight and go have a beer.
  5. Ted. What ever you're smoking. Pass the dube' to the left.
  6. Yeah, 'cause those on the right will never admit to inhaling anyway. :)
  7. Wow Ted! I think this is the most interesting text I have read in the past few months! This is the way of thinking of a topic: by trying to find similarities in a seemingly completely unrelated area! Everything is interconnected in the web of life folks... PS: By the way Ted talks sense :)
  8. While I agree with you about the beauty and power of metaphors in general I'm not so sure this one is especially relevant. Ted's hypothesis: "ORM is like Vietnam". So from a US-centric point of view: "Vietnam was a disaster" => so ORM is a bad thing. But let's take a differen point of view, a far-east or asian one for instance: "Vietnam was a victory against US imperialism" => thus ORM is a symbol of freedom ??? IMHO: a metaphor based on a point of view cannot be shared by everybody and thus is less interesting.
  9. Re: that is the way to think![ Go to top ]

    So from a US-centric point of view:
    "Vietnam was a disaster" => so ORM is a bad thing.

    But let's take a differen point of view, a far-east or asian one for instance:
    "Vietnam was a victory against US imperialism" => thus ORM is a symbol of freedom ???

    IMHO: a metaphor based on a point of view cannot be shared by everybody and thus is less interesting.
    I wondered if anyone would realize this. PJ Murray, CodeFutures Software Data Access Objects and Service Data Objects
  10. Yeah, some symbol of freedom[ Go to top ]

    While I agree with you about the beauty and power of metaphors in general I'm not so sure this one is especially relevant.

    Ted's hypothesis: "ORM is like Vietnam".

    So from a US-centric point of view:
    "Vietnam was a disaster" => so ORM is a bad thing.

    But let's take a differen point of view, a far-east or asian one for instance:
    "Vietnam was a victory against US imperialism" => thus ORM is a symbol of freedom ???

    IMHO: a metaphor based on a point of view cannot be shared by everybody and thus is less interesting.
    Two million people were murdered and tens of millions of more condemned to live in oppression and poverty for decades in the aftermath of the U.S. withdrawal from Vietnam. From just exactly who's point of view was that a good thing?
  11. In my experience, mannual maping did better job then using an ORM tool. In my last 4 projects, I used mannual mapping on 3, and ORM tool (Hibernate) on one. I am much more satisfied with mannual mapping then using Hibernate (performance, manitainability, even development productivity in later stages of development), especialy when working on very complex/large projects. I am now working on .NET project and applying mannual ORM mapping, using all techniques I used in Java projects, and things are going smoothly for now (although JDBC is much better then ADO.NET when using ORM :) ).
  12. very complex/large projects[ Go to top ]

    especialy when working on very complex/large projects
    Wow. I just posted about this on another TSS thread: http://www.theserverside.com/news/thread.tss?thread_id=41091#212260 I'm working on a (thus-far 5 year) project with 857 tables, and Hibernate saved our butts. I wonder how that compares to your experience of "very complex/large projects." And no - I don't work for JBoss.
  13. Re: very complex/large projects[ Go to top ]

    I'm working on a (thus-far 5 year) project with 857 tables, and Hibernate saved our butts. I wonder how that compares to your experience of "very complex/large projects."
    I wonder how many times you get to go through wall fighting with Hibernate in your project with 857 tables? Has Hibarnate been able to support all your mapping needs? Was there times you were not satisfied with the way Hibernate handled specific mapping problems? Are you satisfied with the way Hibernate handle caching? Especialy second level cache which stores objects in name/value maps? What about second level cache memory consumptions? P.S. This is not only about Hibernate, I am sure other ORM tools have their own quirks, even more then Hibernate.
  14. Re: very complex/large projects[ Go to top ]

    I wonder how many times you get to go through wall fighting with Hibernate in your project with 857 tables? Has Hibarnate been able to support all your mapping needs? Was there times you were not satisfied with the way Hibernate handled specific mapping problems? Are you satisfied with the way Hibernate handle caching? Especialy second level cache which stores objects in name/value maps? What about second level cache memory consumptions?

    P.S. This is not only about Hibernate, I am sure other ORM tools have their own quirks, even more then Hibernate.
    Hibernate has done very well for us. Most all trouble we've had has been our own fault. (Not directed at you,) I can't believe anyone would want to maintain raw sql/jdbc code for the amound of data access we have to deal with. To address your caching questions, believe it or not, we currently do not employ a second level cache.
  15. Re: very complex/large projects[ Go to top ]

    To address your caching questions, believe it or not, we currently do not employ a second level cache.
    Do your app use single Hibernate session? Is it a desktop app? Are you using long sessions? If not, I wonder how you do not experience performance problems? Using ORM without caching is totaly wrong for me. One of the greatest benefits of ORM is that it enables easy caching of complex relationships between the entities (when compared with how complex can be caching and cache invalidation in data set oriented designs).
  16. Re: very complex/large projects[ Go to top ]

    I can't go into too many details, but there is a fat Swing client and a JBoss server. We do not use long sessions; requests to the server are stateless. We are dealing with performance problems, but not because of Hibernate. They are usually because of poor programming (loading/transporting more data than necessary) by members of a very large development team. It is being addressed though. I'm glad that caching is working out for you. We might end up using it, but it's not on the short-term radar. Caching is not always what it's cracked up to be, though. It brings complexity to the table that many people naively don't realize. (Not directed at you.)
  17. Using ORM without caching is totaly wrong for me. One of the greatest benefits of ORM is that it enables easy caching of complex relationships between the entities (when compared with how complex can be caching and cache invalidation in data set oriented designs).
    My 2 bits: on a large application (about 500 users working 8 hours a day, mission critical, 3 tiered, business tier Hibernate+Spring+SLSBs) we finally got performance to acceptable levels by turning off second-level Hibernate cache. It didn't help much in the first place anyways, as access patterns that were really slow used HQL or had 1+N problems or trashed the cache. Also, having all that cached objects on the heap slowed GC a lot.
  18. +1 I come back to jdbc after losing time and effort maiking orm work, like all generic tools it never fit your context, making another comparison, orm is like fast food, when jdbc is like home cooking, of course jdbc is not for newbies, like cooking :)
  19. In my experience, mannual maping did better job then using an ORM tool.

    In my last 4 projects, I used mannual mapping on 3, and ORM tool (Hibernate) on one.
    Could you clarify please what advantages exactly your manual mapping brought to the projects and why do you think maintainers of your past projects enjoy supporting them more than projects which use widely accepted persistence frameworks? And why you did not you use iBatis anyway?
  20. wide acc[ Go to top ]

    widely accepted persistence frameworks?
    I think that "wide acceptance" is a bit of a myth...
  21. And why you did not you use iBatis anyway?
    Here here on the iBatis selection. I became disenchanted with Hibernate when I needed to have a more flexibility in the SQL and the child object population. I found iBatis to be a great comprimise. But iBatis isn't the best solution either. One thing I miss from Hibernate that isn't available to iBatis is Polymorphic querying. So even in ith ORM space, there are decisions to be made. John Murray Sobetech
  22. The Object-to-Table Mapping Problem Ted's objections seem to be a proliferation of tables in the database and the accompanying JOINs that will be required for polymorphic queries when using the table per class paradigm, and the inability to enforce nullability constraints when using the table per class hierarchy paradigm. Neither of these problems are specific to OR mapping - you would face the same decisions regarding schema structure regardless of your choice data access implementation. The Dual-Schema Problem Ted objects to holding metadata in two different places. This is amusing because I've seen a tech talk with Ted (I think here of TSS) where he shrugs off such concerns. I think the example he used was the maxlength attribute of a web form input tag and the same constraint defined in the DDL. More importantly, again, this problem isn't specific to OR mapping. Entity Identity Issues Ted's objection here is essentially that OR mappers can't guarantee object identity outside the scope of the persistence context. True enough, but this certainly isn't an intractable problem, simply one that requires awareness. The Data Retrieval Mechansim Concern Ted's basic objection here is that no signle query mechanism by itself is ideal. That's why a good ORM solution like Hibernate offers all three. The Partial-Object Problem and the Load-Time Paradox In practice I haven't found this to be a serious problem. Ted makes some assertions here that aren't quite accurate I think. "an O/R mapping has to make some up-front decisions about when to retrieve these associated objects...Most O/R-M's offer some kind of developer-driven decision-making support, usually some kind of configuration or mapping file, to identify exactly what kind of retrieval policy will be, but this setting is global to the class, and as such can't be changed on a situational basis." This is just plain wrong. Of course retrieval can be altered at runtime. But the most important point to remember in all of this is that ORM is not an all or nothing choice. Use ORM for the easy 95% of cases and fall back to raw SQL for that 5% of problems where ORM doesn't fit. This is where Ted's analogy falls apart.
  23. But the most important point to remember in all of this is that ORM is not an all or nothing choice. Use ORM for the easy 95% of cases and fall back to raw SQL for that 5% of problems where ORM doesn't fit. This is where Ted's analogy falls apart.
    Going that way is usually where the whole project falls apart and Ted's analogy really starts to shine. 0. As Ted mentioned, caching screws you up royaly (truth be told, caching usually screws you even with a 100% pure ORM approach). 1. Your ORM supporting senior developer starts whining that the thing is "not OO anymore", "not elegant", "not done right", "impure", "the way we did it in COBOL". Usually accompanied by doing that 5% with ORM, no matter the cost. 2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM. So they do it their way. To the frustration of those that don't know the non-ORM approach. 3. Suddenly, you have to think about every little feature wheter it falls into 95% or 5% part. 4. Much fun when you have to make your ORM Person play along with non-ORM AddressLine. Before you notice, you have your little Tet offensive (er, developer revolt) and a year you're throwing helicopters (er, Hibernate) overboard your bloated carrier (er, enterprisy application), running away from commies (er, customers).
  24. caching???[ Go to top ]

    0. As Ted mentioned, caching screws you up royaly (truth be told, caching usually screws you even with a 100% pure ORM approach).
    I don't known how you guys configure caching and either how do you "screw up". This is not a difficult problem. Ted seems to have experience with a retarded cache system that doesn't support write through. And what is your concern about caching even without a 100% ORM approach?
  25. Re: caching???[ Go to top ]

    Ted seems to have experience with a retarded cache system that doesn't support write through. And what is your concern about caching even without a 100% ORM approach?
    For starters, you're kinda giving up half of the caching benefits with write-through by caching only reads ;) 1. Invalidating objects in cache when using non-ORM approach. 2. Invalidating objects in cache when their underlying data has been changed outside of the application (Always happens in production. If it doesn't, your data is not really usefull to the others.) 3. Cache coherency across cluster (cue Cameron). 4. Choosing how much to cache (no, WeakHashMap is not the whole answer). 5. Silly applications which thrash the cache by putting in 50.000 objects at once, which they later access only once. 6. Bloody bugs in oscache, like locking up on a wait() that for some reason never gets notified() (easiest to fix of them all). And there is the "session cache" (to use the Hibernate's terminology), which basically turns figuring out which piece of code violated DB constrains from looking at the stack trace into a mildly amusing detective work. On the plus side, you can book that as code reviews and post really good stuff on thedailywtf.com. It's rather evident which problems can happen with any approach.
  26. 2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM.
    I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.
  27. 2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM.


    I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.
    Unless, of course, you are David Copperfield/Harry Potter/... .
  28. If I had a hammer....[ Go to top ]

    2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM.


    I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.

    Unless, of course, you are David Copperfield/Harry Potter/... .
    I think those who learned their development chops coming from a strong database background are more leery of Hibernate than those who come from a coding background. I spent a small eternity writing financial applications so for me writing SQL is as natural as breathing; it is my "hammer" for better or worse. Letting some framework generate the code for me feels wrong since everything I know how to quickly debug and tune is hidden from plain view. Are my queries optimized, are they hitting indexes, can I paste them in TOAD to try them out? Unless I have a pretty deep object model, which seems to be never for the corporate apps I work on, I prefer a lighter framework like iBatis or Spring's SQL templates. They take a lot of the JDBC load off my shoulders without limiting something I'm good at- SQL. Now I may feel different if I had the 800+ tables that one poster talked about. Sheesh. I'm not sure how this post plays into the Nam analogy. ______________ George Coller DevilElephant
  29. Re: If I had a hammer....[ Go to top ]

    I think those who learned their development chops coming from a strong database background
    I am one of those people.
    are more leery of Hibernate than those who come from a coding background.
    I would say, on average, this is true.
    I spent a small eternity writing financial applications so for me writing SQL is as natural as breathing; it is my "hammer" for better or worse.

    Letting some framework generate the code for me feels wrong since everything I know how to quickly debug and tune is hidden from plain view.
    Sure. Knowing what is going on is good. But do you use low level libraries for GUI work? At some point we need to get solving the business needs.
    Are my queries optimized
    With my ORM - Mine are.
    , are they hitting indexes,
    With my ORM - Mine are.
    can I paste them in TOAD to try them out?
    With my ORM - I can.


    Unless I have a pretty deep object model, which seems to be never for the corporate apps I work on, I prefer a lighter framework like iBatis or Spring's SQL templates.
    Which is really funny in light of what others are claiming here (ORM is bad with deep model/lots of objects). Like I said before, small or large, (N)Hibernate lets me concetrate on problem areas.
    They take a lot of the JDBC load off my shoulders without limiting something I'm good at- SQL.

    You can do sql with Hibernate. And other ORMS. I am good at SQL too. But I despise repetitive tasks. I would rather be going to the bank that typing DML.
  30. Won't quote your response but everything you said I'd say is pretty fair. I did say "for better or worse". To be fair to Hibernate (or Toplink, or similar) I think that it just hasn't sunk in right yet in the projects I've been associated with. I personally haven't seen it used well yet. I've seen it abused. I've also seen it used on small 5-table projects where I scratch my head and say why? I've seen it used and have all the queries be pass-through to a stored procedure. So maybe there just aren't that many experts yet and that is hurting ORM's reputation? Where is the Hibernate sweet spot? When does it and its dependencies and configuration overhead start overcoming what the slimmer frameworks or naked JDBC can give you? For that matter, when does losing instant database portability take precedence over using vender-specific query enhancers? I've read most of the Manning Hibernate book and dinked around with it to the point of being semi-proficient. (BTW: ORM is a boring book to read if you don't have an immediate application :-) I can think of use cases where I think Hibernate would be pretty handy - just haven't run into them yet. Or, to be more fair, I've probably run into situations where I could have gone either way and chose the quick and easy road for me and my team. If I had an experienced Hibernater on my team maybe I'd have let him talk me into it. I'd have done it just to get the practical experience for myself. Is Hibernate just too new? Is it over-hyped like EJB 2.0 where some people are using it on every project regardless of its downsides? Are there too many other "good enough" technologies? I'd say probably a little of all that is true. From what I can see it is a solid technology and is well documented. Most of the performance problems I've run into ended up being programmer mistakes. ______________ George Coller DevilElephant
  31. 2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM.


    I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.

    Unless, of course, you are David Copperfield/Harry Potter/... .
    The Nam analogy was confusing enough. Now you are bringing in magic? (I had to stop myself writing 'but anyway, David Copperfield isn't a real wizard, unlike Harry Potter'.....).
  32. 2. Developers that have to do that 5% soon discover that doing that 95% with non-ORM approach is as easy (or sometimes even easier) as with ORM.


    I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.

    Unless, of course, you are David Copperfield/Harry Potter/... .


    The Nam analogy was confusing enough. Now you are bringing in magic? (I had to stop myself writing 'but anyway, David Copperfield isn't a real wizard, unlike Harry Potter'.....).
    lol. True. :) Sadly we don't have our own wizard so we gotta borrow Harry. And even then, you all got the book and we got the movies. And we all know the books are better than the movies.
  33. Sadly we don't have our own wizard so we gotta borrow Harry. And even then, you all got the book and we got the movies. And we all know the books are better than the movies.
    Well, certainly longer, anyway..... here in the UK there is a suspicion that Rowling's books are part of a fitness program for kids - they will need to train in order to lift the next volume back from the bookshop....
  34. Sql on top of POJO[ Go to top ]

    What puzzles me endlessly is how people desregard what is simple and obvious in exchange for arcane and challenging. That's a general observation. Why do we resort to RDBMS and ORM and not have a object database (such as db4o or better) with an sql interface (such as JoSQL or better) to satisfy both the programmesr and the sql tools with their the sql users? First, programmers are back to their beloved POJO's. Period. That's what matters, right! Second, the second class data users have SQL tools for their delight. Why is this wrong? It's the data that matters - why does it have to be relational and not life immitating, i.e. object oriented?
  35. Re: Sql on top of POJO[ Go to top ]

    What puzzles me endlessly is how people desregard what is simple and obvious in exchange for arcane and challenging. That's a general observation.

    Why do we resort to RDBMS and ORM and not have a object database (such as db4o or better) with an sql interface (such as JoSQL or better) to satisfy both the programmesr and the sql tools with their the sql users?

    First, programmers are back to their beloved POJO's. Period. That's what matters, right! Second, the second class data users have SQL tools for their delight.

    Why is this wrong? It's the data that matters - why does it have to be relational and not life immitating, i.e. object oriented?
    I'm not disagreeing with you but am offering a corporate point of view. Relational technology is rock-solid, fast, and mature. Most of the hard problems with building an enterprise-scale database have been solved. (Somehow I feel that an architect working for Oracle would want to slap the back of my head for saying that) I'm talking generally here - of course there are always improvements to be made. But I think the success of an open source database like MySQL shows that much of what you need to get that kind of a project off the ground is out there. There are books, papers, and skilled professionals available. I think db4o or r2d2 or whatever would need to prove itself on a grand scale or be adopted by IBM or Red Hat before it would make it through the conservative gate keepers at most corps. I'm also wondering how good the SQL support would be for an object database? I'm not trying to be overly negative. I'd just need to see the success stories or big-company backing of the technology before I'd stick my neck out on it. I went to the db4o site and it does look professional - impressive. But right on their site they talk about companies that have embedded db4o into their product. http://www.db4o.com/about/customers/ To me there is a big difference from an embedded database and a enterprise database that needs to be shared across several applications and business processes. ______________ George Coller DevilElephant
  36. The point is not db4o or JoSql but the idea that it's time to make the Sql users second class to data access and not the business users of the data, i.e. application developers. Relational databases were the invention of the day, THEN. Now, however, we can move on. The thing holding back is not "robust, enterprise, scaleability" etc. but the universality of SQL. Now, we do have the understanding to build all those features in a object database that can hold terabytes safely; it's not "proven" because of one word: inertia. Everybody IS going to the bank(!) rather than inovate. It's the gas to fuel cell analogy: rdbms is making us money, why bother.
  37. Relational databases were the invention of the day, THEN. Now, however, we can move on. The thing holding back is not "robust, enterprise, scaleability" etc. but the universality of SQL. Now, we do have the understanding to build all those features in a object database that can hold terabytes safely; it's not "proven" because of one word: inertia. Everybody IS going to the bank(!) rather than inovate. It's the gas to fuel cell analogy: rdbms is making us money, why bother.
    My God, this will look lika a bad Date/Pascal clone, but the quoted text is soooo asking for it. OK, here we go: Using that reasoning, we should abandon OO, too. Simula has been standardized in 1968. Codd wrote "A Relational Model of Data for Large Shared Data Banks" in 1970. Should we also move away from the while and for statements? They were also invented THEN, even before relational and OO stuff. Surely we can come with something brand new, less constraining for our boundless programming talents? Imagine if instead of calling a method you could call a statement directy. We could even call this brand new thing "executeTo" ... Or what about math? Crufty Greek invention, thousands of years beyond its sell by date.
  38. It's time to understand[ Go to top ]

    Relational databases were the invention of the day, THEN. Now, however, we can move on.
    Move back to the 1970's you mean? First there was flat files (usually on tape - early on papertape and punched cards). Then someone got the idea of hierarchical and network type databases. Essentially OO databases of their time, but severely limited by procedural programming. That was until about 1970. Then Codd showed the problems with the old approach and showed us all a new one - the relational approach. Now some people want to go back to the pre 1970 era and start using the old network approach again. OO databases are the same old stuff just reinvented by someone who don't really know, or if they do know they choose to forget. They recreate all the old problems and solve few new ones. The so-called Object Relational impedance mismatch is a severely overhyped tiny problem which is solved either by an ORM tool for some or a good database access library on top of JDBC. It could have been even better supported with better access methods inside the RDBMS engines, but even the RDBMS engine developers have problems understanding they could have used pointers inside the engine instead of insisting on almost nothing but B-tree indexed access for joins. Most programmers of today see only objects, objects, objects and have learned little or nothing about large queries, mass updates and so on. There is also very little differentiation between legacy database development with badly designed databases and development where the database is also designed correctly. Of course many developers are stuck with badly designed databases. They may require specialized tools for the purpose. We that design our databases as well need better tools for our different purposes. Those are the really interesting tools to discuss as they show off what the right technology is. Of course that is some kind of ORM tool, but that also gives you direct relational access and have more relational thinking than is typically available now.
  39. Re: It's time to understand[ Go to top ]

    Good points, Nils, but would you care to elaborate on what you mean with...
    It could have been even better supported with better access methods inside the RDBMS engines, but even the RDBMS engine developers have problems understanding they could have used pointers inside the engine instead of insisting on almost nothing but B-tree indexed access for joins.
    ...just a little bit more?
  40. Re: It's time to understand[ Go to top ]

    Good points, Nils, but would you care to elaborate on what you mean with...

    It could have been even better supported with better access methods inside the RDBMS engines, but even the RDBMS engine developers have problems understanding they could have used pointers inside the engine instead of insisting on almost nothing but B-tree indexed access for joins.


    ...just a little bit more?
    Everyone seem to want "fast" access to a whole object graph. To do that from a relational database you can do a join between several tables, possibly many with outer joins to get the whole thing at one time. The framework will read all the rows for you and fill the appropriate objects with data and correct references between them. People assume that read has to be done via first a read of a record from the master table. Then use the join field data to do an indexed access to the next table in the join and so on for each table. They see this as a very costly affair. Then they claim that what they call an "object oriented database" can read all data at ones, possibly using internal pointers in the database to get at the data fast. There is however nothing stopping a relational database management system to also insert pointers inside it's internal structure of data to get at joined rows with the same speed. It could read a row from the master table, there find the internal pointer to a disk location for a joined row and read that directly. An outer join with no more data would see a null pointer and wouldn't have to read any more data at all. It would be faster than a join with data. The database engine could also colocate data for often-used access methods so it could read directly from one or a few disk blocks/pages a whole object graph. Other smart access methods could also be invented for similar purposes. But the main idea here isn't that pointers, colocation or any other "smart" access method is needed in a relational database system. The point is it can be implemented if someone find out it is a good idea. You can with other words get all the benefits people are talking about with object oriented "databases" while not loosing the required benefits of a relational database. People should take a little time out to study what those benefits are, and they would immediately stop talking so much about oo "databases" for anything but very specialized data if even for that.
  41. Hi Nils, although I do agree in general, there a few simplifications to avoid. You are comparing OODBMS implementations with the RDBMS concept. There has also been a lot of research on concepts for OO databases, too. But unlike with RDBMSs many concepts have not really made it into products. Often simple features like Joins in queries are missing. But that is not the point here. The point is, that on implementation layer the differences between RBBMS and OODBMS vanish. The API concept does not mandate the data structures and algorithms used, in German we say: "They all cook with water." A few differences remain: -Class hierarchies and indexes that span a class hierachy -Non-first normal form (NFNF) both aspects of the impedance mismatch. The later one raises problems with RDBMSs, because the entity you work with (Java object) is not the same entity on DB level (multiple rows spread over different tables if mapping a collection). This leads to anomalies e.g. wrt. concurreny control, like that an object is only half-locked or similiar. Christian
  42. Let's accept that close coupling of procedural code and data is a big mistake in any enterprise architecture (it may be workable for shrink wrap apps though). Code and data have very different life cycles and very different change management requirements. Hiding constraints/rules inside a Java class makes it almost inaccessible to code written in other languages. OO databases are leading straight into the very data silo issues that are the biggest problem of most enterprises today. Mixing code and data at the data layer is like hard-coding business rules into UI widgets. There used to be an initiative (naked objects?) that said let's have objects display themselves. Same bad idea. Some things need to be decoupled even if it means more work at one point in time. And, by the way, object orientation and the relational model are roughly the same age. In contrast to the relational model, though, OO doesn't have anything like a sound theoretical foundation. That's one more reason why using objects for representing long lived information, that is accessed from various languages and tools, is a bad idea. There is no reliable way to figure out whether a particular method call in Java has the same effect as a similar call to a C++ member function (other than executing both of them). The same goes for inheritance, etc. You can't have this kind of fluffy semantics in a knowledge representation context. For that very reason OO databases are marketed almost exclusively as embedded databases. They are embedded into one particular application because that's the only way you get any kind of consistency. And it's the only situation where hard coded access paths make sense. There is no cross language OO model and that's an important part of why OO databases are a failure. The other one being that mixing code and data (in the data layer) is a bad idea in the first place.
  43. There used to be an initiative (naked objects?) that said let's have objects display themselves. Same bad idea.
    It wasn't so bad. And it didn't have the objects display themselves. And it isn't "use to be". It has continued on. And there are some new, interesting incarnations. Google JMatter and Essential.
  44. Let's accept that close coupling of procedural code and data is a big mistake in any enterprise architecture (it may be workable for shrink wrap apps though). Code and data have very different life cycles and very different change management requirements.
    No, at all. I can agree only of you are talking about badly designed SW, but it is another story.
    Hiding constraints/rules inside a Java class makes it almost inaccessible to code written in other languages.
    So better to write DB triggers ? No other way ?
    OO databases are leading straight into the very data silo issues that are the biggest problem of most enterprises today. Mixing code and data at the data layer is like hard-coding business rules into UI widgets.
    What is data layer ? Againg, OO is different from bad OO design.
    There used to be an initiative (naked objects?) that said let's have objects display themselves. Same bad idea. Some things need to be decoupled even if it means more work at one point in time.

    And, by the way, object orientation and the relational model are roughly the same age. In contrast to the relational model, though, OO doesn't have anything like a sound theoretical foundation. That's one more reason why using objects for representing long lived information, that is accessed from various languages and tools, is a bad idea.
    Following your suggestion we end up with simple object consistency check methods in all the possible languages: java, perl, C++.
    There is no reliable way to figure out whether a particular method call in Java has the same effect as a similar call to a C++ member function (other than executing both of them). The same goes for inheritance, etc. You can't have this kind of fluffy semantics in a knowledge representation context.
    Now I understand, use your favourite RDBMS procedural language, right ? Guido
  45. Convincing, Encapsulation[ Go to top ]

    Considering how stubborn I am in preferring OO to relational, the fact that you're arguments have raised doubts in me just means you've got a good argument. But, for me, it isn't so much about OO, its about the encapsulation. OO is just one way to achieve encapsulation. So ia an SOA where the entire entrprise doesn't go against a DB but instead apssed messages around on an ESB to service and the services enforce the busines slogic on the data entering/leaving the various systems of record. Its when people start cheating and going straight to the DB to get data which doesn't belong to them (its part of another system they're now bypassing) that you can get in trouble. And when I bring this argument up, I'm often rebutted with "well, you can use stored procedures to enforce the rules." But then you're just following into the same trap you're arguing against -- putting the data and rules/business-logic together. Instead, I think what's important is making sure that the data is cleanly separable from the rules when the platform the rules are implemented in becomes defunct, or some other grand refactoring is needed (e.g. build a new UI on top of same old DB). An RDBMS makes its data readily avilable using a standard query language that thousands of tolls are ready to support. The same can't be said for OODBMS. But even for an SOA or OO solution, so long as you recognize this requirement, you can build in the abilityt o export the data ot some XML format, or an RDBMS table (e.g. data warehouse) just for the sake of having a copy of the state independent of the rules.
  46. Its when people start cheating and going straight to the DB to get data which doesn't belong to them (its part of another system they're now bypassing) that you can get in trouble. And when I bring this argument up, I'm often rebutted with "well, you can use stored procedures to enforce the rules." But then you're just following into the same trap you're arguing against -- putting the data and rules/business-logic together.
    Why exactly is going straight to the database cheating? Especially if the access is readonly? Defining views to restrict what data a user can see (at table, column, and row levels) is somehow bad? Whatever happened to "do the simplest thing that works?" Why are hundreds or thousands of lines of Java code superior to a couple lines of SQL?
  47. Re: Convincing, Encapsulation[ Go to top ]

    Its when people start cheating and going straight to the DB to get data which doesn't belong to them (its part of another system they're now bypassing) that you can get in trouble. And when I bring this argument up, I'm often rebutted with "well, you can use stored procedures to enforce the rules." But then you're just following into the same trap you're arguing against -- putting the data and rules/business-logic together.


    Why exactly is going straight to the database cheating? Especially if the access is readonly? Defining views to restrict what data a user can see (at table, column, and row levels) is somehow bad?

    Whatever happened to "do the simplest thing that works?"

    Why are hundreds or thousands of lines of Java code superior to a couple lines of SQL?
    I'll like to see the application on which you are working. Unless you are writing a lot of SQL that isn't actually used in any application, be it C++ or Java, you have code to get connections, code to query, code to process the results, and code to close connections. If you know of any language that allows you to bypass all that work and just embed SQL, why don't you let all of us know about it?
  48. Missing the point[ Go to top ]

    I'll like to see the application on which you are working. Unless you are writing a lot of SQL that isn't actually used in any application, be it C++ or Java, you have code to get connections, code to query, code to process the results, and code to close connections.
    I never said you didn't. The previous poster's comment that going straight to the DB is "cheating" sounded like applying rules without thinking. There are some things that can be expressed in SQL far more easily, effectively, and efficiently than they can in Java, C#, C++, Python, Ruby, or any language of the sort. There are some things that are almost impossible to express in SQL yet easily expressed in the previously mentioned languages. Pick what works best based on the requirements.
  49. Re: Convincing, Encapsulation[ Go to top ]

    you have code to get connections, code to query, code to process the results, and code to close connections.

    If you know of any language that allows you to bypass all that work and just embed SQL, why don't you let all of us know about it?
    How does that change for XMLDB/OODB … heck even SOA or data/objects/messages on message bus/ESB (whatever other buzzword you want to use). You have to get connection to resources (DBMS or not), you have to at times narrow down the set of objects you want (query), you have to process the "results" (why else do you program) etc. This argument is irrelevant to simplicity of query languages in context.
    ORM is a bandaid for the impedance mismatch. It is a practical solution for the situation today, but its eternal perpetuation is detrimental.
    What solution do you have in mind?
    If the data (for example, customer records) is going to be needed by multiple applications and will be maintained for very long periods, then it's probably better to adopt the 'data first' approach.
    Maybe, but not probably. There is another approach. It is not used as much because many developers (and DBAs and PMs and ...) have trouble thinking beyond the database and the UI.
    I am truly interested in understanding this approach - please elaborate? SQL for all its quirks and incompatibilities is quite portable unless you get into stored-procs. ORMs will overcome these incompatibilities for us - without sacrificing performance too much. Why do we continue to code at lower levels of abstraction? Why don't we try to standardize query languages for other more commonly occurring enterprise problems (K for array/time-series, some kind of Tree-QL for hierarchical data), preferably without pointy brackets. All the talk of O-R impedance-mismatch is hype. Moreover that is not the only impedance to be overcome while programming. If you use some-one else's SOA service - it can bloody well have very high impedance to your own code / model. Reuse is hard (http://www.cs.cmu.edu/~able/publications/archmismatch-icse17/). I think programming the computers is the biggest impedance mismatch between how humans think and solve problems (which no one understands very well) and how we programmers try to make computers do some work for us.
  50. Re: Convincing, Encapsulation[ Go to top ]

    I am truly interested in understanding this approach - please elaborate?
    You've already mentioned them.
    How does that change for XMLDB/OODB … heck even SOA or data/objects/messages on message bus/ESB (whatever other buzzword you want to use). You have to get connection to resources (DBMS or not), you have to at times narrow down the set of objects you want (query), you have to process the "results" (why else do you program) etc. This argument is irrelevant to simplicity of query languages in context.
    It seems you've misunderstood the context.
  51. Why exactly is going straight to the database cheating? Especially if the access is readonly? Defining views to restrict what data a user can see (at table, column, and row levels) is somehow bad?

    Whatever happened to "do the simplest thing that works?"

    Why are hundreds or thousands of lines of Java code superior to a couple lines of SQL?
    Its something you have to balance. The data stored in the database is not always just simple data. Often there are rules to be used when considering the data. If you buil an application full of logic for manipulating business data, then the rules applied when reading and writing data are in the application. When you use bypass that application to get to the data directly, you run the risk of seeing the data incorrectly. Now, much of the time the data is simple enough that the rules you're bypassing are trivial if non-existent, so doing something like running reports right out of the database is low risk. But you may ultimately find that you have to start re-implementing code lookups, unconstrained joins, etc. in your reporting engine that were already implemented in the application. That's where you start getting into dual maintenence.
  52. Enterprise Data Access[ Go to top ]

    "To me there is a big difference from an embedded database and a enterprise database that needs to be shared across several applications and business processes." But databases shouldn't be shared across applications and business processes. Data should be exposed via services (a.k.a service oriented architecture). Integration should occur at the service layer, not at the database layer. In this scenario it is perfectly reasonable to have an oo database holding your enterprise data. If performance is an issue then you develop strategies to cater for those circumstances. The Object-Relational mismatch is one of the biggest barriers to progress of the software industry and I don't think we should just give up and use the last resort of JDBC / SQL. Nigel Dinsdale
  53. Re: Enterprise Data Access[ Go to top ]

    But databases shouldn't be shared across applications and business processes. Data should be exposed via services (a.k.a service oriented architecture). Integration should occur at the service layer, not at the database layer.
    The application developer has to develop methods for each integration case when integrationg at the service layer. That's much more work that shared database approach, where you only have to limit the data being shared. Another problem with service-oriented approach is that it's not very flexible for data integration. For instance, you can "join" data together only on the client (usually by loading way too much) and you can't avoid 1+N problems without changing the service. Of course, if you're sharing just services, service-oriented integration works perfectly. But you usually want more for internal information systems.
  54. Re: Enterprise Data Access[ Go to top ]

    But databases shouldn't be shared across applications and business processes. Data should be exposed via services (a.k.a service oriented architecture). Integration should occur at the service layer, not at the database layer.

    The application developer has to develop methods for each integration case when integrationg at the service layer. That's much more work that shared database approach, where you only have to limit the data being shared.

    Is it really more work? My experience says otherwise. Doing integration at the database level almost always ends up costing more and taking more time.
  55. Re: Sql on top of POJO[ Go to top ]

    Why is this wrong? It's the data that matters - why does it have to be relational and not life immitating, i.e. object oriented?
    Cause then people would have to learn something new. ( I was actually give that as a reason)
  56. Deja-Vu![ Go to top ]

    This whole discussion reminds of the discussion developers would have had when moving from assembly to high level programming languages. I think the use of ORM is part of the inevitable move to higher level development abstractions (SQL is the assembly language of ORM). Sure we can drop down to assembly/SQL level if needed, but most of our work will be at the OO/ORM level, so we can focus on the problem. As for Relational vs OO (a broad category) database management systems, I think some contexts are better suited to RDBMS and some to OODBMS. Developers have/will find competitive advantage in OODBMS for certain applications, whilst others will be better done with an RDBMS. Cheers, Ashley. -- Ashley Aitken Perth, Western Australia mrhatken at mac dot com
  57. ORM and OODB[ Go to top ]

    I really find it amazing how this ORM-bashing subject keeps coming up from time to time. I think it is abvious that quite many people with good reputations and large projects have had very good experiences - so that says it all IMO. I have seen large projects where people feared the complexity of a new ORM product (which indeed should not be underestimated), and instead went with what they already knew, and what seemed so simple - plain SQL/JDBC (and a few "patterns" from some J2EE book). Serveral months down the road the mess had become completely uncontrollable. as far as OODBMS go, I have personally never seen an area of application where an OODBMS was a better fit (admittedly, I have never seen anyone seriously try it either. They keep saying that CAD apps are the ones, because of their very fine-grained and recursive models). No wonder that almost all OODBMS vendors have gone out of business. Even the OO-features that were once built into the major RDBMSs (UDTs, sometimes even with inheritance) are very rarely used in my experience. christian
  58. I think one of the reasons is adoption of these new paradigms. OODBs are much better at what they do, and I'm sure that quite a few brave souls eventually tried it and maybe used it in production, though I think the reliability, performance, extensive R&D, and features of a RDBMS, have never made it in OODB world. Same can be said for XML native databases, though I think these have more traction now than OODBs ever did. I think it all boils down to your data, and not necessarily what your application is written in OO or procedural, etc... I think some data just better fits a different storage schema. Some data is inherently rectangular, though it fits nicely into a relational model. Some data, is not, though it needs a more dynamic storage model (i.e. XML Schema). Have you ever looked at how RDBMS way of storing dynamic attributes is handled, with a large many-to-many association table, that quickly gets out of hand. Though there is still the argument that RDBMS can perform better with such a design and 100 million rows, then can an native XML and/or OODB. That is most likely true, though native XML databases are gaining traction, IMO, especially with XQuery engine implementations and new ways of indexing XML based data. Honestly, I can see XML databases as the next major thing in the database world. Most RDBMS vendors already realize it by integrating XML based technologies into their backend relational stores. I also think that the impedance mismatch between XML and OO and way less then a rectangular storage model (RDBMS) and OO. Ilya
  59. Honestly, I can see XML databases as the next major thing in the database world. Most RDBMS vendors already realize it by integrating XML based technologies into their backend relational stores.
    I hope they do a better job with that in the future than they are doing now though. I know of a project that got into serious trouble last year because of the xml db implementation of ${insert very famous database vendor here} being very buggy (including serious memory leaks). Turned out that even though this comp had been promoting it's xml features for a few years already, we were one of the first companies actually using it in a production system.
  60. Honestly, I can see XML databases as the next major thing in the database world. Most RDBMS vendors already realize it by integrating XML based technologies into their backend relational stores.


    I hope they do a better job with that in the future than they are doing now though. I know of a project that got into serious trouble last year because of the xml db implementation of ${insert very famous database vendor here} being very buggy (including serious memory leaks). Turned out that even though this comp had been promoting it's xml features for a few years already, we were one of the first companies actually using it in a production system.
    Yes, Oracle XMLDB, is probably the most developed XML interface on top of RDBMS out there, but they are not very enthusiastic about it, nor were we. First thing is creation of relational views of XML data, really sucks. You can't really control how the backend relational store is created from XML schema. Second, is a maintenance nightmare. Imagine tables of types that you can't easily view/select from, etc... They bought sleepy cat, but I wonder how they'll integrate the features, or they just did it to take them off the market eventually. I think one of the most developed XML dbs out there right now is RainingData's TigerLogic, based on their pretty proven Pick universal data model. Pick databases have been used in many mission critical systems in government orgs, etc... Ilya
  61. Re: Deja-Vu![ Go to top ]

    This whole discussion reminds of the discussion developers would have had when moving from assembly to high level programming languages.

    I think the use of ORM is part of the inevitable move to higher level development abstractions (SQL is the assembly language of ORM).

    Sure we can drop down to assembly/SQL level if needed, but most of our work will be at the OO/ORM level, so we can focus on the problem.
    Indeed, although I think perhaps a better analogy is with C rather than assembler - I often come across C or C++ developers who just don't get that Java can be efficient, and believe that the manual memory management and raw memory access of their language is important for performance. In the same way, I read (in my view misleading) comments and articles that ORM can't possibly be efficient and raises too many problems to be effective.
  62. Re: Deja-Vu![ Go to top ]

    This whole discussion reminds of the discussion developers would have had when moving from assembly to high level programming languages.

    I think the use of ORM is part of the inevitable move to higher level development abstractions (SQL is the assembly language of ORM).

    Sure we can drop down to assembly/SQL level if needed, but most of our work will be at the OO/ORM level, so we can focus on the problem.

    As for Relational vs OO (a broad category) database management systems, I think some contexts are better suited to RDBMS and some to OODBMS.

    Developers have/will find competitive advantage in OODBMS for certain applications, whilst others will be better done with an RDBMS.

    Cheers,
    Ashley.

    --
    Ashley Aitken
    Perth, Western Australia
    mrhatken at mac dot com
    Most heartily agree. I started to post the same comments last night, but i couldn't get it to sound this reasonable.
  63. I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.
    1. As was probably said in this thread already, JDBC is a poor API for applicaiton developers. It's too low-level. But that's OK, as it must be usefull for other purposes (SQL workspaces, clustered drivers like C/JDBC, caching and logging wrappers and so on). 2. It's tedious only if you provide functionality that noone uses (like INSERT, SELECT/UPDATE/DELETE by PK for each and every table). 3. SQL is not much more verbose than ORM queries. All they save you is projection and PK equivalence on joins. But if you factor in the code that has to "post-process" ORM results (instead of doing it all in the query), you can easily top SQL.
  64. I can't see how you can possibly claim this. ORM is used precisely because so many of us find it hugely simpler and easier than the tedious and verbose combination of SQL + JDBC.
    2. It's tedious only if you provide functionality that noone uses (like INSERT, SELECT/UPDATE/DELETE by PK for each and every table).
    I am not sure I understand you here - you would surely be providing such functionality if you were basically implementing your own ORM?
    3. SQL is not much more verbose than ORM queries. All they save you is projection and PK equivalence on joins. But if you factor in the code that has to "post-process" ORM results (instead of doing it all in the query), you can easily top SQL.
    But that is the point - if you are using Java in your application you won't want to do everything in the query - you will want to retrieve objects and manipulate things in Java. This is the huge advantage in (lack of) verbosity and code cleanliness of transparent persistence.
  65. It's tedious only if you provide functionality that noone uses (like INSERT, SELECT/UPDATE/DELETE by PK for each and every table).
    I am not sure I understand you here - you would surely be providing such functionality if you were basically implementing your own ORM?
    I was comparing SQL-based approach (say with iBatis) and ORM based one.
    But that is the point - if you are using Java in your application you won't want to do everything in the query - you will want to retrieve objects and manipulate things in Java. This is the huge advantage in (lack of) verbosity and code cleanliness of transparent persistence.
    Well, that's the problem - transparent persistance works only for demo apps. You stumble upon roadblocks (which were described by Ted) very soon as you leave simple CRUD. Then you have to sacrifice either transparance (e.g. by choosing appropriate fetch plan for different use cases, using queries or even doing parts outside ORM) or performance (e.g. suffer 1+N or excessive fetching).
  66. Well, that's the problem - transparent persistance works only for demo apps. You stumble upon roadblocks (which were described by Ted) very soon as you leave simple CRUD. Then you have to sacrifice either transparance (e.g. by choosing appropriate fetch plan for different use cases, using queries or even doing parts outside ORM) or performance (e.g. suffer 1+N or excessive fetching).
    Sorry, but this is simply not correct (to put it mildly). Transparent persistence is widely and effectively used in substantial applications. It is the foundation of EJB 3.0, and I don't think anyone could seriously claim that EJB 3.0 persistence is intended to work 'only on demo apps'! Transparent persistence ORMs like Hibernate and JDO have been used for very substantial applications. Using different fetch plans and fetch groups for different use cases is nothing whatsoever to do with sacrificing transparent persistence - it is one of the approaches in JDO that allows transparent persistence to be optimised. Also, any suggestion that transparent persistence necessarily leads to performance loss is wrong - there are mechanisms in modern ORMs to precisely address this and the 1+N fetching issue. It is very easy to come up with a series of potential roadblocks that can get in the way of efficient ORM use, but the fact is that ORM has been used widely and efficiently for years. The roadblocks are often illusory, or we simply manage to deal with them.
  67. Well, that's the problem - transparent persistance works only for demo apps. You stumble upon roadblocks (which were described by Ted) very soon as you leave simple CRUD. [..]
    Sorry, but this is simply not correct (to put it mildly).
    Steve, you cannot win this argument, even though you are correct. (I covered this topic in my "Top 10 ways to botch a Java Enterprise Application" talk at JavaOne.) Basically, anyone who knows both Java and SQL knows that the theoretical maximum throughput will be achieved by having the most optimal SQL being executed at every necessary point in the application, and therefore it is only possible for an ORM to *approach* the efficiency of a perfectly-coded application in the "CRUD" use case, and in all other cases ORMs will fall short. The only problem with the theory is that any application of over (for example) 10kloc cannot be built in such a manner, meaning that as the complexity of the application grows, its ability to grasp micro-optimizations for each conversation with the database become more and more tenuous. Eventually, as the application complexity grows, any decent ORM will tend to wipe the floor with such a "well-optimized" application, because of the ability to separate concerns (and thus achieve predictable worst-case results). I can't scientifically prove any of this, of course, but having seen hundreds of large-scale applications I have a pretty good feeling for things that *tend* to succeed and things that *tend* to fail. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  68. Well, that's the problem - transparent persistance works only for demo apps. You stumble upon roadblocks (which were described by Ted) very soon as you leave simple CRUD. [..]


    Sorry, but this is simply not correct (to put it mildly).


    Steve, you cannot win this argument, even though you are correct. (I covered this topic in my "Top 10 ways to botch a Java Enterprise Application" talk at JavaOne.) Basically, anyone who knows both Java and SQL knows that the theoretical maximum throughput will be achieved by having the most optimal SQL being executed at every necessary point in the application, and therefore it is only possible for an ORM to *approach* the efficiency of a perfectly-coded application in the "CRUD" use case, and in all other cases ORMs will fall short.

    The only problem with the theory is that any application of over (for example) 10kloc cannot be built in such a manner, meaning that as the complexity of the application grows, its ability to grasp micro-optimizations for each conversation with the database become more and more tenuous. Eventually, as the application complexity grows, any decent ORM will tend to wipe the floor with such a "well-optimized" application, because of the ability to separate concerns (and thus achieve predictable worst-case results).

    I can't scientifically prove any of this, of course, but having seen hundreds of large-scale applications I have a pretty good feeling for things that *tend* to succeed and things that *tend* to fail.

    Peace,

    Cameron Purdy
    Tangosol Coherence: The Java Data Grid
    You are right - the general argument is unwinnable, but my main concern with this post was with the statement that 'transparent persistence only works for demo apps'. I found this both wrong but fascinating. To me, it typifies a problem in IT of parochial attitudes - an unwillingess to look beyond the familiar and a tendency to make assumptions about other approaches.
  69. I was comparing SQL-based approach (say with iBatis) and ORM based one.
    ...
    You stumble upon roadblocks (which were described by Ted) very soon as you leave simple CRUD.
    Also, as stated by Ted - iBatis is an ORM. So is any code a developer writes to get "data" from a RDMBS and put it in domain objects.
  70. iBatis and ORM[ Go to top ]

    iBatis is not ORM. Sure it maps objects to relations, but that's about it. First thing that gives it away is that you're not mapping tables to objects, but query results.
  71. Re: iBatis and ORM[ Go to top ]

    iBatis is not ORM. Sure it maps objects to relations, but that's about it. First thing that gives it away is that you're not mapping tables to objects, but query results.
    You say potato ...
  72. Re: iBatis and ORM[ Go to top ]

    iBatis is not ORM. Sure it maps objects to relations, but that's about it. First thing that gives it away is that you're not mapping tables to objects, but query results.

    You say potato ...
    Yea, these ORM discussins aren't much different than discussing God, or corporate strategy, or Lacanian philosophy. In the end, it all depends on what "is" is. And btw, iBatis not being an ORM framework is a good thing.
  73. Re: iBatis and ORM[ Go to top ]

    iBatis is not ORM. Sure it maps objects to relations, but that's about it. First thing that gives it away is that you're not mapping tables to objects, but query results.

    You say potato ...

    Yea, these ORM discussins aren't much different than discussing God, or corporate strategy, or Lacanian philosophy. In the end, it all depends on what "is" is.

    And btw, iBatis not being an ORM framework is a good thing.
    lol. But is a framework. And it allows you to "move data between your Java and .NET objects and a relational database" by use of the "The Data Mapper framework (a.k.a. SQL Maps)".
  74. Re: iBatis and ORM[ Go to top ]

    But is a framework. And it allows you to "move data between your Java and .NET objects and a relational database" by use of the "The Data Mapper framework (a.k.a. SQL Maps)".
    The same also describes JDBC and ADO.NET. Remember, ResultSet is an object, too. (Actually, it's a class, but you get the point).
  75. Re: iBatis and ORM[ Go to top ]

    But is a framework. And it allows you to "move data between your Java and .NET objects and a relational database" by use of the "The Data Mapper framework (a.k.a. SQL Maps)".

    The same also describes JDBC and ADO.NET. Remember, ResultSet is an object, too.

    (Actually, it's a class, but you get the point).
    By objects, we mean "domain objects" or POJOs/POCOs. But yes, it can describe JDBC/ADO.Net if you are moving "data" between objects and recordsets.
  76. Re: iBatis and ORM[ Go to top ]

    By objects, we mean "domain objects" or POJOs/POCOs. But yes, it can describe JDBC/ADO.Net if you are moving "data" between objects and recordsets.
    Ah, this makes more sense, than. Problem is, iBatis object make poor domain objects. Mostly because persistence is not "transparent" at all. For example, objects have to be explicitly saved, programmer has to decide wheter to do INSERT or UPDATE, no N:M mapping support (1:N is decent, though), doesn't help with object identity issues and so on ... Mind you, I miss these features as bad as roadside bombs on my route to work.
  77. Re: iBatis and ORM[ Go to top ]

    Mind you, I miss these features as bad as roadside bombs on my route to work.
    Mind you, there are so many ORM frameworks with so many ORM-QLs; Most of them wll wither in years, the ORM tool you are using and the ORMQL you are speaking can be a bomb to your company.
  78. Re: iBatis and ORM[ Go to top ]

    iBatis not being an ORM framework is a good thing.
    +1
  79. Re: iBatis and ORM[ Go to top ]

    iBatis is not ORM. Sure it maps objects to relations, but that's about it. First thing that gives it away is that you're not mapping tables to objects, but query results.
    You say potato ...
    I say "patata" (posting from Barcelona ;-) Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  80. Re: iBatis and ORM[ Go to top ]


    I say "patata" (posting from Barcelona ;-)
    And I say "ziemniak" or even "kartofel" .. ;) (posting from Warsaw)
  81. Re: iBatis and ORM[ Go to top ]


    I say "patata" (posting from Barcelona ;-)


    And I say "ziemniak" or even "kartofel" .. ;) (posting from Warsaw)
    Tater.
  82. Could you clarify please what advantages exactly your manual mapping brought to the projects and why do you think maintainers of your past projects enjoy supporting them more than projects which use widely accepted persistence frameworks?

    And why you did not you use iBatis anyway?
    It brought performance, maintability and simplicity. We usualy have mini framework/library for database access, that is simple, extremely lightweight, designed using well known patterns and to meet specific needs of the project (no extra features that introduce complexity). This is, in my opinion, better for maintability then using third party framework which might have it's own quirks, extra complexity, lack of documentation in specifc ares, etc...
  83. Could you clarify please what advantages exactly your manual mapping brought to the projects and why do you think maintainers of your past projects enjoy supporting them more than projects which use widely accepted persistence frameworks?

    And why you did not you use iBatis anyway?

    It brought performance, maintability and simplicity. We usualy have mini framework/library for database access, that is simple, extremely lightweight, designed using well known patterns and to meet specific needs of the project (no extra features that introduce complexity). This is, in my opinion, better for maintability then using third party framework which might have it's own quirks, extra complexity, lack of documentation in specifc ares, etc...
    Well this is a matter of taste, I have been in several projects some using homebrew orm mappers some using hibernate and others. From all projects, the funny thing is the homebrew orm mappers worked best. In most Hibernate projects I worked in people were fighting often with the paradigm clash of having session object management and that you lose the objects over request boundaries. Fighting with errors like object already exists (loaded due to internal dependencies) with some critical parts of the documentation which are one lines easily overlooked etc... is the main problem of Hibernate, despite is excellency and good documentation. But documentation, what is the difference, between merge, save update and lock is viable, documentation on how to deal best with many to many relationships within the written docs as well, also how to deal with the clash between web boundaries and hibernate session management. 90% of the time people are fighting with it stem from those areas (and everyone, literally everyone I met so far using it has had those problems to a degree that at one point he was frustrated to a huge extend with it) The funny thing is most homebrew stuff I have used followed the ibatis approach and one followed an opensessioninconversation approach, all those approaches worked fine, the api was less of a mess and people really coded instead of fighted with the orm layer. It is less that the homebrew stuff was better featurewise, it was definitely not (except for the startup times, due to omitting xml) but the apis were way more streamlined, way less clutter (no merge, save, update basically doing the same, but differently you could rely on the object being persistable etc...) The funny thing is, rails orm framework follows also this easy to handle, small api approach, and it seems to work well, no big complaints from anyone having used it in my surrounding, and it has been quite a number of people. I would not say that the orm cannot work, but my biggest gripe is, that you add another complexity layer that in case of many, Hibernate is not the exception but the rule, is almost as big as the database itself. People should not have the message that the object just coming in, having the proper id etc... clases with something already loaded while the last op was a load of something entirely different. People should not be told you can use the orm system in many ways, while in fact a many to many relationship has to be designed in one way to avoid hidden object loads. People should not fight with five different store functions basically doing the same but differently. People should not fight with caching issues once they have to fall back into plain jdbc. So to sum it up, orm can work quite nicely, but the current state of affairs is way too complex.
  84. Could you clarify please what advantages exactly your manual mapping brought to the projects and why do you think maintainers of your past projects enjoy supporting them more than projects which use widely accepted persistence frameworks?

    And why you did not you use iBatis anyway?

    It brought performance, maintability and simplicity. We usualy have mini framework/library for database access, that is simple, extremely lightweight, designed using well known patterns and to meet specific needs of the project (no extra features that introduce complexity). This is, in my opinion, better for maintability then using third party framework which might have it's own quirks, extra complexity, lack of documentation in specifc ares, etc...
    I've had the opposite experience. First, your in-house will inevitably have its own quirks. You're just used to them. The new guy will have to learn your quirks. I'd rather deal with the quirks of a well-known ORM tool that has good docs. Besides being more portable than your in-house, Hibernate, for example, is more wide used. We build our in-house application framework using Spring and Hibernate. Now, when someone has questions, I can give them the Spring or Hibernate book instead of my documentation. In my experience, in-house frameworks are usually more complex, have more quirks, and generally less documentation because the people who wrote it are there. Right up until they leave, then you've got a problem. We've had great success ramping up new people because of the good dessemination of Spring and Hibernate. Most recently, a new senior guy joined and was immediately productive because of the standardization on Spring and Hibernate.
  85. I've had the opposite experience. First, your in-house will inevitably have its own quirks. You're just used to them. The new guy will have to learn your quirks. I'd rather deal with the quirks of a well-known ORM tool that has good docs.
    It will sure have quirks, but because it is order of magnitude simpler it will have order of magnitude less quirks. This is because it is designed to meet the needs of the specific project, and not general ORM mapping tool problem.
    We build our in-house application framework using Spring and Hibernate. Now, when someone has questions, I can give them the Spring or Hibernate book instead of my documentation.
    OK, and I can give Java/C#, Patterns and SQL book, which are sure more popular then Hibarnate/Spring books.
    In my experience, in-house frameworks are usually more complex, have more quirks, and generally less documentation because the people who wrote it are there.
    Regarding complexity and quirks (in the context of the specific project), I believe it is oposite. Regarding docs, you are right.
  86. I've had the opposite experience. First, your in-house will inevitably have its own quirks. You're just used to them. The new guy will have to learn your quirks. I'd rather deal with the quirks of a well-known ORM tool that has good docs.

    It will sure have quirks, but because it is order of magnitude simpler it will have order of magnitude less quirks. This is because it is designed to meet the needs of the specific project, and not general ORM mapping tool problem.
    No, it really won't be an order of magnitude simpler. The reason that popular ORMS (such as Hibernate) are popular is because they are based on the experience of developers who have worked on a very large range of projects and their requirements. For any sufficiently large project, the quirks and requirements of that project will usually converge on what is provided by systems such as JPA, JDO 2.0 or Hibernate. Trying to re-invent all this for a specific project is a serious waste of time.
  87. No, it really won't be an order of magnitude simpler. The reason that popular ORMS (such as Hibernate) are popular is because they are based on the experience of developers who have worked on a very large range of projects and their requirements. For any sufficiently large project, the quirks and requirements of that project will usually converge on what is provided by systems such as JPA, JDO 2.0 or Hibernate. Trying to re-invent all this for a specific project is a serious waste of time.
    Yes it will be order of magnited simpler :). Library of less then 100KB compressed jar with no dependencies is sure order of magnitude simpler then several MBs jar accompanied with the same amount of dependent libraries' MBs! Sure that Hibernate is much more proven and tested as a general ORM tool compared to any in-house library, but the key point is that it is a general ORM tool solving general ORM tool problem, so it MUST be complex as problem that it is trying to solve is inherently VERY complex (Gavin will say that it is not that complex, but I realy think it is). On the other hand, in-house small library that can be accomodated to meet specific project needs could be more efficient in terms of performance and developer productivity (taking into account time spent on 'fighting' the tool, which is common when ORM tools are used).
  88. Yes it will be order of magnited simpler :). Library of less then 100KB compressed jar with no dependencies is sure order of magnitude simpler then several MBs jar accompanied with the same amount of dependent libraries' MBs! Sure that Hibernate is much more proven and tested as a general ORM tool compared to any in-house library, but the key point is that it is a general ORM tool solving general ORM tool problem, so it MUST be complex as problem that it is trying to solve is inherently VERY complex (Gavin will say that it is not that complex, but I realy think it is). On the other hand, in-house small library that can be accomodated to meet specific project needs could be more efficient in terms of performance and developer productivity (taking into account time spent on 'fighting' the tool, which is common when ORM tools are used).
    No, size of library and wide range of functionality does not mean complexity when it comes down to use. I don't understand what you mean by 'fighting' the tool.
  89. Yes it will be order of magnited simpler :). Library of less then 100KB compressed jar with no dependencies is sure order of magnitude simpler then several MBs jar accompanied with the same amount of dependent libraries' MBs! Sure that Hibernate is much more proven and tested as a general ORM tool compared to any in-house library, but the key point is that it is a general ORM tool solving general ORM tool problem, so it MUST be complex as problem that it is trying to solve is inherently VERY complex (Gavin will say that it is not that complex, but I realy think it is). On the other hand, in-house small library that can be accomodated to meet specific project needs could be more efficient in terms of performance and developer productivity (taking into account time spent on 'fighting' the tool, which is common when ORM tools are used).


    No, size of library and wide range of functionality does not mean complexity when it comes down to use.

    I don't understand what you mean by 'fighting' the tool.
    I could be wrong by I think it was Booch: "If your model gets too complex add more classes". And I don't think that SW should be sold "by the pound" (see topic on Derby DB in Mustang). Guido
  90. I could be wrong by I think it was Booch:
    "If your model gets too complex add more classes".
    And I don't think that SW should be sold "by the pound" (see topic on Derby DB in Mustang).


    Guido
    I think this is irrelevant. Hibernate and other ORMs aren't 'models', they are a set of libraries. They can be large because they can include a considerable number of features, based on years of experience of what developers actually need to be productive. These features can include different ways to add functionality to the persistent classes (proxying, enhancement), different ways of querying and different ways of optimising things. In this case, I have no objection at all to software being 'sold by the pound' - I like the choice and versatility offered by the systems. I see no point at all in trying to write 'home grown' ORMs these day.
  91. I could be wrong by I think it was Booch:
    "If your model gets too complex add more classes".
    And I don't think that SW should be sold "by the pound" (see topic on Derby DB in Mustang).


    Guido


    I think this is irrelevant. Hibernate and other ORMs aren't 'models', they are a set of libraries. They can be large because they can include a considerable number of features, based on years of experience of what developers actually need to be productive. These features can include different ways to add functionality to the persistent classes (proxying, enhancement), different ways of querying and different ways of optimising things. In this case, I have no objection at all to software being 'sold by the pound' - I like the choice and versatility offered by the systems. I see no point at all in trying to write 'home grown' ORMs these day.
    Well, I meant that is irrelevant the "weight" of the SW to measure its quality, or better it is not the only quality factor, in either sense (more is worse/complicated, less is better/easy or viceversa - even if Booch citation is dramatically true). And, as rather often, I agree with you that it doesn't make sense to write 'home grown' ORMs (I would say, makeshift) these days. Guido P.S. Hibernate and other JDO implementations ARE models. Of thei own domain, and when the implementation gets too complicated, typically because you classes have no well defined role and do too much, you better add other classes to better capture the essence of your model.
  92. Hibernate and other JDO implementations ARE models. Of thei own domain, and when the implementation gets too complicated, typically because you classes have no well defined role and do too much, you better add other classes to better capture the essence of your model.
    I disagree. For example, my impression and experience of JDO is that things are very well-defined. It is a clean and easy-to-use API.
  93. No, it really won't be an order of magnitude simpler. The reason that popular ORMS (such as Hibernate) are popular is because they are based on the experience of developers who have worked on a very large range of projects and their requirements. For any sufficiently large project, the quirks and requirements of that project will usually converge on what is provided by systems such as JPA, JDO 2.0 or Hibernate. Trying to re-invent all this for a specific project is a serious waste of time.


    Yes it will be order of magnited simpler :). Library of less then 100KB compressed jar with no dependencies is sure order of magnitude simpler then several MBs jar accompanied with the same amount of dependent libraries' MBs! Sure that Hibernate is much more proven and tested as a general ORM tool compared to any in-house library, but the key point is that it is a general ORM tool solving general ORM tool problem, so it MUST be complex as problem that it is trying to solve is inherently VERY complex (Gavin will say that it is not that complex, but I realy think it is). On the other hand, in-house small library that can be accomodated to meet specific project needs could be more efficient in terms of performance and developer productivity (taking into account time spent on 'fighting' the tool, which is common when ORM tools are used).
    Well, maybe you've had better experiences with in-house persistence than I've had. The last in-house one that I used was a nightmare. In fact, this tool was the icing that sent me looking for a better solution. It was poorly documented. It didn't work well. It was hard to use, hard to extend, and just bad all around. Hibernate was easy to get up and running and required very little fighting, at least for us. We had a vast array of resources to query when questions did arise instead of being at the mercy of people long since gone. We've had even junior developers up and running very quickly because of our use of Hibernate. Now, I've used an ORM tool that didn't seem quite as straightforward, so I guess it depends on the tool.
  94. In my experience, mannual maping did better job then using an ORM tool.

    In my last 4 projects, I used mannual mapping on 3, and ORM tool (Hibernate) on one. I am much more satisfied with mannual mapping then using Hibernate (performance, manitainability, even development productivity in later stages of development), especialy when working on very complex/large projects.

    I am now working on .NET project and applying mannual ORM mapping, using all techniques I used in Java projects, and things are going smoothly for now (although JDBC is much better then ADO.NET when using ORM :) ).
    This is totally opposite of my experience. Hibernate and NHibernate allow me to concentrate on other things. Sure, there are some things they force me to deal with, but it much less than "rolling my own". I am using NHIbernate on a new project. It allowed me to crank out the app. Very quickly. I've done enough the other way to know how long it takes.
  95. In my experience, mannual maping did better job then using an ORM tool.

    In my last 4 projects, I used mannual mapping on 3, and ORM tool (Hibernate) on one. I am much more satisfied with mannual mapping then using Hibernate (performance, manitainability, even development productivity in later stages of development), especialy when working on very complex/large projects.

    I am now working on .NET project and applying mannual ORM mapping, using all techniques I used in Java projects, and things are going smoothly for now (although JDBC is much better then ADO.NET when using ORM :) ).

    This is totally opposite of my experience. Hibernate and NHibernate allow me to concentrate on other things. Sure, there are some things they force me to deal with, but it much less than "rolling my own". I am using NHIbernate on a new project. It allowed me to crank out the app. Very quickly. I've done enough the other way to know how long it takes.
    A company that I used to work for switched to NHibernate after first having used ADO.NET for a .NET project. They were totally happy after the switch, and felt they improved on maintainability and flexibility.
  96. A company that I used to work for switched to NHibernate after first having used ADO.NET for a .NET project. They were totally happy after the switch, and felt they improved on maintainability and flexibility.
    I am not talking about not using ORM, but about not using an ORM tool in complex/misssion critical projects. In that context, using ADO.NET under the ORM layer is much less pleasant experiance then using JDBC.
  97. In my experience, mannual maping did better job then using an ORM tool.
    I don't think the ORM tool advocates are saying that ORM tools are better that manual coding. They are saying that they are easier to use, require less skills, are faster than manual coding, etc, .... in many projects. So if ORM actually works in a specific project, it probably makes commercial sense. PJ Murray, CodeFutures Software Data Access Objects and Service Data Objects
  98. In my experience, mannual maping did better job then using an ORM tool.


    I don't think the ORM tool advocates are saying that ORM tools are better that manual coding. They are saying that they are easier to use, require less skills, are faster than manual coding, etc, .... in many projects. So if ORM actually works in a specific project, it probably makes commercial sense.



    PJ Murray, CodeFutures Software
    Data Access Objects and Service Data Objects
    I agree on that.
  99. I don't think the ORM tool advocates are saying that ORM tools are better that manual coding.
    I am. I would rather trust a well-designed ORM tool to produce, on average, better SQL for dozens of different queries than I could on (for example) Oracle. Part of the reason I use a high-performance ORM is that I know that people with more expertise than me at writing Oracle SQL have contributed to the product. I expect this ORM product to know the differences between Oracle versions and produce the appropriate optimised SQL. I also expect this ORM product to know how to produce good SQL for other systems too - for different versions of PostgreSQL, and MySQL for example, so that I neither have to manually code optimised SQL for each type of database, or have to try and reasonable code portable SQL. So, I do expect ORM tools to be better than manual coding.
  100. ...Actually, it gets worse than that--if the object hierarchy continues to grow, say to include Professor, Staff, Undergrad (inherits from Student), and a whole hierarchy of AdjunctEmployees (inheriting from Staff), and the program wants to find all Persons whose last name is Smith, then JOINs must be done for every derived class in the system, since the semantics of "find all Persons" means that the query must seek data on the PERSON table, but then do an expensive set of JOINs to bring in the rest of the data from across the rest of the database, pulling in the PROFESSOR table to fetch the rest of the data....
    If you have a column that can be used to classify data, you can use as discriminator and you will no longer have JOIN to child tables, since last name is logically placed in the root table PERSON. There are lots of truths in this document, however if the developer understand how the ORM engine works, he can optimize the mapping to avoid expensive queries.
  101. I have read the first half of the article; and his understanding of the Vietnam war is rather shallow. Luckily on the ORM side he has some points...
  102. There really isn't such a mismatch when you are comparing a 4th normal form model with a domain object model. There is no reason that the relational model cannot be implemented in OO. The problems stem from the way RDBMS's are implemented and tuned. RDBMS's use foreign keys to represent relationships instead of references, for example. Also, 4th normal and even 3rd normal form are rarely implemented because of performance considerations. DBAs have been known to do a number of things with schemas that may improve performance but reduce flexibility and are down right ugly. In green field development it is possible to prevent or mitigate most ORM problems and maintain a reasonable mapping between the domain object model and the schema. Triggers and stored procedures can handle many denormalization issues and you can generally avoid a number of RDBMS conventions that create ORM problems. Factless identifiers as primary keys map better to objects than real data, for example. Another example is not to breakup non-aggregate objects (3rd normal form entities) across tables. Keep at least the "master" object all in one table. Use them as building blocks. Even without ORM I wouldn't do that, but you see it done. Of course, when you are given a legacy database to use, one that may be poorly modeled or highly denormalized, there are not easy answers, especially if you are sharing it. Unfortunately in my experience that is the most common case.
  103. Retrieval Policy[ Go to top ]

    From the article:
    Most O/R-M's offer some kind of developer-driven decision-making support, usually some kind of configuration or mapping file, to identify exactly what kind of retrieval policy will be, but this setting is global to the class, and as such can't be changed on a situational basis.
    JDO2 provides just such a retrieval on a situational basis - it's called "fetch-groups". One retrieval you could retrieve all fields of an object, and the next you could retrieve just the key ones depending on your requirement.
  104. I iz a wunderin....[ Go to top ]

    Since we is havin fun with colorful metaphors, I wonder if we could liken Microsoft's place in technology to somethin like a Chernobyl Accident (http://en.wikipedia.org/wiki/Chernobyl_accident) seeing how it's kind of a run-away reaction that cain't be contained and has contaminated technology for many generations to come now? Talk amongst yerselves!
  105. Some good reasons to use ORMs: - caching: ORMs can do lots of smart things that DB can't - maintenance: maybe if Ted had to maintain SQL queries with hundreds of columns, he might appreciate writing them using an ORM Anyway, it doesn't seem to me that Ted has looked in details at an ORM (at least hibernate). Hibernate configuration and mapping capabilities are very powerful. It is usually possible to "tune" it to generate SQL very close to what you would have written by hand.
  106. Ted has also written an article comparing LINQ to ORM, which contains a lot of nonsense (see http://msdn.microsoft.com/netframework/default.aspx?pull=/library/en-us/dndotnet/html/linqcomparisons.asp) This article got one of the worst ratings for any article on msdn (go to "see this in msdn library" and scroll down) A 4GL like integration of database access into the language has failed so far on the one hand and on the other hand it does not target the same problems as ORM does. Christian
  107. I love the smell of Sql in the morning ... S.
  108. I love the smell of Sql in the morning ...

    S.
    Best post of the day! BTW, you are dating yourself.
  109. I love the smell of Sql in the morning ...

    S.
    No words to comment the beauty of the post. I think I will make it my screen saver. Guido
  110. I've only had time to skim through Ted's blog but noticed.
    find all Persons whose last name is Smith, then JOINs must be done for every derived class in the system, since the semantics of "find all Persons" means that the query must seek data on the PERSON table, but then do an expensive set of JOINs to bring in the rest of the data from across the rest of the database, pulling in the PROFESSOR table to fetch the rest of the data, not to mention the UNDERGRAD, ADJUCTEMPLOYEE, STAFF, and other tables
    Not true! If you're only seeking all the smiths you only load all the instances of the person class where the lastname = smith. At this point you're not interested in the fields from the generalisation classes so you don't load them.
    Inheritance mapping isn't the end of it; associations between objects, the typical 1:n or m:n cardinality associations so commonly used in both SQL and/or UML, are handled entirely differently: in object systems, association is unidirectional, from the associator to the associatee (meaning the associated object(s) have no idea they are in fact associated unless an explicit bidirectional association is established), whereas in relational systems the association is actually reversed, from the associatee to the associator (via foreign key columns). This turns out to be surprisingly important, as it means that for m:n associations, a third table must be used to store the actual relationship between associator and associatee, and even for the simpler 1:n relationships the associator has no inherent knowledge of the relations to which it associates--discovering that data requires a JOIN against any or all associated tables at some point. (When to actually retrieve that data is a subject of some debate--see the Loading Paradox, below).
    Not quite sure what the point is here. With a unidirectional association only one side knows about the relationship, same for objects as for tables. m:n needs a join table - yea Not enough time to read it all now, will I come back to it later - I don't think so. Same with JDBC or ORM do I have time.
  111. I've only had time to skim through Ted's blog but noticed.

    find all Persons whose last name is Smith, then JOINs must be done for every derived class in the system, since the semantics of "find all Persons" means that the query must seek data on the PERSON table, but then do an expensive set of JOINs to bring in the rest of the data from across the rest of the database, pulling in the PROFESSOR table to fetch the rest of the data, not to mention the UNDERGRAD, ADJUCTEMPLOYEE, STAFF, and other tables


    Not true! If you're only seeking all the smiths you only load all the instances of the person class where the lastname = smith. At this point you're not interested in the fields from the generalisation classes so you don't load them
    Good point! Also this particular example is a poorly designed example both from an OO point of view as well as a DB point of view. PROFESSOR, UNDERGRAD, ADJUCTEMPLOYEE, etc. should not be inherited from PERSON as oftentimes a PERSON can be any number of these types of people. This type of relationship is better modelled as a Role. In this case there would be a Role class associated with Person and there would be a corresponding Role table. This would also alleviate the need for expensive joins, though some of course would still be necessary. And for simple queries like looking for all people named "Smith" no join is necessary as you have already pointed out.
  112. It seems Ted likes metaphors. So let's propose him a meta-metaphor (e.g. a metaphor about his own metaphor). ================================ In his famous article published in 1898 "Cars are the Austerlitz of the Industry" Ned Oldwar explained that cars are the biggest mistake of the Industry based on a large utopy. He wrote: "...car are useless because their engines are inefficient. A huge power would be required to go faster than 50mph, and this will never happen. They are too much expensive, they are just a leisure for lazy and arrogant aristocrats. Cars are not safe, especially when they meet trees. Cars are not robust and you can be stuck anytime in the middle of nowhere. Cars are going too fast on roads and streets that have been designed for horses not cars. Cars are not comfortable. There are no enough gas stations in the country and the oil industry is not prepared for that. Policemen are not equipped to catch robbers escaping with a car...". We have to agree Ned, as a travel expert, raised the right questions. Nevertheless at the end of his article he envisions 4 possibilities for the future: 1. People will eventually massively drive cars. I just mention this one for the sake of completeness. This is unlikely to happen for all the reasons I've detailed before. 2. People will continue to travel riding horses or walking. This is a simple, reasonable scenario. 3. People will stop travelling. It is time to understand that travelling is a huge waste of time and energy. Good citizens working 10 hours a day, 6 days a week should be too tired to travel on Sunday. Travels prevent you from focusing on important things. 4. As mother Nature will recognize there is a fast-growing need for travels, so two wings will grow in the back of human beings as an artifcat of Darwinism. Exactly like Giraffes have a long neck because they wanted to eat leaves at the top of trees. This is the most pragmactic scenario. I would not be surprized that before one century people will gently fly in the sky.
  113. Wow, for all this discussion, there is very little mention of what I suspect is the crux of the matter. If you are designing an application around data transfer objects (DTOs), or plain old "records", you are not likely to see much advantage from ORM. In fact, some developers are bound to struggle with a tool as sophisticated as Hibernate. It's a big gun, so if you shoot yourself in the foot with it, you'll make a big hole. If you are designing an application around a domain object model, you are likely to benefit from ORM. And Hibernate is an excellent ORM tool. The difference is whether you are building business logic into your objects that are mapped to the database, or they are just structures that hold data. Object bigots will complain that the record/DTO approach is "not object oriented." This is really just name calling. The DTO approach will work fine in many cases, and you can use tools like iBatis or Spring's JDBC framework to avoid some of the messier and error prone aspects of JDBC coding. You can even (gasp) leverage stored procedures in the (horrors) database. This style is probably just better suited to the dispositions of some teams/organizations. But if you have a problem that lends itself to a domain object model, and you have developers that really understand objects, that can be a beautiful thing. I suspect a lot of the problem with ORM occurs from teams that are fundamentally oriented towards records/DTOs, but they adopt Hibernate because of the hype, or because they find some tool that will automagically generate a bunch of (data transfer) objects fromt the database metadata. --Jeff Wright ThotWave Technologies
  114. Exactly. The problem is that the proper object-oriented "domain-model" architectural pattern is still rarely used. So the only advantage of ORM in most projects is a static classes-tables mapping. Static aspects of ORM are trivial. There is no real need to use an ORM tool in that case. Transactions, concurrency, consistency of the state - those are the aspects that are really challenging and when ORM tools show their power. http://www.enterpriseware.eu
  115. WARNING I have not read the answers to the topic, nor the referred article in depth. WARNING 2 I am not an USA citizen. But I cannot resist to express my feeling. First I have found rather disgusting the use of a enormous tragedy (not only for the USA people) the Vietnam was as arguments in favour of some thesis on totally different topics (at least because noone dies of ORM). More, and worse, the parallel is used to "explain" what is well-known and well-documented since the epoch (read here "hot-water discovery"). Not mentioning the history rewrite attempt (even if it was not the primary purpose of the article) I think that large part of ORM troubles (beside the intrinsic impedance-mismatch) reside in a substantial ignorance of OOA and OOD techniques. I mean ORM troubles are frequently caused by bad object models. On specific topics of the article I have the impression that the authors starts with ORM then somewhere he loses the leading 'O'. I mean that I feel as if once you have an initial mapping, it is normal that the object model does not rule the system anymore. Just like OOA, OOD an ORM being a path to have a relational schema and from that point everything is thought in RDBMS terms. If you have your object model driving the system, there are no entity identity issues, no tuples, no tables, no RDBMS modelling concepts. It is used as a storage engine of object states. But, again, this story has already been written by (selected readings I like, in no particular order): 1. Scott Ambler: various articles on persistence 2. James Rumbaugh et al: Object-Oriented Modeling and Design 3. William Kent: Data and Reality Guido.
  116. Missing the point[ Go to top ]

    The principle means of abstraction in the relational model and OO are complete opposites of one another. In the relational model, data is the principle means of abstraction. You can define views and/or queries that contain all sorts of logic - and implement considerable levels of abstraction - but in the end it all looks like data. Constraints are declaritive purely declaritive and expressed in predicate logic that is dependent on state. Procedural logic is tacked on and somewhat klunky, because there are some things that are simply much easier to do in an imperative fashion. Navigational semantics are non-existent. Creating read-only abstractions (views) is easy and intuitive, creating modifiable ones is very difficult. All actions are explicitly contextual with the context being the entire dataset. Defining constraints on data outside the internal dataset is difficult to impossible. And did I mention I am unaware of any complete implementations of the relational model? The OO model, on the other hand, is procedure-centric. The principle means of abstraction is the interface, which consists entirely of methods/functions. Data is a second class citizen and almost entirely navigational in nature (in a pure OO language everything is an object, and you navigate references from one object to another). Constraints are encapsulated in methods and are not an explicit part of the interface. Context is implicit in objects, and frequently undefined. Effective objects can be built relatively easily using composition/aggregation. Both paradigms have substantial advantages and disavantages. Personally, I think data - complete with declarative constraints and explicit context - should be made a first class part of the programming model. There are some cases where the data-centric view makes for a much cleaner abstraction. There are some cases where the behavior-centric view makes a much cleaner abstraction. The two aren't fundamentally incompatible. The famed object/relational impedence mismatch occurs when you want to represent the same aspect of your model as both data and behavior. Let attributes be attributes (whether derived or not) and methods be methods. Constraints apply to attributes and signatures to methods. Interfaces represent the contract specifying the attribtues and methods provided by an object. ORM is a bandaid for the impedance mismatch. It is a practical solution for the situation today, but its eternal perpetuation is detrimental.
  117. Re: Missing the point[ Go to top ]

    ORM is a bandaid for the impedance mismatch. It is a practical solution for the situation today, but its eternal perpetuation is detrimental.
    ORM is perhaps 'overused' because some Java developers are not confortable with relational databases. It's worth considering the business context of an application before deciding on a 'data first' versus 'logic first' approach. If the data (for example, customer records) is going to be needed by multiple applications and will be maintained for very long periods, then it's probably better to adopt the 'data first' approach. If the business logic is more important than the data, which will not be stored/reused/maintained in the longer term, then the productivity of the ORM approach makes sense, especially when a Java developer is not comfortable with relational databases. PJ Murray, CodeFutures Software Data Access Objects and Service Data Objects
  118. Re: Missing the point[ Go to top ]

    then the productivity of the ORM approach makes sense, especially when a Java developer is not comfortable with relational databases.
    As has been pointed out before (in other discussions), using an ORM does not absolve the developer of having a good knowledge of relational databases.
    ORM is perhaps 'overused' because some Java developers are not confortable with relational databases.
    ORMs are, perhaps, underused because some Java developers are not comfortable with OOP. ;)
    If the data (for example, customer records) is going to be needed by multiple applications and will be maintained for very long periods, then it's probably better to adopt the 'data first' approach.
    Maybe, but not probably. There is another approach. It is not used as much because many developers (and DBAs and PMs and ...) have trouble thinking beyond the database and the UI.
  119. Re: Missing the point[ Go to top ]

    then the productivity of the ORM approach makes sense, especially when a Java developer is not comfortable with relational databases.

    As has been pointed out before (in other discussions), using an ORM does not absolve the developer of having a good knowledge of relational databases.
    Indeed. Also, there can be good reasons why a Java developer is not 'comfortable' with relational databases, or at least some aspects of them. The lack of easy portability between different products is one of these. This can be a major reason for using ORM.

    ORM is perhaps 'overused' because some Java developers are not confortable with relational databases.

    ORMs are, perhaps, underused because some Java developers are not comfortable with OOP. ;)
    Agreed!

    If the data (for example, customer records) is going to be needed by multiple applications and will be maintained for very long periods, then it's probably better to adopt the 'data first' approach.

    Maybe, but not probably. There is another approach. It is not used as much because many developers (and DBAs and PMs and ...) have trouble thinking beyond the database and the UI.
    True.
  120. Re: Missing the point[ Go to top ]

    ORM is perhaps 'overused' because some Java developers are not confortable with relational databases
    ... and they are creating redundant ORM-QLs which are not comfortable to DBA & non-Java developers ...
  121. Re: Missing the point[ Go to top ]

    ORM is perhaps 'overused' because some Java developers are not confortable with relational databases


    ... and they are creating redundant ORM-QLs which are not comfortable to DBA & non-Java developers ...
    These ORM-QLs aren't redundant. They have the major benefit of real portability. The relational database vendors have had a very long time to fully implement existing standards for query languages, and they have not done so. Given the poor situation with SQL, it is not surprising that Java tool and API developers have come up with alternatives. Also, I find it hard to believe that a DBA or non-Java developer who can deal with all the various dialects of SQL, especially in the context of stored procedures, would have any trouble at all with EJBQL or HQL.
  122. Re: Missing the point[ Go to top ]

    Also, I find it hard to believe that a DBA or non-Java developer who can deal with all the various dialects of SQL, especially in the context of stored procedures, would have any trouble at all with EJBQL or HQL.
    Epecially when there is no such things Object-Realational impedance. ;)
  123. Re: Missing the point[ Go to top ]

    These ORM-QLs aren't redundant. They have the major benefit of real portability.
    If it is really not redundant. You should say "ORM-QL is not redundant" instead of "ORM-QLs aren't redundant". We have remade all the wheels one by one, how can you still claim that they aren't redundant ? You have misunderstood "real portability" also. The elegance of SQL is learn once use everywhere (VB/Java/.Net/Console ...). This is skill-portability & resource- portability. Do ORM-QLs offer such portability ?
    I find it hard to believe that a DBA or non-Java developer who can deal with all the various dialects of SQL
    I believe that most developers (DB/JAVA/VB/.Net) can deal with SQL much better than various dialects ORM-QLs.
    especially in the context of stored procedures, would have any trouble at all with EJBQL or HQL.
    You have perfectly mix up SQL and Procedure-SQL. It is true that there are some kinds of Procedure-SQLs such as TSQL, PL/SQL, PSQL etc, this has made Procedure-SQLs fall into same situation as ORM-QLs where there’s EJBQL, JDOQL, HQL etc. But Procedure-SQLs are for stored procedures, while SQL / ORM-QLs are for Query. Do you mean you will write your stored procedures with ORM-QLs eventually ?
  124. Re: Missing the point[ Go to top ]

    If it is really not redundant. You should say "ORM-QL is not redundant" instead of "ORM-QLs aren't redundant". We have remade all the wheels one by one, how can you still claim that they aren't redundant ?
    Absolutely. The matter of having multiple implementations is of no relevance.
    You have misunderstood "real portability" also. The elegance of SQL is learn once use everywhere (VB/Java/.Net/Console ...). This is skill-portability & resource- portability.
    Yes, but which SQL? MySQL SQL? Oracle SQL? SQL Server SQL? For anything more than the simplest queries, there are incompatibilities between these.
    Do ORM-QLs offer such portability ?
    Each individual SQL dialect can be used in VB,Java etc. but only effectively on one database. Each ORM-QL can be used primarily from Java, but on a wide range of databases. ORM-QLs have a different type of portability - one I personally consider more useful for my work. Some ORM-QLs offer additional portability. Hibernate query language can also be used in .NET via NHibernate.
    I find it hard to believe that a DBA or non-Java developer who can deal with all the various dialects of SQL

    I believe that most developers (DB/JAVA/VB/.Net) can deal with SQL much better than various dialects ORM-QLs.
    This is simply a statement of belief, backed by no evidence. The evidence for my statement is that the syntax of some ORM-QLs (such as HQL) is very much like SQL. No competent developer with any SQL experience (whatever the SQL dialect) should have trouble understanding the HQL: select from Person where id = 123
    especially in the context of stored procedures, would have any trouble at all with EJBQL or HQL.


    You have perfectly mix up SQL and Procedure-SQL. It is true that there are some kinds of Procedure-SQLs such as TSQL, PL/SQL, PSQL etc, this has made Procedure-SQLs fall into same situation as ORM-QLs where there’s EJBQL, JDOQL, HQL etc. But Procedure-SQLs are for stored procedures, while SQL / ORM-QLs are for Query.
    A meaningless distinction. CREATE PROCEDURE is part of SQL standards.
    Do you mean you will write your stored procedures with ORM-QLs eventually ?
    No, as you can write the equivalent logic in Java.
  125. Re: Missing the point[ Go to top ]

    The matter of having multiple implementations is of no relevance.
    It depends on the meaning "multiple implementations" : 1. One standard multiple implementations 2. Redundant standards Redundant implementations 3. No standard Redundant implementations For ORM, it is sadly either 2 or 3.
    No competent developer with any SQL experience (whatever the SQL dialect) should have trouble understanding the HQL: select from Person where id = 123
    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*". If you are writing "select * from Person where id = 123" in all other environments such as VB and SQL*Plus, why don’t you keep it consistent ? To make things worse, you invent other versions for JDOQL and EJBQL too.
    No, as you can write the equivalent logic in Java.
    Have you ever built ETL systems without stored procedures ?
  126. Re: Missing the point[ Go to top ]

    2. Redundant standards Redundant implementations
    3. No standard Redundant implementations

    For ORM, it is sadly either 2 or 3.
    Sorry, but this makes no sense to me. I have no idea what you mean by 'redundant'. To me, this term means 'unncessary'. I have already covered one reason why I believe ORM-QLs are necessary for at least some situations - the obvious and well-documented incompatibilities of SQL implementations. Also, there are standard implementations: JDOQL 2.0 and EJBQL.

    No competent developer with any SQL experience (whatever the SQL dialect) should have trouble understanding the HQL:

    select from Person where id = 123


    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".

    If you are writing "select * from Person where id = 123" in all other environments such as VB and SQL*Plus, why don’t you keep it consistent ? To make things worse, you invent other versions for JDOQL and EJBQL too.
    This is what I would call 'reaching' - stretching a point to an extreme where it makes no sense. I can't take seriously any claim that such minor issues would cause major problems for understanding of query languages when there are significant differences between dialects of SQL (different functions available, different types of join, different column types). There are very good reasons why these query languages aren't exactly like SQL - they are designed for retrieving objects, not records, and the syntax is appropriate. One query language - JDOQL has major differences from SQL because it is more versatile - it can be used for the handling of non-relational persistence.
    Have you ever built ETL systems without stored procedures ?
    Yes. Java is a great way to do this.
  127. Re: Missing the point[ Go to top ]

    Have you ever built ETL systems without stored procedures ?


    Yes. Java is a great way to do this.
    Are you serious that you build ETL/Warehouse systems without stored procedures ? Would you please share some more information ?
  128. Re: Missing the point[ Go to top ]

    Are you serious that you build ETL/Warehouse systems without stored procedures ? Would you please share some more information ?
    Sure. You mentioned ETL (Extract, Transform and Load). I do a considerable amount of this type of work. I believe ORMs are a great tool for this, because of their portability - for example, I can trivially change the Load destination (and I do!), just by switching the mapping file. Using JDO I can extract from a wide range of sources, including non-relational, using basically the same code - the flexibility is awesome. Contrary to popular belief, ORMs can be fast and memory-efficient - I use a well-known JDO product and it handles very large transactions efficiently. Some people hold the view that ORMs are appropriate for some tasks, but for large batch processes SQL is required. I don't, or at least I believe it is work trying a quality ORM first - the efficiency of such products for this kind of work can be surprisingly good.
  129. Re: Missing the point[ Go to top ]

    Sure.

    You mentioned ETL (Extract, Transform and Load). I do a considerable amount of this type of work. I believe ORMs are a great tool for this, because of their portability - for example, I can trivially change the Load destination (and I do!), just by switching the mapping file. Using JDO I can extract from a wide range of sources, including non-relational, using basically the same code - the flexibility is awesome. Contrary to popular belief, ORMs can be fast and memory-efficient - I use a well-known JDO product and it handles very large transactions efficiently. Some people hold the view that ORMs are appropriate for some tasks, but for large batch processes SQL is required. I don't, or at least I believe it is work trying a quality ORM first - the efficiency of such products for this kind of work can be surprisingly good.
    OK. Consider a warehouse with hundreds GB of data, will you still use JDO for even simple tasks such as data-rolling ?
  130. Re: Missing the point[ Go to top ]

    OK. Consider a warehouse with hundreds GB of data, will you still use JDO for even simple tasks such as data-rolling ?
    No. Why are you trying to debate the use of ORM by discussing a very particular situation with hundreds of GB of data? This is an extreme case. Simply moving the discussion from case to case until you find one in which even I will agree that ORM is inappropriate is not a valid debating technique for discussing the merits of ORM in general.
  131. Re: Missing the point[ Go to top ]

    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".
    Please note that Person is *not* a table and so has *no* columns to select. Person is an *object class* and has *attributes*, associations/aggregations and inheritance. Missing this point causes a lot of troubles, as you can see. Guido.
  132. Re: Missing the point[ Go to top ]

    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".

    Please note that Person is *not* a table and so has
    *no* columns to select.
    Person is an *object class* and has *attributes*, associations/aggregations and inheritance.
    Missing this point causes a lot of troubles, as you can see.

    Guido.
    No, in Java there are no aggregations/associations or "object classes". Maybe in UML or in developer's heads, but definitely not in Java. Even attributes aren't called attributes, but properties (and quite a lot of Java gurus hate them). Not to mention that writing "select from Person where id = 123" is just about the worst query one could write according to the current ORM dogma (something about (proscribed) surrogate keys not being visible to the users).
  133. Re: Missing the point[ Go to top ]

    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".

    Please note that Person is *not* a table and so has
    *no* columns to select.
    Person is an *object class* and has *attributes*, associations/aggregations and inheritance.
    Missing this point causes a lot of troubles, as you can see.

    Guido.

    No, in Java there are no aggregations/associations or "object classes". Maybe in UML or in developer's heads, but definitely not in Java. Even attributes aren't called attributes, but properties (and quite a lot of Java gurus hate them).

    Not to mention that writing "select from Person where id = 123" is just about the worst query one could write according to the current ORM dogma (something about (proscribed) surrogate keys not being visible to the users).
    This seems to me to be missing the point, which was whether or not ORM query languages can be reasonably easily understood and used by those used to SQL. It seems to me that those arguing that they aren't are the ones being dogmatic.
  134. Re: Missing the point[ Go to top ]

    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".

    Please note that Person is *not* a table and so has
    *no* columns to select.
    Person is an *object class* and has *attributes*, associations/aggregations and inheritance.
    Missing this point causes a lot of troubles, as you can see.

    Guido.

    No, in Java there are no aggregations/associations or "object classes". Maybe in UML or in developer's heads, but definitely not in Java. Even attributes aren't called attributes, but properties (and quite a lot of Java gurus hate them).

    Not to mention that writing "select from Person where id = 123" is just about the worst query one could write according to the current ORM dogma (something about (proscribed) surrogate keys not being visible to the users).


    This seems to me to be missing the point, which was whether or not ORM query languages can be reasonably easily understood and used by those used to SQL. It seems to me that those arguing that they aren't are the ones being dogmatic.
    I don't know if completely understand your post, anyway I try to clarify mine. I didn't mean to be dogmatic. Simply I wanted to remember that talking about ORM the 'O' part cannot be dropped. If an application uses an ORM tool, the application uses the 'O' and the tool manages the 'RM'. Said that the application cannot query the entities it manages in terms of tables and columns. Obviously, this has nothing to do with optimization needs that lead to some SQL in precise situations. Just like *nix systems are in C with some assembler here and there. Guido.
  135. Re: Missing the point[ Go to top ]

    Yes, I understand "select from Person where id = 123" – Syntax Error! Someone has carelessly missed the selecting columns or "*".

    Please note that Person is *not* a table and so has
    *no* columns to select.
    Person is an *object class* and has *attributes*, associations/aggregations and inheritance.
    Missing this point causes a lot of troubles, as you can see.

    Guido.

    No, in Java there are no aggregations/associations or "object classes". Maybe in UML or in developer's heads, but definitely not in Java. Even attributes aren't called attributes, but properties (and quite a lot of Java gurus hate them).

    Not to mention that writing "select from Person where id = 123" is just about the worst query one could write according to the current ORM dogma (something about (proscribed) surrogate keys not being visible to the users).


    This seems to me to be missing the point, which was whether or not ORM query languages can be reasonably easily understood and used by those used to SQL. It seems to me that those arguing that they aren't are the ones being dogmatic.

    I don't know if completely understand your post, anyway I try to clarify mine.
    I didn't mean to be dogmatic. Simply I wanted to remember that talking about ORM the 'O' part cannot be dropped.
    If an application uses an ORM tool, the application uses the 'O' and the tool manages the 'RM'.
    Said that the application cannot query the entities it manages in terms of tables and columns.
    Obviously, this has nothing to do with optimization needs that lead to some SQL in precise situations.
    Just like *nix systems are in C with some assembler here and there.

    Guido.
    Sorry if I wasn't clear. I agree with you. What I meant was that when objections to ORM resort to issues of the syntax of the query language, including complaints as to why it is not simply pure SQL or something similar (which, as you say does not make sense and we are dealing with objects), then this seems to be stretching things in my view, and making major issues out of minor ones.
  136. Re: Missing the point[ Go to top ]

    This seems to me to be missing the point, which was whether or not ORM query languages can be reasonably easily understood and used by those used to SQL. It seems to me that those arguing that they aren't are the ones being dogmatic.
    The point was that ORM languages are hard to understand as OO terminology is so ill-defined. I'd say that the problem SQL users have with ORM query languages is not that they're hard to understand. The problem is that ORM queries are not powerfull (just joins and selections) and that they are useless with SQL tools (might seem trivial, until you have to get the execution plan or when you have to change the query).
  137. Re: Missing the point[ Go to top ]

    This seems to me to be missing the point, which was whether or not ORM query languages can be reasonably easily understood and used by those used to SQL. It seems to me that those arguing that they aren't are the ones being dogmatic.

    The point was that ORM languages are hard to understand as OO terminology is so ill-defined.
    In what way?
    I'd say that the problem SQL users have with ORM query languages is not that they're hard to understand. The problem is that ORM queries are not powerfull (just joins and selections) and that they are useless with SQL tools (might seem trivial, until you have to get the execution plan or when you have to change the query).
    It is true that things may not seem as powerful with ORM queries, but that is because they are used for a different purpose, and not everything is defined in terms of the ORM query. For example, in JDO there are fetch groups and plans that allow specification of how and when things are retrieved. The idea behind much modern use of ORM queries is that they express only part of what happens in terms of retrieval and persistence in general terms (such as 'fetch instances of this object'), and a significant part (I would say the majority for well-written applications) happens transparently, or through other aspects of the API that are designed to allow things to be customised and optimised. As for ORM queries being useless in SQL tools, well the reason for that is that they aren't SQL! As has been discussed, they can have a different purpose and range of use. There are powerful commercial tools for the analysis and optimisation of ORM query languages (such as JDOQL), and now that things with JPA have been standardised, I would expect many more tools to be available for this kind of thing. Of course, when SQL is needed, there is nothing to stop you using it - JDO 2.0 allows for the use of SQL in the retrieval of objects (but, of course, you lose portability).
  138. Re: Missing the point[ Go to top ]

    and that they are useless with SQL tools
    Actually, no they are not. Check out what can be done with Hibernate Tools and your fav SQL tool. I do it all the time.
  139. Re: Missing the point[ Go to top ]

    Can Hibernate Tool do code completion (aka Ctrl+Space)? At least PLSQL Developer can for SQL.
  140. Re: Missing the point[ Go to top ]

    Can Hibernate Tool do code completion (aka Ctrl+Space)? At least PLSQL Developer can for SQL.
    Yes. And as I am typing HQL it will show the corresponding SQL. :) Can PLSQL developer generate a full functioning (maintainable and changable) web application? At least Hibernate Tools can do that. ;)
  141. Re: Missing the point[ Go to top ]

    It is true that there are some kinds of Procedure-SQLs such as TSQL, PL/SQL, PSQL etc, this has made Procedure-SQLs fall into same situation as ORM-QLs where there’s EJBQL, JDOQL, HQL etc. But Procedure-SQLs are for stored procedures, while SQL / ORM-QLs are for Query. Do you mean you will write your stored procedures with ORM-QLs eventually ?
    Just write them with Java ;-) .. that's why ORMs are popular. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  142. Re: Missing the point[ Go to top ]

    Just write them with Java ;-) .. that's why ORMs are popular.
    I have no objection to building business logic with Java, but I don't see significant relationship between this and the popularity/fade of ORMs.
  143. Re: Missing the point[ Go to top ]

    Just write them with Java ;-) .. that's why ORMs are popular.


    I have no objection to building business logic with Java, but I don't see significant relationship between this and the popularity/fade of ORMs.
    It is because ORMs allow the business logic to be more fully in Java, and less in vendor-specific SQL or stored procedures. ORMs (especially with transparent persistence) allow a clean separation of persistence from business logic.
  144. Re: Missing the point[ Go to top ]

    It is because ORMs allow the business logic to be more fully in Java, and less in vendor-specific SQL or stored procedures.
    Nonsense. Keeping the business logic "more fully in Java" is a function of the developer. It has nothing to do with ORM.
  145. Re: Missing the point[ Go to top ]

    It is because ORMs allow the business logic to be more fully in Java, and less in vendor-specific SQL or stored procedures.


    Nonsense. Keeping the business logic "more fully in Java" is a function of the developer. It has nothing to do with ORM.
    It has a lot to do with ORM (or rather with good modern ORMs). You can express business logic almost entirely in pure Java. With transparent persistence, you can express changes to those objects and retrieve those objects entirely in Java. You can even apply persistence to legacy code and objects. Without ORM, or with a poorly designed ORM, you have to mix persistence logic with business logic.
  146. Re: Missing the point[ Go to top ]

    ...Just write them with Java ;-) .. that's why ORMs are popular.
    ORMs are popular because organizations need a way to reign-in the majority of developers who don't have a clue on how to manage data efficiently.
  147. Re: Missing the point[ Go to top ]

    ...Just write them with Java ;-) .. that's why ORMs are popular.


    ORMs are popular because organizations need a way to reign-in the majority of developers who don't have a clue on how to manage data efficiently.
    Sorry, but I don't see how 'having a clue on how to manage data efficiently' in any way helps with issues like query portability, or clean separation of persistence logic and business logic.
  148. Re: Missing the point[ Go to top ]

    Sorry, but I don't see how 'having a clue on how to manage data efficiently' in any way helps with issues like query portability, or clean separation of persistence logic and business logic.
    You're one of the majority...don't feel bad.
  149. Re: Missing the point[ Go to top ]

    Sorry, but I don't see how 'having a clue on how to manage data efficiently' in any way helps with issues like query portability, or clean separation of persistence logic and business logic.


    You're one of the majority...don't feel bad.
    I thought that might be the reason. Perhaps you could educate me. Please explain how 'managing data efficiently' helps deal with, for example, the issue of SQL portability. I would be interested to know how such knowledge would enable me to write stored procedures in Oracle that could work in PostgreSQL. Or perhaps you could explain the issues of lack of SQL standards compliance in major databases could also be solved by 'managing data efficiently'? I am sure there are many of us (in fact, the majority) who would benefit from this knowledge.
  150. Re: Missing the point[ Go to top ]

    These ORM-QLs aren't redundant. They have the major benefit of real portability.

    If it is really not redundant. You should say "ORM-QL is not redundant" instead of "ORM-QLs aren't redundant". We have remade all the wheels one by one, how can you still claim that they aren't redundant ?

    You have misunderstood "real portability" also. The elegance of SQL is learn once use everywhere (VB/Java/.Net/Console ...). This is skill-portability & resource- portability. Do ORM-QLs offer such portability ?

    I find it hard to believe that a DBA or non-Java developer who can deal with all the various dialects of SQL

    I believe that most developers (DB/JAVA/VB/.Net) can deal with SQL much better than various dialects ORM-QLs.

    especially in the context of stored procedures, would have any trouble at all with EJBQL or HQL.


    You have perfectly mix up SQL and Procedure-SQL. It is true that there are some kinds of Procedure-SQLs such as TSQL, PL/SQL, PSQL etc, this has made Procedure-SQLs fall into same situation as ORM-QLs where there’s EJBQL, JDOQL, HQL etc. But Procedure-SQLs are for stored procedures, while SQL / ORM-QLs are for Query. Do you mean you will write your stored procedures with ORM-QLs eventually ?
    Again you forget something : Programming languages don't interpret SQL so of course it's usable by any language. They just build a query string and spit it out to the DB because the Database doesn't care about your program design. But how do you adress OO concepts using pure SQL? How do you map table fields to objects? Hint : manually or using a result mapper like iBatis. Now if you want a true ORM tool that does the mapping automatically for any queries, you need a query language which is interpreted by your programming language so table fields can be mapped to objects properties based on their origins. Therefore ORM-QL languages are less portable because you need an interpreter. Once again, ORM-QL languages aren't a fork of SQL.... They weren't invented just for the sake of portability. Although it's a very welcome side effect.
  151. Re: Missing the point[ Go to top ]

    Hint : manually or using a result mapper like iBatis.
    Right! A good library on top of jdbc is good enough. "Simplicity is the biggest advantage" -- from ibatis.org
  152. Re: Missing the point[ Go to top ]

    Hint : manually or using a result mapper like iBatis.


    Right!

    A good library on top of jdbc is good enough. "Simplicity is the biggest advantage" -- from ibatis.org
    True in some cases but in some other cases I really prefer an ORM framework.
  153. Re: Missing the point[ Go to top ]

    Hint : manually or using a result mapper like iBatis.


    Right!

    A good library on top of jdbc is good enough. "Simplicity is the biggest advantage" -- from ibatis.org
    Depends. A good library on top of JDBC certainly does not simplify things if you want both high-performance querying and portability. I think this is confusing 'working at a low level' with 'simplicity'. A quality ORM (such as good JDO 2.0 implementation) is what I would call simple.