No ORM: The simplest solution?

Blogs

Blogs: No ORM: The simplest solution?

  1. No ORM: The simplest solution? (117 messages)

    In this blog entry, Angsuman Chakraborty, founder and CEO of software development firm Taragana, Inc. and self-declared ORM-free developer, describes the "framework" solution that he claims puts him at his most productive. Chakraborty believes that ORM frameworks "have a steep learning curve and tend to shoe horn your architecture into their model." As a SQL-happy developer, he uses just two facilities over JDBC.
    1. I needed a way to put all the SQL queries and DDL's in a separate file. This allows me or a DBA to later analyze the query with a fine tooth comb and optimize if necessary. It also allows me to easily change them without changing the code. Most of all cleanliness of the solution is appealing. At this point I am sure you are thinking of iBatis. I tried iBatis. Initially I liked it and thought I had my solution. However as I went down the lane I realized it too gave me features that I didn't need. Even this was more complicated than I needed. All I needed was a HashMap saved to a file in XML format. And my database class should support query execution by name (think key-value). 2. Secondly I needed connection pooling to prevent opening and closing too many connections and also running out of connections. I found a nice solution in Proxool. Additionally it supports having multiple connection profiles in a simple text file. 3. I needed to integrate these two capabilities in a simple database class along with utility methods like cleanly closing connection and logging the query data.
    What do you think of this bare-bones approach? Are you bypassing or embracing ORM?

    Threaded Messages (117)

  2. Re: No ORM: The simplest solution?[ Go to top ]

    What do you think of this bare-bones approach? Are you bypassing or embracing ORM?
    At the moment I am embracing ORM with enthusiasm. Far from "shoe-horning me into a model", it allows me to concentrate on pure and clean OOP development, rather than having to think about database interfaces and SQL. I don't need to store SQL in a separate file so that it can be changed because I use a portable query language that works without change on all the databases I want to use (of course, the ORM allows me to externalise SQL if I want). I don't need to worry about connection pooling - the ORM manages this for me. The performance is excellent, and the portability is amazing. A couple of days ago I migrated a substantial application from PostgreSQL to Oracle. All that was needed to change two lines in a mapping file. The application ran fine, passing all tests. How much cleaner can you get?
  3. ORM's are a overkill[ Go to top ]

    1> ORMs are really a overblown solution to a small problem which has been given a mouthfull name - 'OR Impedence Mismatch' I guess plain JDBC provides all what we need to access database. I have seen through my experience that unless we write horrible alogorithms in our Java business tier, 95% of performance issues are ultimately traced back to data access layer. To be very specific to the SQLs that we write. Let me put it in a different way, J2EE/Java developers hate or are generally ignorant to SQLs, and treat databases as black boxes. Hence we pay the price with poor SQLs leading to performance clog. I believe to hide this deficiency in J2EE developers we have this overblown solution called ORM. I prefer more close control over my SQLs and love something like a Spring JDBC but not ORM. 2> I strongly believe that the buzzword of portability is really funny. I can bet on the fact that 95% of the cases projects dont change their databases. So there is no question of portability.So if Entity Beans were bad i guess ORMs are equally bad.
  4. I hear this view a lot - data is tabular so get used to it ;-) Maybe but applying business using beans is a lot easier than ResultSets! So let's not take this too far. The major problems of having SQL in your code are: 1) The queries cannot be shared amongst other applications; a real issue as you move beyond noddy apps. 2) The queries cannot be tuned by a DBA - they frequently never see them until after go live. 3) Schema changes really hurt and impact analysis is not easy I prefer simple stored procedures to alleviate this problem. This gets away from SQL in the code, maintains control and hides the schema from the domain objects. Using iBatis to access stored procedures is a neat way to unify queries and also introduce declarative caching. A rowset might do just as well although iBatis provides better stored procedures mapping. Of course Spring pulls it all together and manages the transactions. I see a tremendous reduction in code; a good separation of concerns and plenty of scope for optimizations / refactoring. ray
  5. 1) The queries cannot be shared amongst other applications; a real issue as you move beyond noddy apps.

    Why share queries but not reuse the API using the query?
  6. The major problems of having SQL in your code are: 1) The queries cannot be shared amongst other applications; a real issue as you move beyond noddy apps. 2) The queries cannot be tuned by a DBA - they frequently never see them until after go live. 3) Schema changes really hurt and impact analysis is not easy
    1) As someone else already mentioned, you shouldn't have to share queries. You should use a SOA and reuse the business service. If you are not doing this, then duplicating the SQL is the least of your problems. 2) Queries CAN be tuned by the DBA. To do so requires a close working relationship between app dev teams and the DBA team. I have done this on both small and large application projects and it REALLY WORKS WELL. 3) Schema changes have the same impact, whether the SQL is imbedded in the app or is written in stored procedures. In the case of embedded SQL it does require a better relationship between the app teams and the DBA. Also, if the application DAO code is well written, it will minimize the application impact.
    I prefer simple stored procedures to alleviate this problem. This gets away from SQL in the code, maintains control and hides the schema from the domain objects.
    This blog is about ORM, so I'll try to be very brief about stored procedures. In short, they do not scale well, they tend to be bloated with business logic (instead of encapsulating in the code), and they don't perform well (despite what most DBAs claim). The worse thing about stored procs is that they promote division between app teams and DBAs. I wrote a white paper within my company based on my own research on this topic. Stored procs are great for creating DBA tools, but are a very poor application container.
  7. Re: No ORM: The simplest solution?[ Go to top ]

    I want everything but the ORM. I want connection pooling, sql generation, cursors, db independence, preparing statements, dynamic finding, caching, transaction control, etc. Except I don't want to map tables to objects. I want to have a helper or factory object do it for me. I want a pure pojo domain object (no annotations!) and I want to get rid of DTOs. But most of all, I don't want to use HQL or XML or a GUI. I'd rather use handwritten SQL (OCR'd in if need be) than any of those.
  8. Re: No ORM: The simplest solution?[ Go to top ]

    I want everything but the ORM.
    Unless you have taken a vow to not touch any ORM :), you may check out JDX OR-Mapper that should meet your needs well. connection pooling – yes (JDX has a sophisticated self-healing connection pooling facility) sql generation – yes (JDX generates all the necessary SQL dynamically) cursors – yes (through object-streaming) db independence – yes (JDX supports most popular databases including Oracle, SQL Server, DB2, MySQL, and Sybase) preparing statements – yes (JDX implicitly uses prepared statements for optimal performance) dynamic finding – yes (JDX supports named queries with runtime binding of parameters as well as dynamically generated predicates) caching – yes (for read-only objects) transaction control – yes (JDX allows you to bunch multiple operations in one transaction; it can also work with external transaction managers)
    Except I don't want to map tables to objects. I want to have a helper or factory object do it for me.
    OR-Mapping specification for JDX is defined declaratively in a simple text format. There is no XML used. Mapping can come from a file or from the database. Such mapping can also be generated through JDX’s reverse-engineering tool (available through command line as well as GUI).
    pure pojo domain object (no annotations!)
    JDX supports pure pojo domain objects without any annotations.
    I want to get rid of DTOs.
    Sure, domain model pojo’s themselves can be used for data transfer.
    But most of all, I don't want to use HQL or XML or a GUI.
    JDX does not impose any query language. You can express query predicates in the familiar SQL syntax with the added facility of using field names (vs. column names) and path expressions in them. JDX does not require XML for anything. GUI is optional. However, you may take advantage of the GUI tools for forward-engineering, reverse-engineering, and mapping verification.
    I'd rather use handwritten SQL (OCR'd in if need be) than any of those.
    You may easily use the WHERE clause of your handwritten SQLs for queries. You may also use the SQL pass-thru mechanism to execute any arbitrary SQL statements through JDX. JDX is simple, non-intrusive, and flexible. It is lightweight too – the runtime jar is less than 420 KB and does not have any external dependencies on any other libraries except for the JDBC driver and the JVM. A free evaluation version of JDX OR-Mapper is available at http://www.softwaretree.com. -- Damodar Periwal Software Tree, Inc. Simplify Data Integration
  9. I agree with the poster. For simple to medium web apps with no complex transactions this is better (lighterweigh, but sufficient) approach than full blown ORM framework. You can also add support for single phase transactions by just putting transactional context (JDBC connection in this case) in thread local variable. Also usefull is generic DAO support with generic CRUD methods and spring like API with callbacks. This is all simple to build. We also had similar approach on some projects (in Java and .NET) and it is very productive.
  10. And this approach is no less OO then using an ORM mapper. You can still (and should) have domain model, although it is not transactional. If your transactions are simple, unit-of-work that ORM provides is just an overhead and major bloat. Adding support for caching is also relatively easy with this approach.
  11. Of course, if your objects are involved in complex transactions, that is far beyond simple CRUD, ORM with unit-of-work shines.
  12. Re: No ORM: The simplest solution?[ Go to top ]

    And this approach is no less OO then using an ORM mapper. You can still (and should) have domain model, although it is not transactional.

    If your transactions are simple, unit-of-work that ORM provides is just an overhead and major bloat.

    Adding support for caching is also relatively easy with this approach.
    All I say is that my experience is the opposite. This sounds like bloat to me. You are having to add support for cacheing yourself. ORM does this for you. You have to add your own support for transactions. ORM does this for you. You have to write your own DAO objects with CRUD methods. ORM does this for you. It may be simple to build, but why bother? For me, lightweight is having to write less code because an tried-and-tested ORM product does things for me. I think this really comes down to personal preference. If you are experienced with a particular ORM, then you will probably find development using that ORM light and fast, but there can be a steep learning curve getting to know a product well. Also, I would definitely agree with Erik - use a mature ORM with good support, or you could end up wasting time on matters such as performance (and then the ORM will seem unwieldy and a source of overhead). The one thing I would say is not just a matter of experience and opinion is that the non-ORM approach is not as OO - ORMs usually allow easily use of full OOP features such as inheritance, in contrast to other approaches.
  13. Re: No ORM: The simplest solution?[ Go to top ]

    For me, lightweight is having to write less code because an tried-and-tested ORM product does things for me.
    Major impact to productivity with full blown ORM with unit-of-work (especially in stateless environments) is that your object had to be 'connected' in some way to ORM framework. Either it is Hibernate Session, Persistence Manager or other. Disconnecting and reconnecting to for example Hibernate Session is rather pain in statless environments. No meter how authors of ORM framework try to make it easy, it is still pain, because of the nature of the problem (having to support unit-of-work, lazy load and such stuff in transparent way). It also affects performance, not much, but mesurabely.
    The one thing I would say is not just a matter of experience and opinion is that the non-ORM approach is not as OO - ORMs usually allow easily use of full OOP features such as inheritance, in contrast to other approaches.
    This simply is not true. You can design your domain model without ORM even more freely and with less constraints then with ORM! Modern ORM does have constraints on your POJOs, small and not worthy constrains, but they have. Try declaring final getter for association property and see if it can be lazy loaded with Hibernate.
  14. Re: No ORM: The simplest solution?[ Go to top ]

    For me, lightweight is having to write less code because an tried-and-tested ORM product does things for me.

    Major impact to productivity with full blown ORM with unit-of-work (especially in stateless environments) is that your object had to be 'connected' in some way to ORM framework. Either it is Hibernate Session, Persistence Manager or other. Disconnecting and reconnecting to for example Hibernate Session is rather pain in statless environments. No meter how authors of ORM framework try to make it easy, it is still pain, because of the nature of the problem (having to support unit-of-work, lazy load and such stuff in transparent way).
    I really don't believe this is a major impact to productivity. You don't have to keep objects connected and you don't need to use lazy loading - for example, in JDO, it is trivial to fetch an object and the immediately make it 'transient' along will any associated objects fetched along with it (or you can 'detach' for later possible re-persisting). You can use as much or as little transparent persistence as you wish, and you can persist and retrieve objects without any use of transactions. You can use ORM pretty much statelessly.
    The one thing I would say is not just a matter of experience and opinion is that the non-ORM approach is not as OO - ORMs usually allow easily use of full OOP features such as inheritance, in contrast to other approaches.


    This simply is not true. You can design your domain model without ORM even more freely and with less constraints then with ORM! Modern ORM does have constraints on your POJOs, small and not worthy constrains, but they have. Try declaring final getter for association property and see if it can be lazy loaded with Hibernate.
    My point was that you have to, for example, implement mechanisms for storage of subclasses in relational systems yourself. With ORMs this can be largely automatic. In JDO I need only list the subclass in a .JDO file, and that is it. Also, just because Hibernate has certain constraints on the properties of POJOs does not mean that this is true for ORMs in general. JDO 2.0 has very few constraints indeed. When there are such constraints (such as the requirement for a no-args constructor - which can be private) they can usually be implemented, if necessary, transparently by the ORM product.
  15. Re: No ORM: The simplest solution?[ Go to top ]

    JDO 2.0 has very few constraints indeed. When there are such constraints (such as the requirement for a no-args constructor - which can be private) they can usually be implemented, if necessary, transparently by the ORM product.
    Indeed "few" can be reduced to only one constraint, or no constraints at all since implementations do that for you. JDO is a true POJO persistence API, not JPA, Hibernate and others.
  16. Re: No ORM: The simplest solution?[ Go to top ]

    Indeed "few" can be reduced to only one constraint, or no constraints at all since implementations do that for you.

    JDO is a true POJO persistence API, not JPA, Hibernate and others.
    Well, not quite. For example, in Collections and Maps where you want to persist the elements as separate objects rather than in a BLOB the elements have to be either persistent or immutable types such as String or Date. But this is far better than the 'entities only' restriction of JPA. However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.
  17. Re: No ORM: The simplest solution?[ Go to top ]

    Indeed "few" can be reduced to only one constraint, or no constraints at all since implementations do that for you.

    JDO is a true POJO persistence API, not JPA, Hibernate and others.


    Well, not quite. For example, in Collections and Maps where you want to persist the elements as separate objects rather than in a BLOB the elements have to be either persistent or immutable types such as String or Date. But this is far better than the 'entities only' restriction of JPA.

    However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.
    Col. elements do not have to be immutable neither persistent classes. Theorically, JDO can persist any type mapped to any data type and be able to query them, but will depend on how flexible is the implementation. The only major common scenario not supported in JDO impls today is the capability of persisting collections of collections by not serializing them. JPA should be enough for most users, however people should be aware of its limitations otherwise can get frustated with ORM in general
  18. Re: No ORM: The simplest solution?[ Go to top ]

    Col. elements do not have to be immutable neither persistent classes.
    They don't? I had assumed that the JDO 2.0 spec was pretty clear about this (section 6.4 of the spec) - only primitive, persistent or immutable types are guaranteed to be persisted if they are fields of persistence classes or elements of Collection, Set, Map etc. But then I may not be up to date with it. I would be interested if this is not the case, as it makes JDO 2.0 even more attractive.
    The only major common scenario not supported in JDO impls today is the capability of persisting collections of collections by not serializing them.

    JPA should be enough for most users, however people should be aware of its limitations otherwise can get frustated with ORM in general
    My view is that developers will come across limitations of JPA pretty quickly. It is surely common to want to have a persistent Map with keys which are immutable objects such as Strings rather than entities. Of course, most implementations will be able to handle more than this, but it is puzzling why the bar was set so low for the JPA spec.
  19. Re: No ORM: The simplest solution?[ Go to top ]

    However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.
    If JDBC code is well designed and structured, it's not bloat at all! Imagine this: - you have Spring like data access API with callbacks - you have transaction API that holds transactional resources (in this case JDBC connections) in thread local variables - you have connection pooling, - you have all your SQL in XML templates, from where you can fetch it, transform it (for dynamic SQL) and even make your SQL tamplates how swap when they change All of these features does not require more then couple of hundreds LOC to implement. With all this, your JDBC code is clean and concise, no bloat at all. If you add a bit of code generation, you will be more productive then with Hibernate if the project is medium to large in size. The other story is if your organisation standardize on Hibernate and have tens of projects that useses it or similar thing. I was talking about pure technical productivity of the approach.
  20. All of these features does not require more then couple of hundreds LOC to implement.
    I meant all of these features separately will need couple of hundred LOC to implement. All together a thousand+ LOC.
  21. Re: No ORM: The simplest solution?[ Go to top ]

    However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.

    If JDBC code is well designed and structured, it's not bloat at all! Imagine this:
    - you have Spring like data access API with callbacks
    - you have transaction API that holds transactional resources (in this case JDBC connections) in thread local variables
    - you have connection pooling,
    - you have all your SQL in XML templates, from where you can fetch it, transform it (for dynamic SQL) and even make your SQL tamplates how swap when they change

    All of these features does not require more then couple of hundreds LOC to implement.

    With all this, your JDBC code is clean and concise, no bloat at all. If you add a bit of code generation, you will be more productive then with Hibernate if the project is medium to large in size.

    The other story is if your organisation standardize on Hibernate and have tens of projects that useses it or similar thing. I was talking about pure technical productivity of the approach.
    Writing a considerable number of extra lines to re-implement something that is already provided and available in existing packages via a concise API is the definition of bloat. Modern ORMs may have a considerable number of features, but this no way means that code that uses them is bloated. All the things you mention are already present in a quality ORM. For example, JDO 2.0 implementations have callbacks, has a transaction API that can hold connections in thread local variables, connection pooling, the option for SQL and JDO queries to be stored in external XML files, and both SQL and JDOQL can be constructed dynamically. There is not single feature you have mentioned that is not already provided by any quality ORM, and I cannot think of a single example of how explicit JDBC code can be more concise than the transparent object persistence and retrieval of modern ORMs. It is surely technically vastly more productive to use a product that has been well-designed and quality tested and has thousands of other users than to write all this yourself, aside from the possible future support issues that may arise when others attempt to deal with your 'home grown' solution. As your project grows there will be an increasing number of issues such as clustering and cacheing that you will need to deal with. There is no point at all in dealing with this yourself.
  22. Re: No ORM: The simplest solution?[ Go to top ]

    Writing a considerable number of extra lines to re-implement something that is already provided and available in existing packages via a concise API is the definition of bloat.
    It is not big deal to wright all this stuff as you might think. For me, major bloat is seeing one hundred+ deep stack traces when using several popular frameworks together (ORM + container + web framework for example).
    Modern ORMs may have a considerable number of features, but this no way means that code that uses them is bloated.
    Sure they have, but these features are far beyond what average single project needs. That's my point, people end using something that they do not need for the specific project, adding complexity, dependency on third party feamework and surely pay performance price (sometimes insignificant, but sometimes mesurable).
    It is surely technically vastly more productive to use a product that has been well-designed and quality tested and has thousands of other users than to write all this yourself, aside from the possible future support issues that may arise when others attempt to deal with your 'home grown' solution. As your project grows there will be an increasing number of issues such as clustering and cacheing that you will need to deal with. There is no point at all in dealing with this yourself.
    Sure modern Java ORMs are high quality. Sure there are a lot of resources around about them. Sure a lot of people are familiar with them. But sometimes (more often then you might think), they are not needed at all and just can make trouble: In one of the latest projects we used Hibernate. It was a medium size web app with a lot of need for state management. We used JSF to ease state management in the presentation layer. The result: we ended seeing Hibernate and JSF fighted each other as ORM unit-of-work was not 'compatible' with state management in JSF (connecting/disconnecting objects from ORM is a pain, at least with Hibnernate). Not that API is complex, on the contrary, but there is an 'impedance mismatch'. A lot of pepole had the same problem. At the end we had to build ourselves support for 'long session' and Seam like conversations (Seam was in early beta at that time). We could be much more productive in that project without using ORM... And the irony was that I made a decision to use Hibernate :( Hibernate is good. Hibernate is exelent. But for a simple web app (not small, not big, but simple) it is just an overhead.
  23. Re: No ORM: The simplest solution?[ Go to top ]

    Writing a considerable number of extra lines to re-implement something that is already provided and available in existing packages via a concise API is the definition of bloat.

    It is not big deal to wright all this stuff as you might think. For me, major bloat is seeing one hundred+ deep stack traces when using several popular frameworks together (ORM + container + web framework for example).
    In any web framework, you are dealing with deep stack frameworks. You get equal 'bloat' if you are dealing with traces into your own ORM implementation or if you are dealing with a standard ORM. The difference with a standard ORM is that someone else will have had to deal with the same issue. The depth of the stack trace is of no consequence - what matters is what has happened in your code.
    Modern ORMs may have a considerable number of features, but this no way means that code that uses them is bloated.


    Sure they have, but these features are far beyond what average single project needs. That's my point, people end using something that they do not need for the specific project, adding complexity, dependency on third party feamework and surely pay performance price (sometimes insignificant, but sometimes mesurable).
    If people are using additional features then either you don't have the appropriate control over what they use, or they require more than your ORM implementation provides. I still don't get why you think that reliance on third partly frameworks implies performance loss. When I use a third party framework, I am honest enough to accept that the writers of that framework might have written better, more finely tuned, code that I could have produced.
    It is surely technically vastly more productive to use a product that has been well-designed and quality tested and has thousands of other users than to write all this yourself, aside from the possible future support issues that may arise when others attempt to deal with your 'home grown' solution. As your project grows there will be an increasing number of issues such as clustering and cacheing that you will need to deal with. There is no point at all in dealing with this yourself.


    Sure modern Java ORMs are high quality. Sure there are a lot of resources around about them. Sure a lot of people are familiar with them. But sometimes (more often then you might think), they are not needed at all and just can make trouble:

    In one of the latest projects we used Hibernate. It was a medium size web app with a lot of need for state management. We used JSF to ease state management in the presentation layer. The result: we ended seeing Hibernate and JSF fighted each other as ORM unit-of-work was not 'compatible' with state management in JSF (connecting/disconnecting objects from ORM is a pain, at least with Hibnernate). Not that API is complex, on the contrary, but there is an 'impedance mismatch'. A lot of pepole had the same problem. At the end we had to build ourselves support for 'long session' and Seam like conversations (Seam was in early beta at that time). We could be much more productive in that project without using ORM...

    And the irony was that I made a decision to use Hibernate :(
    Hibernate is good. Hibernate is exelent. But for a simple web app (not small, not big, but simple) it is just an overhead.
    I am afraid that this suggests you have not sufficiently researched how ORMs can integrate with web frameworks, but I can understand your position - this is not a simple process. Seam is a new solution, but there are others. I have used the JSF integration with Spring and JDO with excellent results. Now that I have this integration working, it certainly isn't an overhead, it is a re-useable approach that simplifies things for new projects. May I add one further comment? You persist [sic] in mentioning Hibernate in the context of ORM. Although Hibernate is very popular and is of good quality, Java persistence is more than Hibernate. You should investigate alternate JPA implementations such as TopLink, and you should look at the excellent JDO 2.0 specifications, and quality implementations - JPOX, Kodo, Xcalia.
  24. Re: No ORM: The simplest solution?[ Go to top ]

    In any web framework, you are dealing with deep stack frameworks. You get equal 'bloat' if you are dealing with traces into your own ORM implementation or if you are dealing with a standard ORM.
    I and the original poster were talking not about home grown ORM implementations, but about simple home grown data access libraries, without unit-of-work, lazy load, complex mapping support etc... These libraries occupy no more then about 5 entries in the stack trace.
    I still don't get why you think that reliance on third partly frameworks implies performance loss. When I use a third party framework, I am honest enough to accept that the writers of that framework might have written better, more finely tuned, code that I could have produced.
    Sure, but the writers or third-party framework are building general ORM solution and home grown library is targeting specific project or group of projects and thus can be much simpler and more performant. If we have 100+ deep stack traces, there must be are a lot of method invocation. Although it is very fast on modern JVMs, nothing is fast enough if it is repeating multiple times, passing various layers of abbstraction, which are, of course, required to build good designed general ORM solution, but not required at all for simple but usefull (in specific context) data acess library.
    I am afraid that this suggests you have not sufficiently researched how ORMs can integrate with web frameworks, but I can understand your position - this is not a simple process.
    Yes, I didn't have time to thorughly research integration of web frameworks with ORM, but, the time I invested in that was enough to conclude two things (I believe that a lot of pragmatic people would conclude the same, but maybe I am wrong): - There is an 'impedance missmatch' between component oriented web frameworks (like JSF, Tapestry...) and ORM and this 'impedance miscmatch' have been not addresed by any third-party solution until recently (Seam, Spring Continuations). - The amount of time someone need to research the problematics about this (integrating ORM with component based web frameworks) is order of magnitude more then writing a simple but well designed data acess library!
    Seam is a new solution, but there are others. I have used the JSF integration with Spring and JDO with excellent results. Now that I have this integration working, it certainly isn't an overhead, it is a re-useable approach that simplifies things for new projects.
    So, you have to build your own integration library to integrate JDO with Spring and JSF. Sorry but I have to ask how much time do you invested in it? Is your library usefull with other framework stacks, for example TopLink with Oracle EJB3 and ADF faces. I am sure it's not because these three are completely different beasts with thir own specific functionality, quirks and problems too, although some of them might be based on same JCP specifications. That's my point, we Java programmers are lately very obsessed with frameworks (I was too) but sometimes simple solutions are far then enough!
    You should investigate alternate JPA implementations such as TopLink, and you should look at the excellent JDO 2.0 specifications, and quality implementations - JPOX, Kodo, Xcalia.
    The time needed to investigate all this is far beyond writting my own stuff that will fit my own project needs! I believe that time is much better invested in the concepts of how these frameworks work, how they are designed, what patterns they use, and steal ideas from them, rather then investing time in how this framework integrates with that, what are the constraints of this framework and what are the quirks of n.mm version of that framework.
  25. Re: No ORM: The simplest solution?[ Go to top ]

    In any web framework, you are dealing with deep stack frameworks. You get equal 'bloat' if you are dealing with traces into your own ORM implementation or if you are dealing with a standard ORM.

    I and the original poster were talking not about home grown ORM implementations, but about simple home grown data access libraries, without unit-of-work, lazy load, complex mapping support etc... These libraries occupy no more then about 5 entries in the stack trace.
    What is the issue anyway with the size of the stack trace?
    I still don't get why you think that reliance on third partly frameworks implies performance loss. When I use a third party framework, I am honest enough to accept that the writers of that framework might have written better, more finely tuned, code that I could have produced.


    Sure, but the writers or third-party framework are building general ORM solution and home grown library is targeting specific project or group of projects and thus can be much simpler and more performant.
    I disagree. ORMs are usually written to have a range of features and capabilities. This does not mean that any of these individual features or capabilities is less performant or complex to use. I use an ORM products specifically because it is far simpler and much more performant that anything I could code by hand - my code is simple and the ORM handles cacheing of objects and the generation of optimised SQL.
    If we have 100+ deep stack traces, there must be are a lot of method invocation. Although it is very fast on modern JVMs, nothing is fast enough if it is repeating multiple times, passing various layers of abbstraction, which are, of course, required to build good designed general ORM solution, but not required at all for simple but usefull (in specific context) data acess library.
    The depth of method invocation is not a measure of performance, and neither is the level of abstraction.
    So, you have to build your own integration library to integrate JDO with Spring and JSF. Sorry but I have to ask how much time do you invested in it? Is your library usefull with other framework stacks, for example TopLink with Oracle EJB3 and ADF faces. I am sure it's not because these three are completely different beasts with thir own specific functionality, quirks and problems too, although some of them might be based on same JCP specifications.
    Of course I haven't built my own integration library. I have simply used what is already available in Spring. Spring also supports the other frameworks you mention.
    That's my point, we Java programmers are lately very obsessed with frameworks (I was too) but sometimes simple solutions are far then enough!
    But you haven't used a simple solution. You have had to implement a range of things yourself. The simple solution is to use a product that provides these.
    You should investigate alternate JPA implementations such as TopLink, and you should look at the excellent JDO 2.0 specifications, and quality implementations - JPOX, Kodo, Xcalia.


    The time needed to investigate all this is far beyond writting my own stuff that will fit my own project needs!
    Surely it is part of the role of a professional developer to keep up-to-date with such things.
    I believe that time is much better invested in the concepts of how these frameworks work, how they are designed, what patterns they use, and steal ideas from them, rather then investing time in how this framework integrates with that, what are the constraints of this framework and what are the quirks of n.mm version of that framework.
    I think this is fundamentally mistaken. The result of this attitude would be that we would end up with endless re-inventions of the same thing, all with their own quirks and all with their own integration and support issues. Far better to pick a quality API, such as JDO 2.0 or JPA, and get to know it well. Then you can quickly find an implementation that suits your requirements and you will find that all the work of integration is done for you with products like Spring.
  26. Re: No ORM: The simplest solution?[ Go to top ]

    Of course I haven't built my own integration library. I have simply used what is already available in Spring. Spring also supports the other frameworks you mention.
    Well, Spring support for JSF, at least in version 1.xx, is very, very limited. You can just inject Spring beans in JSF managed beans' properties. To fully suport JSF, there must be the trasnsparence in usage of Spring and JSF managed beans (seeing JSF managed beens in Spring as Spring beans and vice-versa) as well as support for aditional contexts (request, session comes to mind, but also conversation and process scope would be usefull). Spring-JSF and Spring 2.0 have some of that but they are not production ready.
    But you haven't used a simple solution. You have had to implement a range of things yourself. The simple solution is to use a product that provides these.
    No, I have used simper solution :). Product that you use may have steep learning curve, thick documentation that talk about features your project does not need, has relatively complex setup, frequent maintenance release schedule which you have to be aware of and such things. There is a lot of time and work associated with that. If your project does not need advanced ORM features like unit-of-work, clustered caching or transparent lazy-loading then it is an overhead. I can happily call Save methods on my DAOs instead of having unit-of-work, I can happily have simple caching of objects (easy to implement if you have domain model), I can happily code lazy-load in my domain model by hand or code generator. Instead of writing mapping declaration in XML file or whatever is used, I can have two callback methods, one that binds object to the prepared statement and the other that reads it from the ResultSet and that's where my mapping is specified. If I choose to go with code generation, then my mapping is in metadata (or DSL if you want to call it that way). So, business code is not more complex then with full blown ORM, but the system is lightweighter.
    Surely it is part of the role of a professional developer to keep up-to-date with such things.
    OK, the part of the role of a professional developer is to keep up-to-date with such things, to use them when he/her needs them and not to use when can get job done without them :)
    I think this is fundamentally mistaken. The result of this attitude would be that we would end up with endless re-inventions of the same thing, all with their own quirks and all with their own integration and support issues.
    Yes, but 'my' quirks and 'my' integration and 'my' support is simpler as 'my' stuff is built for 'my' project needs! I do not have to provide general solution. 'My' library is designed to meet 'my' project specific needs, no more, no less.

    Far better to pick a quality API, such as JDO 2.0 or JPA, and get to know it well. Then you can quickly find an implementation that suits your requirements and you will find that all the work of integration is done for you with products like Spring. Uh, Spring integration with various technologies is broad but often is lacking depth. JSF is most painfull example.
  27. Re: No ORM: The simplest solution?[ Go to top ]

    Spring-JSF and Spring 2.0 have some of that but they are not production ready.
    I am using Spring-JSF in production.
    But you haven't used a simple solution. You have had to implement a range of things yourself. The simple solution is to use a product that provides these.

    No, I have used simper solution :). Product that you use may have steep learning curve, thick documentation that talk about features your project does not need, has relatively complex setup, frequent maintenance release schedule which you have to be aware of and such things. There is a lot of time and work associated with that. If your project does not need advanced ORM features like unit-of-work, clustered caching or transparent lazy-loading then it is an overhead.
    Oh come on. There is time and work involved in skipping the bits you don't need in the documentation? I really can't believe that there is more time involved in researching a relatively simple API like JDO, as against writing all this yourself. This all sounds like stretching a point to me; a set of straw man and worst-case arguments to justify re-inventing the wheel. I really do suggest you take a look at the superb documentation on the JDO 2.0 RI site - JPOX. Any suggestion that setup is complex, or there is a steep learning curve is just plain wrong.
    I can happily call Save methods on my DAOs instead of having unit-of-work, I can happily have simple caching of objects (easy to implement if you have domain model), I can happily code lazy-load in my domain model by hand or code generator.

    Instead of writing mapping declaration in XML file or whatever is used, I can have two callback methods, one that binds object to the prepared statement and the other that reads it from the ResultSet and that's where my mapping is specified. If I choose to go with code generation, then my mapping is in metadata (or DSL if you want to call it that way). So, business code is not more complex then with full blown ORM, but the system is lightweighter.
    Yes, your business code is certainly more complex that with 'full-blown' ORM. I don't have to have any callback methods that read things from result sets. This is automatic with ORM. I don't have to write code for lazy loading, or cacheing.
    Surely it is part of the role of a professional developer to keep up-to-date with such things.


    OK, the part of the role of a professional developer is to keep up-to-date with such things, to use them when he/her needs them and not to use when can get job done without them :)

    I think this is fundamentally mistaken. The result of this attitude would be that we would end up with endless re-inventions of the same thing, all with their own quirks and all with their own integration and support issues.

    Yes, but 'my' quirks and 'my' integration and 'my' support is simpler as 'my' stuff is built for 'my' project needs!
    I do not have to provide general solution. 'My' library is designed to meet 'my' project specific needs, no more, no less.
    There is the assumption that your project is 'special' in some way, that you are doing something unique that has not been done before. All it usually takes is a quick hunt through the public forums of ORM products to see that this is not the case. And, of course, your work is wasted when you move to a new project which you assume has different needs....
  28. Re: No ORM: The simplest solution?[ Go to top ]

    I am using Spring-JSF in production.
    I am just curious, how do you handle session state in your web app? Do you keep JDO Persistent Manager per HTTP session or you disconnect and reconnect object when storing them in HTTP session? Either way you have and overhead that is not necesary if your app does not need unit-of-work. And it does not need it if it has simple transactions involving at most couple of objects.
    I really can't believe that there is more time involved in researching a relatively simple API like JDO, as against writing all this yourself.
    JDO API may be simple if you look at the JDO JavaDoc. But if you look at the JDO Spec and JDO Vendors Documentation it starts to be quite complex (I am not saying dificult). And I reapeat, someone does not need to reimplement all this, just the stuff that's needed in the specific project and this is often quite a bit.
    Yes, your business code is certainly more complex that with 'full-blown' ORM. I don't have to have any callback methods that read things from result sets.
    But you have to specify mappings. Specifiying mapping in an XML file and writing for example just two callback methods described earlier in a generic DAO calss is merely the same task. Note that only callback methods have to be implemented, as DAO could be generic and inherit for example BaseDAO which could have CRUD methods and uses callback as strategies)
    There is the assumption that your project is 'special' in some way,
    Yes, that's the point. Every project is special in some way.
    that you are doing something unique that has not been done before.
    No, I'm not. But, I am doing the same thing in different context. I can reuse patterns and techniques, but reusing freameworks is somewhat harder. Patterns and techniques can be adapted to specific context easier, but frameworks usualy could not. So frameworks end up trying to be as general as they can, increasing overhead and complexity by definition.
    And, of course, your work is wasted when you move to a new project which you assume has different needs....
    No, it's not :). I've learned patterns, I've learned techniques. And projects could have simmilar, yet different needs. Similarity yet difference is much better handled with patterns and techniques then with frameworks. We have a homegrown data access library written jn Java (I've described the features in previous posts). We recently started a project that uses .NET as this was requested so. We ported our data access library to .NET within a day. Java specific stuff was removed and .NET specific stuff was started to be added as the project needs dictated. We do not have a need for ORM in our project, although we have domain model and mid-complex transactions involving several objects. Our business domain model spans nearly 100 objects but we handle repetition with code generation. We kept dependency on third party libraries minimal. We do not need unit-of-work and clustered caching in this project. We need lazy load, but that is handled with code generation. We are happy with our approach. We do not have to search forums when we hit the wall. We can tune our code to meet expected performance requirements. We are not reinventing the wheel, we are just making the wheel as we have seen how others are doing that. And we made a wheel for our specific vehicle, to meet the best it's needs.
  29. Re: No ORM: The simplest solution?[ Go to top ]

    I am just curious, how do you handle session state in your web app? Do you keep JDO Persistent Manager per HTTP session or you disconnect and reconnect object when storing them in HTTP session? Either way you have and overhead that is not necesary if your app does not need unit-of-work. And it does not need it if it has simple transactions involving at most couple of objects.
    Maintaining persistence managers for such a length of time is not a good idea. I allow Spring to generate persistence managers for me as I need them. It does this transparently. Where is the overhead in using single method calls to disconnect and reconnect objects? Using JDO to do this is simple. It is concise. The code you need is minimal.
    JDO API may be simple if you look at the JDO JavaDoc. But if you look at the JDO Spec and JDO Vendors Documentation it starts to be quite complex (I am not saying dificult). And I reapeat, someone does not need to reimplement
    You only need to read complex parts of a vendors documentation of you are doing complex things.
    But you have to specify mappings.
    Actually, for simple projects, you don't, or at least only minimally. You can let the vendors tools generate schemas or reverse-map them for you.
    Specifiying mapping in an XML file and writing for example just two callback methods described earlier in a generic DAO calss is merely the same task. Note that only callback methods have to be implemented, as DAO could be generic and inherit for example BaseDAO which could have CRUD methods and uses callback as strategies)
    But there you go again - mentioning things that you have to do that are already provided by a product! Why should I have to generate a BaseDAO, implement the CRUD methods, and the callbacks? I really can't see the point.
    No, I'm not. But, I am doing the same thing in different context. I can reuse patterns and techniques, but reusing freameworks is somewhat harder. Patterns and techniques can be adapted to specific context easier, but frameworks usualy could not. So frameworks end up trying to be as general as they can, increasing overhead and complexity by definition.
    No, this isn't true. Object-relational mapping is very well established and specifications like JDO 2.0 have been based on years of experience of specific use cases, and not just generalities.
    No, it's not :). I've learned patterns, I've learned techniques. And projects could have simmilar, yet different needs. Similarity yet difference is much better handled with patterns and techniques then with frameworks.
    Again, I think you are wrong here.
    We have a homegrown data access library written jn Java (I've described the features in previous posts). We recently started a project that uses .NET as this was requested so. We ported our data access library to .NET within a day.
    And what was the point, when products like NHibernate are already there, and will handle far more use cases that your code? And now you have to handle support issues with your home-grown framework on two platforms.
    We do not need unit-of-work and clustered caching in this project.
    So what happens when you do? Are you going to write that yourself as well?
    We need lazy load, but that is handled with code generation.
    Again, this is already handled by existing products.
    We are happy with our approach. We do not have to search forums when we hit the wall. We can tune our code to meet expected performance requirements.
    As against obtaining a product that has already dealt with such requirements.
    We are not reinventing the wheel, we are just making the wheel as we have seen how others are doing that. And we made a wheel for our specific vehicle, to meet the best it's needs.
    You haven't re-invented the wheel - you have re-invented a whole pile of them: DAO, callbacks, lazy loading, transaction handling, performance tuning, support for multiple platforms... and if you ever have to deliver this code to a third party, they will have to learn your framework.
  30. Re: No ORM: The simplest solution?[ Go to top ]

    Maintaining persistence managers for such a length of time is not a good idea. I allow Spring to generate persistence managers for me as I need them. It does this transparently.
    Sure it is not. But if you want your UI to be statefull, you need either 'long session' or connection/disconnection of objects. If you are using ORM in a web app and do not need either, then your UI is probably stateless. If your UI is statless, then you probably do not need unit-of-work either. You then end up using ORM only to bind objects to tables (OK, there is caching too). You could do that much simpler way. If you want to use a third-party solution there is iBatis for that.
    Actually, for simple projects, you don't, or at least only minimally. You can let the vendors tools generate schemas or reverse-map them for you.
    As you said, that can work for simple projects. I would add that could work for prototyping and even not even for simple projects.
    But there you go again - mentioning things that you have to do that are already provided by a product! Why should I have to generate a BaseDAO, implement the CRUD methods, and the callbacks? I really can't see the point.
    Because you are avoiding dependency on third party product, you have better control on your system, better flexibility, better performance. I've tried ORM framework and tried no ORM framework approach. The productivty is about the same, especialy on bigger projects, where investment in custom infrastructure pays out more. What you get with automatic mapping you loose on hitting the wall, here and there, like we did trying to use ORM with statefull web UI. Completely other story is when you need ORM features that are hard to build: - transparent unit-of-work - transparent lazy-load - transparent clustered caching If your project need this, then you must go with ORM that provides this.
    And what was the point, when products like NHibernate are already there, and will handle far more use cases that your code?
    NHibernate is port of Java Hibernate 2.0 with uncertain future. I wouldn't bet my company's product on something like that!
    And now you have to handle support issues with your home-grown framework on two platforms.
    I do not have complex support issues. My library is just couple of hundred lines of code.
    So what happens when you do? Are you going to write that yourself as well?
    Then, but only then, when there is a hard-to-build feature that I need, then I would go to ORM. Sure, why not :) ?
    You haven't re-invented the wheel - you have re-invented a whole pile of them:

    DAO, callbacks, lazy loading, transaction handling, performance tuning, support for multiple platforms...
    Yes, why not? You have to do mappings, write quiries in yet another query language, configure your ORM, performance tune (lazy-load or outer join fetching, whether you will use proxies or not, the size of batch fetcing etc..) These are similar things in the amount of time needed.
    and if you ever have to deliver this code to a third party, they will have to learn your framework.
    I do no have a framework, just plain stupid and simple data access library based on some well-known patterns! And some simple DAO layer above that uses the library for which I hope is good designed and easily understood. At least by members of my team. That is enough in my context.
  31. I do not have complex support issues. My library is just couple of hundred lines of code.
    Actually, to be fair, it's a about one thousand non empty, non comment lines of code in the .NET version.
  32. Re: No ORM: The simplest solution?[ Go to top ]

    Maintaining persistence managers for such a length of time is not a good idea. I allow Spring to generate persistence managers for me as I need them. It does this transparently.

    Sure it is not. But if you want your UI to be statefull, you need either 'long session' or connection/disconnection of objects. If you are using ORM in a web app and do not need either, then your UI is probably stateless. If your UI is statless, then you probably do not need unit-of-work either. You then end up using ORM only to bind objects to tables (OK, there is caching too). You could do that much simpler way. If you want to use a third-party solution there is iBatis for that.
    But that is my point. ORM IS a simple way, even for simply binding objects to tables. My experience is that it is easy to set up, easy to use, and makes my code clean and simple to understand. Just take a look at examples on sites such as JPOX and you will see what I mean. I don't use iBatis because it does not allow me the flexibility I need. I prefer to avoid the explicit use of SQL, as this inhibits portability. As I mentioned in an earlier post on this thread, I have moved substantial applications between different database products trivially easily through the use of ORM with portable query languages.
    Actually, for simple projects, you don't, or at least only minimally. You can let the vendors tools generate schemas or reverse-map them for you.


    As you said, that can work for simple projects. I would add that could work for prototyping and even not even for simple projects.
    Well, these kinds of tools are widely used, so I would suggest you are wrong here.
    But there you go again - mentioning things that you have to do that are already provided by a product! Why should I have to generate a BaseDAO, implement the CRUD methods, and the callbacks? I really can't see the point.


    Because you are avoiding dependency on third party product, you have better control on your system, better flexibility, better performance.
    Yes, but at what stage does this stop? JDBC drivers have different qualities, and are third-party - do we bypass them and write our own socket-level interfaces to the database? Standards like JDO and JPA are a huge step forward. They recognise that development is progressing by allowing us to work at higher levels of abstraction. Just as JDBC abstracts the technical details of communication with the database, so ORM abstracts the technical details of data retrieval, cacheing, type conversions etc. I guess we each choose the level of abstraction that suits us. I choose to work at a higher level than you, and I have seen no performance or development penalty as a result - quite the reverse.
    What you get with automatic mapping you loose on hitting the wall, here and there, like we did trying to use ORM with statefull web UI.
    ORM is being used very successfully with stateful web UIs. TSS is an example. There are approaches like Seam that show that ORM and stateful web UIs are a good fit.
    And what was the point, when products like NHibernate are already there, and will handle far more use cases that your code?

    NHibernate is port of Java Hibernate 2.0 with uncertain future. I wouldn't bet my company's product on something like that!
    But it is open source, and free. There is no more risk that using your own code.
    You haven't re-invented the wheel - you have re-invented a whole pile of them:

    DAO, callbacks, lazy loading, transaction handling, performance tuning, support for multiple platforms...

    Yes, why not? You have to do mappings, write quiries in yet another query language, configure your ORM, performance tune (lazy-load or outer join fetching, whether you will use proxies or not, the size of batch fetcing etc..) These are similar things in the amount of time needed.
    No, it isn't. Because the ORM infrastructure is already there. With your approach you are having to map, write the queries, tune them, performance tune and so on and write the persistence mechanism yourself. And anyway, you are discussing the worst case. A good ORM will have reasonable defaults for performance, cacheing, lazy loading etc.
    and if you ever have to deliver this code to a third party, they will have to learn your framework.

    I do no have a framework, just plain stupid and simple data access library based on some well-known patterns! And some simple DAO layer above that uses the library for which I hope is good designed and easily understood. At least by members of my team. That is enough in my context.
    I have seen this kind of thing so often - a small solution that is only for internal use, that then grows (and grows more complex), and then has to be used by new people who have to be trained to use it and understand how it works. I consider one major benefit of standard approaches like ORMs to be a form of insurance against future requirements.
  33. Re: No ORM: The simplest solution?[ Go to top ]

    ORM is being used very successfully with stateful web UIs. TSS is an example.
    Please do not quote TSS site as a good example of ORM usage. There are a lot of better forum software out there that do not use ORM. Post preview has been implemented only recently. No post editing still. These are basic forum features.
    There are approaches like Seam that show that ORM and stateful web UIs are a good fit.
    Yes, but you need Seam for that. Yet another framework in the framework soup. Yet another dependency. Yet another framework to learn. Yet another rope to hang myself. Seam is also fresh framework I would think twice before using it in a production app.
    Yes, but at what stage does this stop? JDBC drivers have different qualities, and are third-party - do we bypass them and write our own socket-level interfaces to the database?
    JDBC is fine. Actually it is great, it is one of the most well designed and successfull Java APIs (along with Servlet API)! It is at low level enough, yet easy to use. Though you need a thin layer above to reduce the coding. But if you ever have to deal with ADO.NET for example, you will learn to appreciate JDBC even more! (ADO.NET is realy, realy very bad for anything but the grid on the form ;) )
  34. Yes, but you need Seam for that. Yet another framework in the framework soup. Yet another dependency. Yet another framework to learn. Yet another rope to hang myself. Seam is also fresh framework I would think twice before using it in a production app.
    Please do not get me wrong. Seam is great! The idea of having type of state like conversation and process is essencial in building rich UIs and having it on statless web is tremendous benfit. But Seam's area of potential usage is rather limited to building web apps with rich statefull UIs. That's not bad, actually it is good, as frameworks should focus on one particual thing and abstract it right. We, developers, have to conclude when and how to use them.
  35. Re: No ORM: The simplest solution?[ Go to top ]

    ORM is being used very successfully with stateful web UIs. TSS is an example.

    Please do not quote TSS site as a good example of ORM usage. There are a lot of better forum software out there that do not use ORM. Post preview has been implemented only recently. No post editing still. These are basic forum features.
    Features are irrelevant - I was talking about performance.
    There are approaches like Seam that show that ORM and stateful web UIs are a good fit.


    Yes, but you need Seam for that. Yet another framework in the framework soup. Yet another dependency. Yet another framework to learn. Yet another rope to hang myself. Seam is also fresh framework I would think twice before using it in a production app.
    No, you don't need Seam for that. Seam is simply something that illustrates that this works. Many of use use ORM and stateful web UIs without such assistance.
    Yes, but at what stage does this stop? JDBC drivers have different qualities, and are third-party - do we bypass them and write our own socket-level interfaces to the database?


    JDBC is fine. Actually it is great, it is one of the most well designed and successfull Java APIs (along with Servlet API)! It is at low level enough, yet easy to use. Though you need a thin layer above to reduce the coding. But if you ever have to deal with ADO.NET for example, you will learn to appreciate JDBC even more! (ADO.NET is realy, realy very bad for anything but the grid on the form ;) )
    You have not responded to my point. There is definitely a considerable variation in the quality of JDBC drivers, even for a single database product. You can't just say 'it is fine'. It is a third-party solution with bugs and variable quality. But we still use it, because to write this ourselves would be tedious and pointless. And it is very successful. I use the same argument for my use of ORM. I think in, say, 5 years time, the debate will be over and ORM will be the accepted way to interface with databases for almost any Java project. I don't think we are going to progress further in this discussion, as you ORM as having overheads and I don't. However, I am sure we are both productive. My view is that ORM will, and can, do a lot more than you seem to think it can. It can be far more finely (and easily) tuned to fit a wide variety of use cases. Given that, I see few if any cases where I would would use direct JDBC in preference, but that is just my approach.
  36. Re: No ORM: The simplest solution?[ Go to top ]

    This was interesting discussion indeed, maybe we should be back at work :) One question though, are you working for a company that is software vendor or in an IT department of a company from other industry? (My company is software vendor, but I also worked in an IT department) Maybe that is the cause why we cannot agree? Because, when we are selling a product, dependency on third-party is not at all wellcome (additional maintenance (in the eyes of the customer), possible costs (Customer: 'Oh, I have to pay for this license too!')). For an internal business app produced by company's IT depeartnemt, reliance on standardised third-party tools is, of course, very wellcome.
  37. Re: No ORM: The simplest solution?[ Go to top ]

    This was interesting discussion indeed, maybe we should be back at work :)

    One question though, are you working for a company that is software vendor or in an IT department of a company from other industry? (My company is software vendor, but I also worked in an IT department) Maybe that is the cause why we cannot agree?
    I think I am just highly keen on the use of specifications and standards at as high a level of possible. I have been down the 'write it yourself' route before, and seen others do this, and it has not always worked out well. Long experience with this has led me to want to use abstractions and stick to standards wherever possible.
    Because, when we are selling a product, dependency on third-party is not at all wellcome (additional maintenance (in the eyes of the customer), possible costs (Customer: 'Oh, I have to pay for this license too!')).

    For an internal business app produced by company's IT depeartnemt, reliance on standardised third-party tools is, of course, very wellcome.
    At the moment my coding is mainly for internal use. However, I see see good potential for use of third party frameworks even for products. This gives the customer choice. Not only would ORM allow my customer the choice of persistence product, because I ensure I conform to JSRs, I can potentially give them the choice of ORM as well. I can say 'for general use, you may wish to use this open source product. For higher performance and support, you may wish to use this commercial product. These are the configuration files you need to change, and here are the tuning parameters we recommend.' APIs like JDO 2.0 are very rich indeed, and I have found that I can write substantial software without moving beyond the spec, and with no reliance on vendor extensions (one of my concerns about JPA 1.0 is that this may not be the case). My view is that the more we move towards different parts of our applications being 'commodities', and reasonably interchangeable, the better. I am sure that many people would consider this very idealistic, but it is the way I develop, and it works for me. This explains why I am so keen on mapping approaches rather than wrapping approaches like ActiveRecord, and explains why I disagree with you.
  38. Re: No ORM: The simplest solution?[ Go to top ]

    At the moment my coding is mainly for internal use. However, I see see good potential for use of third party frameworks even for products. This gives the customer choice. Not only would ORM allow my customer the choice of persistence product, because I ensure I conform to JSRs, I can potentially give them the choice of ORM as well.
    Well, customers usualy do not care about ORMs, JRSs and such stuff, they usualy care about working software that solves their business problem, the lower cost, the better.
    I can say 'for general use, you may wish to use this open source product.
    And when you mention open source to customers, there are usually two sort of reactions :) : - Ooooh, that does not work, nor scale, has no documentation and no support, but is full of bugs. We do not want that s..., sorry, stuff. - Oh, it is free, we want it! :) Sorry, just kidding :). It is interesting that the higher profile and education is the customer, he/she usually values open source more. For example, we had a project for ECB (European Central Bank) and they valued Open Source a lot, we used Apache, Tomcat, Open LDAP, Perl and such stuff. I mean, they got the money, but they like open source, they wanted Java, used Oracle and they will to pay less for the software. The other customers of our company are from bookmaking (betting) industry (mostly from UK but also from other parts of the world). They have less money, prefer Microsoft solutions and are mostly agnostic to open source. But this is a problem for the sales stuff, not us anyway.
  39. Re: No ORM: The simplest solution?[ Go to top ]

    Well, customers usualy do not care about ORMs, JRSs and such stuff, they usualy care about working software that solves their business problem, the lower cost, the better.
    An example of how customers care about JSRs is when we provide software that runs on either JSP+Servlet containers or fuller J2EE platforms. JSRs mean that well-written software can sometimes only need to specify which app server the software needs - just the J2EE version. I can see the same thing potentially happening with ORMs - a customer may have considerable Xcalia or Kodo experience, and may wish to use your software as part an system that uses that ORM. Soon, you will be able to specify that your software has JDO 2.0 compliance or JPA 1.0 compliance, and the customer could choose the ORM product that they have the most experience with. Also, I have in mind some very specific software that I have been working on in the past (I can't go into details), where as Java port, considerations like the type of ORM would be very useful, as the customers could well also be developers.
    Sorry, just kidding :).

    It is interesting that the higher profile and education is the customer, he/she usually values open source more.

    For example, we had a project for ECB (European Central Bank) and they valued Open Source a lot, we used Apache, Tomcat, Open LDAP, Perl and such stuff. I mean, they got the money, but they like open source, they wanted Java, used Oracle and they will to pay less for the software.

    The other customers of our company are from bookmaking (betting) industry (mostly from UK but also from other parts of the world). They have less money, prefer Microsoft solutions and are mostly agnostic to open source.

    But this is a problem for the sales stuff, not us anyway.
    That is a very interesting observation.
  40. This discussion has been very interesting. As I have some strong opinions in this area myself, I cannot be silent any longer :-) Is there more overhead with ORM? This is actually a bit of a loaded question. The answer is that there is extra overhead with ORM, AND there is extra overhead without it. With ORM, there is extra processing overhead related to the additional object instantiations that occur. This leads to additional memory and CPU overhead above that of using well-coded JDBC implementations. Depending on complexity, it may also lead to additional database processing overhead. Using straight JDBC, there is additional coding overhead associated with the repeated coding of processing query result sets and setting up DB inserts and updates. The real question becomes, "Does the benefit of ORM justify its cost?". We debated this within our organization. And there are still some debates on-going. The major benefit has been seen as significant reduced time in coding. We conducted a couple of brief but telling experiments. 1. A developer took existing JDBC code and converted it to an ORM implementation using Hyberbernate, showing a 50% reduction in code related to data access. Upon closer inspection, the original JDBC code was badly and very inefficiently written. Optimizing the original code, as any good developer will, reduced the code savings to 25%, reducing the overall code savings for the project to less than 10%. 2. A seasoned developer who was not familiar with ORM began using hybernate to develop the data access layer of an application. It became obvious that signficant training was necessary to successfully implement an ORM using Hybernante, as the implementation failed miserably in the areas of performance and scalability. Again, the main question "Does the benefit justify the cost?". Costs Benefits ----- -------- Performance 10% of total coding effort Scalability Additional memory Additional CPU DB Resources Training I believe the message that began this blog describes the correct approach for all applications. Though progress has been made, ORM continues to struggle in areas of complexity and processing overhead, just as it did 10 years ago. ORM's recent surge has nothing to do with the progress of ORM, but the progress of computing power. However, computing power still comes at a cost, does the benefit of ORM out way it?
  41. Re: No ORM: The simplest solution?[ Go to top ]

    I believe the message that began this blog describes the correct approach for all applications. Though progress has been made, ORM continues to struggle in areas of complexity and processing overhead, just as it did 10 years ago. ORM's recent surge has nothing to do with the progress of ORM, but the progress of computing power. However, computing power still comes at a cost, does the benefit of ORM out way it?
    I am afraid I disagree with this premise. Yet again, I would like to emphasise that ORM != Hibernate, or rather it is considerably more than Hibernate. Having used both JDBC and ORM for substantial projects, my conclusion is that ORM is a far better way to manage complexity, and I found that for a large data model, my reduction in data access code was considerably more than 25% or 50%. I cannot now imagine attempting to develop an application with more than few tables without ORM. You can't judge the benefits or otherwise of Java ORM in general by investigating one product. Hibernate has its use cases, and other ORM products have theirs. Many ORM products have an excellent reputations for scalability and performance, and I know personally of projects that have required such performance and ORM has delivered, so I just can't agree that ORM is only growing in use because of increased computing power. Many companies have invested considerable resources to produce high-performance products. In all honesty, I don't believe that if you use a good ORM product, there is performance overhead that has to be set against other benefits.
  42. Re: No ORM: The simplest solution?[ Go to top ]

    I am afraid I disagree with this premise. Yet again, I would like to emphasise that ORM != Hibernate, or rather it is considerably more than Hibernate. Having used both JDBC and ORM for substantial projects, my conclusion is that ORM is a far better way to manage complexity, and I found that for a large data model, my reduction in data access code was considerably more than 25% or 50%.


    When you say you have experienced code reduction more than 25% or 50%, do you have measureable examples, or is it based on estimates? What I found is that those who are proponents of ORM often tend to quote the ORM advertising as to the savings they get. If you are really experiencing more than 50% savings, I would suggest the original JDBC code was very badly written to begin with (just as it was in our case).
    You can't judge the benefits or otherwise of Java ORM in general by investigating one product. Hibernate has its use cases, and other ORM products have theirs. Many ORM products have an excellent reputations for scalability and performance, and I know personally of projects that have required such performance and ORM has delivered, so I just can't agree that ORM is only growing in use because of increased computing power. Many companies have invested considerable resources to produce high-performance products. In all honesty, I don't believe that if you use a good ORM product, there is performance overhead that has to be set against other benefits.


    I wasn't suggesting that ORM can't perform or scale. It's a matter of how much it costs to do so. The point is it takes considerable training of staff to develop ORM-based solutions that perform well. ORM-based solutions are always less efficient in memory and CPU utilization due to the extra overhead of the tool. Therefore, how much additional cost is related to performance and scaling? Performance and scalability can almost always be achieved, regardless of the solution, by simply throwing enough hardware at it. The question is, "How much extra resources are required to make an ORM solution perform and scale?". The cost of the solution is more than just lines of code. I chose Hybernate as an example because it requires no licensing cost, and because it is arguably the most popular ORM tool. If the cost/benefit analysis doesn't work favorably for Hybernate, then it certainly won't work favorably for expensive commercial products.
  43. Re: No ORM: The simplest solution?[ Go to top ]

    When you say you have experienced code reduction more than 25% or 50%, do you have measureable examples, or is it based on estimates? What I found is that those who are proponents of ORM often tend to quote the ORM advertising as to the savings they get. If you are really experiencing more than 50% savings, I would suggest the original JDBC code was very badly written to begin with (just as it was in our case).
    This is actual experience. It was explicit JDBC. It may have been badly written because it was so explicit, but I then spend some time devising my own framework, my own method of handling DAOs. And I then realised I was effectively writing my own ORM, which was a waste of time.
    I chose Hybernate as an example because it requires no licensing cost, and because it is arguably the most popular ORM tool. If the cost/benefit analysis doesn't work favorably for Hybernate, then it certainly won't work favorably for expensive commercial products.
    This is an incorrect conclusion. The price of a tool is often not a significant part of its cost/benefit analysis. There may be additional support and tuning costs that a for one tool that another might not require, and those costs could come to far more than the purchase or licensing costs of a product. For example, I looked into a free tool years ago for an ORM-based application. To get it to work with the volume of data required would have meant spending a considerable amount of (costly) time. Instead, I went for a commercial solution. The vendor helped with the tuning of the product (it was their time spend on this, not mine) and the application worked fine and with good performance. I am pretty surprised that posters here are able to put so many anti-ORM case studies and so many anti-ORM opinions relatively unchallenged (except by me). I know from personal experience, and dealings with others, that ORM can be simple and very productive. ORM obviously needs better marketing, as a combination of experiences with poor quality ORMs (or certain ORMs used in situations they weren't designed for) with (occasionally) a Not Invented Here attitude obviously seems to be inhibiting its acceptance by some developers. ORM is definitely not just for situations where you need high-performance scaling and clustering. My experience is that it can be a hugely simplifying approach to Java development at any scale.
  44. Re: No ORM: The simplest solution?[ Go to top ]

    No offense, but the guy(not Steve) is calling it "Hybernate". How much can he know about a tool he can't spell?
  45. Re: No ORM: The simplest solution?[ Go to top ]

    This is actual experience. It was explicit JDBC. It may have been badly written because it was so explicit, but I then spend some time devising my own framework, my own method of handling DAOs. And I then realised I was effectively writing my own ORM, which was a waste of time.
    This is exactly the point I'm trying to make. Much of the touted benefit of ORM is saving programming time. The fact is that much of the savings is comparing bad programming to good implementation with ORM. Not really a good comparison. It doesn't take building an elaborate framework for efficient data access. In our case, we utilized an open-source connection pool and built a BaseDAO class that handled much of the mundane pieces. The remainder was simply good programming, factoring out common code, which is a practice not just for data access, but for coding in general.
    I am pretty surprised that posters here are able to put so many anti-ORM case studies and so many anti-ORM opinions relatively unchallenged (except by me). I know from personal experience, and dealings with others, that ORM can be simple and very productive. ORM obviously needs better marketing, as a combination of experiences with poor quality ORMs (or certain ORMs used in situations they weren't designed for) with (occasionally) a Not Invented Here attitude obviously seems to be inhibiting its acceptance by some developers.
    I think what you are finding on this forum are many who see through the hype, and/or have truly evaluated the TCO of ORM. What I find surprising is the number of people I have encountered over the years who have failed to look at the total cost of ORM, but simply get caught up in the hype. If you are not measuring the additional cost required to achieve scalability and performance using ORM, then you really have no idea how much it costs, and are simply buying in on the hype of saving lines of code, and even the code savings are hyped based on improper comparison.
  46. Re: No ORM: The simplest solution?[ Go to top ]

    This is exactly the point I'm trying to make. Much of the touted benefit of ORM is saving programming time. The fact is that much of the savings is comparing bad programming to good implementation with ORM. Not really a good comparison. It doesn't take building an elaborate framework for efficient data access. In our case, we utilized an open-source connection pool and built a BaseDAO class that handled much of the mundane pieces. The remainder was simply good programming, factoring out common code, which is a practice not just for data access, but for coding in general.
    My point is that apart from the very smallest of projects, even good programming is largely a waste of time compared with ORM. Why bother even writing a BaseDAO?
    I am pretty surprised that posters here are able to put so many anti-ORM case studies and so many anti-ORM opinions relatively unchallenged (except by me). I know from personal experience, and dealings with others, that ORM can be simple and very productive. ORM obviously needs better marketing, as a combination of experiences with poor quality ORMs (or certain ORMs used in situations they weren't designed for) with (occasionally) a Not Invented Here attitude obviously seems to be inhibiting its acceptance by some developers.


    I think what you are finding on this forum are many
    Well, two of you so far; but then there aren't that many arguing against you either.
    who see through the hype, and/or have truly evaluated the TCO of ORM.
    Have you? So far you have only reported experiments with Hibernate. Hibernate, good thought it is, does not reflect the full range and capabilities of ORM on Java. If you have evaluated other products, and can report on this, I would be interested.
    What I find surprising is the number of people I have encountered over the years who have failed to look at the total cost of ORM, but simply get caught up in the hype. If you are not measuring the additional cost required to achieve scalability and performance using ORM, then you really have no idea how much it costs, and are simply buying in on the hype of saving lines of code, and even the code savings are hyped based on improper comparison.
    Of course I am measuring the additional cost required to achieve scalability and performance! That is exactly why I went for a commercial product that had already put the effort in to achieve those goals, and when I had issues, they would put additional work in for me. This is exactly why I would recommend that developers do more than just go for for cheapest possible solution, or even the most popular solutions. (They may do the job, but this needs to be evaluated). Take a look through forums, check the experiences of others. Once you have done this, you can almost certainly find a product that will deliver the goods. This prevents you re-inventing the wheel and having to deal with scalability and performance issues in your own code, especially when the time comes to expand your data model or move into clustered and highly cached situations.
  47. Re: No ORM: The simplest solution?[ Go to top ]

    My point is that apart from the very smallest of projects, even good programming is largely a waste of time compared with ORM. Why bother even writing a BaseDAO?
    The reason that you should write your own BaseDAO instead of using an ORM tool is because of lower TCO. Keep in mind that writing a BaseDAO is not writing an ORM tool. It is simply factoring out common code, not automating the mapping of objects to relational data. Such automation is where the high cost of ORM starts coming in to play.
    Have you? So far you have only reported experiments with Hibernate. Hibernate, good thought it is, does not reflect the full range and capabilities of ORM on Java. If you have evaluated other products, and can report on this, I would be interested.
    I have been looking at ORM products for more 10 years. I've investigated products such as TopLink, prior to the days of Java. The problems are essentially the same. What I have experienced, regardless of product, is increased memory usage by 20%-30%, increased CPU usage by 20%-30%, and increased database CPU usage by 10%-20%, all while seeing performance decrease by 10%-50%. The cost of ORM has decreased over the past 10 years, not because ORM has improved, but because memory and CPU costs have decreased. Taking the low end of the ranges mentioned above will increase costs for high volume applications in the 10s of thousands of dollars (and likely higher when considering CPU-based licensing costs of applications running on servers which require additional CPU). These are the costs that proponents of ORM never consider (that I've seen).
    Of course I am measuring the additional cost required to achieve scalability and performance!
    I would like to see those numbers. How much is it costing for the addtional memory, CPU, CPU-based licenses of products, commercial ORM product, and training on ORM compared with how much you are saving?
  48. Re: No ORM: The simplest solution?[ Go to top ]

    I have been looking at ORM products for more 10 years. I've investigated products such as TopLink, prior to the days of Java.
    What other products? So far you have mentioned Hibernate, and pre-Java TopLink.
    The problems are essentially the same. What I have experienced, rega1rdless of product, is increased memory usage by 20%-30%, increased CPU usage by 20%-30%, and increased database CPU usage by 10%-20%, all while seeing performance decrease by 10%-50%.
    Such amazingly precise figures... do you have the standard deviations on those? (I am a statician, and am facinated when someone can say something so exact as 'regardless of product' CPU increases by 20-30%). Major organsations must be somewhat perverse in choosing to use products that are so disadvantageous. You might wish to suggest to Ebay that they abandon their use of ORM for their site.... or perhaps they might know what they are doing and you are mistaken?
    The cost of ORM has decreased over the past 10 years, not because ORM has improved, but because memory and CPU costs have decreased.
    That is false. I have also been using ORM on and off for over a decade (I evaluated Gemstone with Smalltalk). The quality of ORM has improved beyond recognition. There was nothing like the flexible and powerful query languages we have today, for example.
    Taking the low end of the ranges mentioned above will increase costs for high volume applications in the 10s of thousands of dollars (and likely higher when considering CPU-based licensing costs of applications running on servers which require additional CPU). These are the costs that proponents of ORM never consider (that I've seen).
    That is because costs aren't on that scale. I am using Kodo, and you are wildly out (in the high end) in terms of your pricing. CPU-based licensing costs are ancient history in my experience.
    Of course I am measuring the additional cost required to achieve scalability and performance!


    I would like to see those numbers. How much is it costing for the addtional memory, CPU, CPU-based licenses of products, commercial ORM product, and training on ORM compared with how much you are saving?
    Of course I don't need additional memory or CPU - that is just FUD - in fact ORM is saving me memory because of its good cache management - I am running transactions with hundreds of thousands of objects on a laptop PC. As for costs, I am using Kodo JDO. You can find the price from BEA. Your estimates are wildly exaggerated. I have no idea where you are getting your ideas of what ORM requires, but it isn't anything I recognise.
  49. Re: No ORM: The simplest solution?[ Go to top ]

    What other products? So far you have mentioned Hibernate, and pre-Java TopLink.
    At the time I used Toplink, it was the leader in the market for ORM. Now Hybernate is arguably the leader in the Java space. I don't see where wasting time on non-market leaders is going to change anything.
    Major organsations must be somewhat perverse in choosing to use products that are so disadvantageous. You might wish to suggest to Ebay that they abandon their use of ORM for their site.... or perhaps they might know what they are doing and you are mistaken?
    Plain and simple, they are not evaluating the total cost of the solution. As such, they have no idea how much the solution is costing them, the very point I'm trying to make. And as you are dancing around the topic, I suggest you have no idea how much the solution is costing you either. You still haven't produced the numbers of you costs vs. your savings. I gave you numbers based on my experience over the years, both with a non-java product early in ORM and with a Java product that is a current product.
    That is false. I have also been using ORM on and off for over a decade (I evaluated Gemstone with Smalltalk). The quality of ORM has improved beyond recognition. There was nothing like the flexible and powerful query languages we have today, for example.
    Sorry, I didn't word this very well. I agree that ORM has made signficant improvements over the years. What I meant to say is that the reduced cost of ORM has little to do with those improvements as compared with the lower cost of CPU and memory.
    That is because costs aren't on that scale. I am using Kodo, and you are wildly out (in the high end) in terms of your pricing. CPU-based licensing costs are ancient history in my experience.
    I provided numbers based on my experience. They are absolutley correct. The fact that you can't provide numbers tells me that you really haven't measured it objectively.
    Of course I don't need additional memory or CPU - that is just FUD - in fact ORM is saving me memory because of its good cache management - I am running transactions with hundreds of thousands of objects on a laptop PC. As for costs, I am using Kodo JDO. You can find the price from BEA. Your estimates are wildly exaggerated. I have no idea where you are getting your ideas of what ORM requires, but it isn't anything I recognise.
    I hope you can some day look at this objectively, and not ignore the facts. Licensing cost is a significant cost, whether it's an App server (e.g. BEA Weblogic Server) or a database (e.g. Oracle). Whenever CPU requirements increase, those costs increase right along with them. Don't confuse features such as caching with ORM. Caching has been around much longer than ORM. It just so happens that ORM tools incorporate it. You don't have to use an ORM tool for caching. That's a completely separate topic.
  50. Re: No ORM: The simplest solution?[ Go to top ]

    What other products? So far you have mentioned Hibernate, and pre-Java TopLink.

    At the time I used Toplink, it was the leader in the market for ORM. Now Hybernate is arguably the leader in the Java space. I don't see where wasting time on non-market leaders is going to change anything.
    Because market leaders aren't always the market leaders because of quality in all areas, and to make any claims based on a sample of two products in a market with so many is statistically of little or no use. By that argument you would believe Windows was the ultimate OS for all situations.....
    Major organsations must be somewhat perverse in choosing to use products that are so disadvantageous. You might wish to suggest to Ebay that they abandon their use of ORM for their site.... or perhaps they might know what they are doing and you are mistaken?

    Plain and simple, they are not evaluating the total cost of the solution. As such, they have no idea how much the solution is costing them, the very point I'm trying to make. And as you are dancing around the topic, I suggest you have no idea how much the solution is costing you either. You still haven't produced the numbers of you costs vs. your savings. I gave you numbers based on my experience over the years, both with a non-java product early in ORM and with a Java product that is a current product.
    Sorry, but given the choice to believe your figures (especially based on what you admit is a limited experience of the area), or my own experience of a wider range of ORMs and the major success of projects based on ORM, I go for the latter.
    That is false. I have also been using ORM on and off for over a decade (I evaluated Gemstone with Smalltalk). The quality of ORM has improved beyond recognition. There was nothing like the flexible and powerful query languages we have today, for example.

    Sorry, I didn't word this very well. I agree that ORM has made signficant improvements over the years. What I meant to say is that the reduced cost of ORM has little to do with those improvements as compared with the lower cost of CPU and memory.
    That is because costs aren't on that scale. I am using Kodo, and you are wildly out (in the high end) in terms of your pricing. CPU-based licensing costs are ancient history in my experience.

    I provided numbers based on my experience. They are absolutley correct. The fact that you can't provide numbers tells me that you really haven't measured it objectively.
    I can't give you pricing because this has probably changed since BEA has taken over Solarmetric. But I can give you the fact that your price estimates are way, way too high. I have certainly measured things objectively. I am working in areas which involve relatively high volume batch processing - loading and transforming hundreds of thousands or even millions of records into databases. Years ago I had been doing this by JDBC+SQL for portability (although it would have been even faster with proprietary loading tools). Benchmarking showed that the database inserts and processing were the major factor in the time taken for the processes to run. I switched to a commercial ORM (at the time it was called JDOGenie). What was the time difference? None. No need for extra CPU, no need for extra memory, even for processing very large transactions. I don't make claims without evidence, which is why I don't accept that your figures for increased CPU or memory are in any way applicable to ORM in general, and certainly not for the products I have used.
    Of course I don't need additional memory or CPU - that is just FUD - in fact ORM is saving me memory because of its good cache management - I am running transactions with hundreds of thousands of objects on a laptop PC.

    As for costs, I am using Kodo JDO. You can find the price from BEA. Your estimates are wildly exaggerated.
    I have no idea where you are getting your ideas of what ORM requires, but it isn't anything I recognise.

    I hope you can some day look at this objectively, and not ignore the facts.
    I hope you can. The price is available to you right now from BEA. I have given you facts above.
    Caching has been around much longer than ORM. It just so happens that ORM tools incorporate it. You don't have to use an ORM tool for caching. That's a completely separate topic.
    My point is that ORM was cacheing for me transparently, with no work on my part. I can even add clustered cacheing transparently.
  51. Re: No ORM: The simplest solution?[ Go to top ]

    Because market leaders aren't always the market leaders because of quality in all areas, and to make any claims based on a sample of two products in a market with so many is statistically of little or no use. By that argument you would believe Windows was the ultimate OS for all situations.....
    Now you are reaching. The fundamentals of ORM are essentially the same, no matter what product you might be using. Though the overhead of object creation and additional CPU may vary from product to product, it does exist in every ORM product. Comparing it to Windows is REALLY reaching. Windows is the undisputed leader in desktop OS. It isn't the leader in hosting high-end computing environemnts. Enough said.
    Sorry, but given the choice to believe your figures (especially based on what you admit is a limited experience of the area), or my own experience of a wider range of ORMs and the major success of projects based on ORM, I go for the latter.
    I choose to measure TCO, not buy into product hype. The numbers I have quoted are from experience, not made up. As long as you refuse to measure TCO, we really have nothing else to discuss here.
    I can't give you pricing because this has probably changed since BEA has taken over Solarmetric. But I can give you the fact that your price estimates are way, way too high.
    Let me give an example based on the numbers I mentioned before. If I have a high-volume, database intensive application built on Weblogic Server, which requires 4 CPU's using straight JDBC, it will require 1 additional CPU if using an ORM tool, such as Hybernate. Weblogic Server lists at more than $20,000/CPU + the hardware cost of the CPU + the labor cost for adding the CPU (and typically CPU's are added in pairs). If you look at Oracle licenses due to additional DB resource usage, the cost can go even higher.
    I have certainly measured things objectively. I am working in areas which involve relatively high volume batch processing - loading and transforming hundreds of thousands or even millions of records into databases. Years ago I had been doing this by JDBC+SQL for portability (although it would have been even faster with proprietary loading tools). Benchmarking showed that the database inserts and processing were the major factor in the time taken for the processes to run. I switched to a commercial ORM (at the time it was called JDOGenie). What was the time difference? None. No need for extra CPU, no need for extra memory, even for processing very large transactions. I don't make claims without evidence, which is why I don't accept that your figures for increased CPU or memory are in any way applicable to ORM in general, and certainly not for the products I have used.
    The results you mention here tell me two things; 1) The original JDBC code was poor to begin with or the SQL was very simple and 2) you didn't measure the memory and CPU utilization of the application. ORM has overhead and there is a cost to using it. Architects and developers often choose to use it without measuring the real cost. And that is certainly their choice. I choose to understand the full cost and benefit of a solution before making a choice. As I have done over the past decade+, I will continue looking at and evaluating data access methods and tools. Whether advances on current ORM technology or some other new method comes along, if it provides value to my company, I will absolutely jump on board. But value is measured in total benefit vs. total cost. Ignoring some of the costs will do my company a great disservice. As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.
  52. Re: No ORM: The simplest solution?[ Go to top ]

    Because market leaders aren't always the market leaders because of quality in all areas, and to make any claims based on a sample of two products in a market with so many is statistically of little or no use.

    By that argument you would believe Windows was the ultimate OS for all situations.....

    Now you are reaching. The fundamentals of ORM are essentially the same, no matter what product you might be using. Though the overhead of object creation and additional CPU may vary from product to product, it does exist in every ORM product.

    Comparing it to Windows is REALLY reaching. Windows is the undisputed leader in desktop OS. It isn't the leader in hosting high-end computing environemnts. Enough said.
    Comparing it to Windows was not reaching - just illustrating the pointlessness of taking just the market leader as representative. And regarding the fundamentals of ORM, you are wrong, as you would know if you had used more products. There can be different approaches like proxying or enhancement, and different types of object lifecycle. These can affect memory use and performance, especially when dealing with large numbers of objects. Different approaches require different amounts of object creation, for example.
    I choose to measure TCO, not buy into product hype. The numbers I have quoted are from experience, not made up. As long as you refuse to measure TCO, we really have nothing else to discuss here.
    This was not product hype, it was actual experience. I did not imply your numbers were made up, I implied that they were not representative of ORM in general, just the limited market segment you have experience of.
    I can't give you pricing because this has probably changed since BEA has taken over Solarmetric. But I can give you the fact that your price estimates are way, way too high.

    Let me give an example based on the numbers I mentioned before. If I have a high-volume, database intensive application built on Weblogic Server, which requires 4 CPU's using straight JDBC, it will require 1 additional CPU if using an ORM tool, such as Hybernate. Weblogic Server lists at more than $20,000/CPU + the hardware cost of the CPU + the labor cost for adding the CPU (and typically CPU's are added in pairs). If you look at Oracle licenses due to additional DB resource usage, the cost can go even higher.
    Why should use of an ORM require an additional CPU? If you research this, you will find that for years developers have been using ORM to improve the performance of app servers by substituting the ORM for EJB persistence. This is has been a common use case for JDO. Far from requiring additional CPU resources, it reduces this requirement.
    I have certainly measured things objectively. I am working in areas which involve relatively high volume batch processing - loading and transforming hundreds of thousands or even millions of records into databases. Years ago I had been doing this by JDBC+SQL for portability (although it would have been even faster with proprietary loading tools). Benchmarking showed that the database inserts and processing were the major factor in the time taken for the processes to run. I switched to a commercial ORM (at the time it was called JDOGenie). What was the time difference? None. No need for extra CPU, no need for extra memory, even for processing very large transactions. I don't make claims without evidence, which is why I don't accept that your figures for increased CPU or memory are in any way applicable to ORM in general, and certainly not for the products I have used.

    The results you mention here tell me two things; 1) The original JDBC code was poor to begin with or the SQL was very simple and 2) you didn't measure the memory and CPU utilization of the application. ORM has overhead and there is a cost to using it.
    (1) It was not poor, but it was relatively simple, but then who cares? ORM saved me writing the SQL, and gave me the same performance. (2) I certainly did measure the memory and CPU utilisation, and I told you I did. The database was the primary user of CPU in both cases. I was extremely concerned about memory, due to the size of the transaction. The memory use was not excessive - no extra CPU or memory was required as you implied would be the case.
    Architects and developers often choose to use it without measuring the real cost.
    What a broad accusation!
    I choose to understand the full cost and benefit of a solution before making a choice.

    As I have done over the past decade+, I will continue looking at and evaluating data access methods and tools.
    By implication, others don't do this, which is nonsense.
    But value is measured in total benefit vs. total cost. Ignoring some of the costs will do my company a great disservice.
    Costs need to be investigation by looking at a wide range of solutions; you can't just pick one product and assume you are providing anything like a balanced judgement.
    As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.
    No, it really isn't. This might only be the case if you pick one product that might not be the best suited to your requirements, and assume if that fails that using your own code is the answer. There can often be better ways.
  53. Re: No ORM: The simplest solution?[ Go to top ]

    This discussion probably has long since lost its fruitfullness. I don't think we are going to agree. I contend that you are still not looking at total cost, and I believe that has clouded your judgement.
    Comparing it to Windows was not reaching - just illustrating the pointlessness of taking just the market leader as representative. And regarding the fundamentals of ORM, you are wrong, as you would know if you had used more products. There can be different approaches like proxying or enhancement, and different types of object lifecycle. These can affect memory use and performance, especially when dealing with large numbers of objects. Different approaches require different amounts of object creation, for example.
    The reason I said it is stretching is because I would apply the same reasoning to selecting an OS for my organization as I have been mentioning for ORM. I would evaluate total cost. For example, Linux is becoming a viable option for desktop computing, and has benefits of better security and less vulnerability to viruses than Windows. However, the costs of using it (even if it were free) outweigh those benefits. I'm fully aware of optimization techniques in coding. The fact remains that, no matter how optimized, there is still overhead in an ORM tool. You can deny that until you are blue in the face, and it will still be true. Providing the underlying code infrastructure to accomodate mapping of user defined objects to user defined database schemas will always require more object creation than manually mapping for a specific need. If you have found something that somehow magically does this with no extra overhead, then please show me the JDBC and respective ORM examples to prove it. You might actually be the first!
    Why should use of an ORM require an additional CPU? If you research this, you will find that for years developers have been using ORM to improve the performance of app servers by substituting the ORM for EJB persistence. This is has been a common use case for JDO. Far from requiring additional CPU resources, it reduces this requirement.
    That's assuming you use CMP. I don't. My custom JDBC will always be more efficient, requiring less memory and CPU than ORM.
    (1) It was not poor, but it was relatively simple, but then who cares? ORM saved me writing the SQL, and gave me the same performance.
    My experience has been that simple is what ORM does best. The more complex the data access transactions, the more ORM costs (in memory and CPU). So it saved you from writing simple SQL. What did it cost? There is always a cost! Denying the cost aspect doesn't make it untrue, just simply implies you don't understand it.
    (2) I certainly did measure the memory and CPU utilisation, and I told you I did. The database was the primary user of CPU in both cases. I was extremely concerned about memory, due to the size of the transaction. The memory use was not excessive - no extra CPU or memory was required as you implied would be the case.
    Actually, you didn't say you measured it, you simply stated you didn't require more. That doesn't necessarily mean you measured it. It might mean it fit within the same memory configuration the original application fit within, which likely was more than the application acutally needed in order to accomodate growth. The fact is, if you actually measure it, you will see extra objects created, utilizing more memory, and requiring more CPU for garbage collection. If you truly have found a tool that has absolutely no overhead, send some examples so I can try it out myself. Maybe you have stumbled on to something!
    By implication, others don't do this, which is nonsense.
    Not implying that at all. I know others do, I work with some of them. I'm suggesting that "many don't", and suggesting you don't either, at least in the case of ORM.
    Costs need to be investigation by looking at a wide range of solutions; you can't just pick one product and assume you are providing anything like a balanced judgement.
    As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.
    No, it really isn't. This might only be the case if you pick one product that might not be the best suited to your requirements, and assume if that fails that using your own code is the answer. There can often be better ways.
    There are many factors that go into making a market leader. Though market share is a significant factor, there are many others. Most people don't have time to evaluate every product that is released. If there is a truly inovative and beneficial product, it will make a lot of noise and evolve itself as a market leader (maybe the product you are using will do so). You don't have to look at many products to understand a technology and its benefit to an organization. You need to have a good understanding of what the technology is attempting to accomplish, the experience to understand the implications, and for the few products that are truly market leaders, a solid evaluation plan that measures the complete cost and benefit.
  54. Re: No ORM: The simplest solution?[ Go to top ]

    This discussion probably has long since lost its fruitfullness. I don't think we are going to agree. I contend that you are still not looking at total cost, and I believe that has clouded your judgement.
    And I contend that you have looked at far too narrow a range of products (if 'one' can be called a range) to make any assessment of what the costs of ORM might be.
    The reason I said it is stretching is because I would apply the same reasoning to selecting an OS for my organization as I have been mentioning for ORM. I would evaluate total cost. For example, Linux is becoming a viable option for desktop computing, and has benefits of better security and less vulnerability to viruses than Windows. However, the costs of using it (even if it were free) outweigh those benefits.

    I'm fully aware of optimization techniques in coding. The fact remains that, no matter how optimized, there is still overhead in an ORM tool. You can deny that until you are blue in the face, and it will still be true.
    Then I will get blue in the face, as I deny it. I deny that there is any overhead in ORM if you are going to do anything like that in your own framework - populating objects from a database, persisting them, perhaps cacheing a few.
    Providing the underlying code infrastructure to accomodate mapping of user defined objects to user defined database schemas will always require more object creation than manually mapping for a specific need.
    No, it won't. Either you map the objects, or the ORM does. The differences for a specific need are minimal, unless you are using perhaps just one or two objects. For a sufficient size, you will be simply replicating what the ORM does.
    If you have found something that somehow magically does this with no extra overhead, then please show me the JDBC and respective ORM examples to prove it.
    I might actually work on doing that :)
    Why should use of an ORM require an additional CPU? If you research this, you will find that for years developers have been using ORM to improve the performance of app servers by substituting the ORM for EJB persistence. This is has been a common use case for JDO. Far from requiring additional CPU resources, it reduces this requirement.

    That's assuming you use CMP. I don't. My custom JDBC will always be more efficient, requiring less memory and CPU than ORM.
    So you keep stating, but all you can say so far is that it will be more efficient and require less memory and CPU than Hibernate. That is certainly not saying the same thing. You still have not specified why the use of a lightweight ORM should require an entire extra CPU, whether it is a replacement for CMP or not.
    (1) It was not poor, but it was relatively simple, but then who cares? ORM saved me writing the SQL, and gave me the same performance.

    My experience has been that simple is what ORM does best. The more complex the data access transactions, the more ORM costs (in memory and CPU). So it saved you from writing simple SQL. What did it cost? There is always a cost! Denying the cost aspect doesn't make it untrue, just simply implies you don't understand it.
    Firstly you are wrong - the broad opinion is that ORM does complex best. I am trying to justify its use for the simpler cases. I can give you a specific case where ORM has saved me considerable money. It allows me to develop, test, and deploy on different databases. I can and do develop largely on PostgreSQL and then deploy on Oracle. This is largely problem-free. The time saved in having to write high-performance vendor-specific SQL is considerable, and the reduced cost of CPU and hardware because I don't need to use Oracle all the time is also highly significant.
    (2) I certainly did measure the memory and CPU utilisation, and I told you I did. The database was the primary user of CPU in both cases. I was extremely concerned about memory, due to the size of the transaction. The memory use was not excessive - no extra CPU or memory was required as you implied would be the case.

    Actually, you didn't say you measured it, you simply stated you didn't require more. That doesn't necessarily mean you measured it.It might mean it fit within the same memory configuration the original application fit within, which likely was more than the application acutally needed in order to accomodate growth.
    But a while back you were talking about the additional cost of memory, CPUs and CPU licences. This clearly demonstrates that that was incorrect.
    The fact is, if you actually measure it, you will see extra objects created, utilizing more memory, and requiring more CPU for garbage collection.
    That depends entirely on the use case.
    If you truly have found a tool that has absolutely no overhead, send some examples so I can try it out myself. Maybe you have stumbled on to something!
    Yes, I have. It is called a good JDO product with the ability to manage memory well, with features like fetch groups and fetch plans to keep memory use to a minimum, and which uses the capabilities of things such as JDO object lifecycle states to allow large transactions to be run with minimal memory. But that is not the point. You are talking about costs. There are costs involved with the development and support of your own framework, no matter how small. It does not take that many hours to cover the cost of the extra memory or CPU.
    Costs need to be investigation by looking at a wide range of solutions; you can't just pick one product and assume you are providing anything like a balanced judgement.
    As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.

    No, it really isn't. This might only be the case if you pick one product that might not be the best suited to your requirements, and assume if that fails that using your own code is the answer. There can often be better ways.

    There are many factors that go into making a market leader. Though market share is a significant factor, there are many others. Most people don't have time to evaluate every product that is released. If there is a truly inovative and beneficial product, it will make a lot of noise and evolve itself as a market leader (maybe the product you are using will do so). You don't have to look at many products to understand a technology and its benefit to an organization. You need to have a good understanding of what the technology is attempting to accomplish, the experience to understand the implications, and for the few products that are truly market leaders, a solid evaluation plan that measures the complete cost and benefit.
    Yes, you really do have to look at many products to understand a technology and its benefits, certain in this case. There are major differences in technology, targetted use cases and features of Java ORMs; even within those that target the same JSR, be it JDO or JPA. You simply can't have a good understanding of a technology in this area without such evaluations - taking a look at just one product can give a seriously flawed idea of what others in the market can do. It is certainly not a solid evaluation plan in the case of Java ORM, any more than it would be a solid evaluation plan in tha case of other Java frameworks.
  55. Yes, you really do have to look at many products to understand a technology and its benefits, certain in this case. There are major differences in technology, targetted use cases and features of Java ORMs; even within those that target the same JSR, be it JDO or JPA
    When you do get a chance, you may also want to look into JDX OR-Mapper that adheres to the following KISS (Keep It Simple and Straightforward) principles: http://www.softwaretree.com/products/njdx/whitepaper/KISSPrinciples.pdf An ORM layer does bring some extra overhead in terms of performance and memory-usage. However, some of this is alleviated by the ORM product's optimizations like connection pooling, use of prepared statements, minimizing the number of database trips, and caching. Of course, an application developers using straight JDBC can also do these optimizations, but it would be a fairly non-trivial and time-consuming exercise. Further, such hand-coded optimizations will be hard to leverage from one project to another unless one has created a mini OR-Mapper as part of this exercise. By the way, in the overall performance equation, the major cost comes from the network latency of the database calls. If you also add the latency of the web tier, the performance overhead of an efficient ORM layer may be insignificant from the overall application performance POV. There would sure be some learning curve in using any new technology. But I believe that the resulting simplicity of the code, eventual productivity increase, and easier maintenance will more than compensate for the cost associated with using a good ORM framework. -- Damodar Periwal Software Tree, Inc. Simplify Data Integration http://www.softwaretree.com
  56. An ORM layer does bring some extra overhead in terms of performance and memory-usage.
    But that is one of the issues I disagree with. I don't believe it does. I think this is often a myth based on the false idea that having additional layers of abstraction necessarily imposes performance or resource penalties.
  57. An ORM layer does bring some extra overhead in terms of performance and memory-usage.


    But that is one of the issues I disagree with. I don't believe it does. I think this is often a myth based on the false idea that having additional layers of abstraction necessarily imposes performance or resource penalties.
    Steve, I think we are more-or-less on the same page in terms of advantages of ORM. In some cases, the performance of some poorly designed JDBC applications can even improve significantly with optimal use of a good ORM framework. Most of my comments were meant as a response to other posts citing drawbacks of ORM solutions. -- Damodar
  58. An ORM layer does bring some extra overhead in terms of performance and memory-usage.


    But that is one of the issues I disagree with. I don't believe it does. I think this is often a myth based on the false idea that having additional layers of abstraction necessarily imposes performance or resource penalties.

    Steve, I think we are more-or-less on the same page in terms of advantages of ORM. In some cases, the performance of some poorly designed JDBC applications can even improve significantly with optimal use of a good ORM framework. Most of my comments were meant as a response to other posts citing drawbacks of ORM solutions.

    -- Damodar
    I agree with you in almost anything. Where we differ is that I believe that with a really good quality ORM, the performance of even typical JDBC applications can improve significantly. ORM developers have put a lot of work into the production of very high quality optimised SQL, for example.
  59. Re: No ORM: The simplest solution?[ Go to top ]

    Steve, Speaking as someone who has no axe to grind either for or against ORM, I would like to say that you do your viewpoint no favours by repeatedly stating that ORM products impose no overhead. When people point out that an ORM must impose some extra overhead, it would behoove you to accept that as a simple and self-evident truth, because that's what it is. For sure, the overhead might be minimal; it might even be insignificant: but it definitely exists. The far more important point is that the use of the ORM might well mean that the overhead is more than compensated for. To repeatedly argue that an extra layer of (quite complex) software imposes no overhead makes you sound like a zealot, and it detracts from the point you're trying to make. Well, it does to me, anyway - I hate to see intelligent people arguing like politicians...
  60. Re: No ORM: The simplest solution?[ Go to top ]

    Steve,

    Speaking as someone who has no axe to grind either for or against ORM, I would like to say that you do your viewpoint no favours by repeatedly stating that ORM products impose no overhead.

    When people point out that an ORM must impose some extra overhead, it would behoove you to accept that as a simple and self-evident truth, because that's what it is. For sure, the overhead might be minimal; it might even be insignificant: but it definitely exists. The far more important point is that the use of the ORM might well mean that the overhead is more than compensated for.

    To repeatedly argue that an extra layer of (quite complex) software imposes no overhead makes you sound like a zealot, and it detracts from the point you're trying to make.

    Well, it does to me, anyway - I hate to see intelligent people arguing like politicians...
    I don't believe I have been arguing the point as you state, because the matter is overhead over what? If you are simply accessing a few values by JDBC, then of course ORM will have overhead. What has been put forward several times on this thread is that even if you write your own data access layer, implement your own form of object loading (possibly with lazy loading), and add some cacheing, then ORM will have overhead over all that. I am saying that I really doubt it does. I see people assume that there must somehow be overhead for several reasons - depth of abstraction ("there is a large stack trace, and all that code must slow things down"), object creation and memory and complexity. Depth of abstraction need not impact performance much if at all; it depends what happens at the various layers, but depth of itself means little. Object creation can be fast, especially when you are dealing with systems that use enhancement, and implementations of ORMs like JDO can be very good at handling memory efficiently. Finally, much of the complexity can be to do with features, not operation. You can make things simply by not using things - transactions, fetch groups, cache, lazy loading and so on can all be ignored. When you leave those out you can end up with a pretty simple system. If I have had the phrase 'no overhead at all' slip out unqualified, then I was mistaken. What I meant was that compared to what some posters have been implementing themselves, I see little or no overhead in terms of memory of performance if you choose the right ORM implementation (the last point is key). I am tempted to put actual figures on a blog article to justify this; the problem is that it may be futile, as some may claim that any overhead is a justification to write home-grown solutions, and one can always tune the ORM little more, or optimise the SQL a little more to tweak things one way or the other. Still, it might be an interesting exercise. I guess I do see this as political, which is perhaps why I have been pushing this point.... my view is that the time when we write our own frameworks should be long past. And what has been suggested be written in-house in some posts here certainly justifies the use of the word 'framework'!
  61. Re: No ORM: The simplest solution?[ Go to top ]

    I am tempted to put actual figures on a blog article to justify this; the problem is that it may be futile, as some may claim that any overhead is a justification to write home-grown solutions, and one can always tune the ORM little more, or optimise the SQL a little more to tweak things one way or the other. Still, it might be an interesting exercise. I guess I do see this as political, which is perhaps why I have been pushing this point.... my view is that the time when we write our own frameworks should be long past. And what has been suggested be written in-house in some posts here certainly justifies the use of the word 'framework'!
    My point hasn't just been overhead, it has been TCO. I have yet to see a post in favor of ORM tools describe the benefit in terms of TCO. I see statements like "it saved me from writing JDBC code", "it provides caching", "it follows standards", etc., but no one talks about the actual savings that can be provided to a company, accounting for all the benefits and costs associated (see my earlier post from a few weeks ago). I personally don't buy into hype nor advertisement. When EJB came out I was a skeptic (used CORBA at the time), but was won over once I saw and measured the benefits. I will do exactly the same with ORM, once I see that it saves my company money. At this point, my personal experience tells me otherwise. In the end, its the value we bring to the companies that employ us that matters, not learning new things, saving a few lines of code, etc. When looking at benefit, the entire picture must be considered. I'm simply not seeing that happen from the promoters of ORM in this blog.
  62. Re: No ORM: The simplest solution?[ Go to top ]

    My point hasn't just been overhead, it has been TCO. I have yet to see a post in favor of ORM tools describe the benefit in terms of TCO.
    As described in this white paper, the use of a good ORM product can typically cut down the LOC for data integration logic significantly. The appendix of the following project report shows how the new DAL implementation with NJDX OR-Mapper shrunk the code size of the Order module from 234 lines to 68 lines (a 70% reduction). So the benefits of using OR-Mapping technology are many including: - Improved developer productivity due to elimination of endless lines of low-level SQL code - Simpler, intuitive, and object-oriented code - Less maintenance hassles as the code is easier to understand and evolve - More robust applications as the OR-Mapping engine is well tested in many different application scenarios - In many cases, better performing applications because of in-built optimizations of the OR-Mapping engine - Accelerated time-to-market driving up top-line revenues Assuming that the typical data integration coding takes 50% of the total effort of the development of a typical application, a 50% reduction in that effort by using an OR-Mapper results in 25% reduction in the overall cost of development. That is a significant ROI (Return On Investment). Further advantages like code modularity, better understandability, robustness, reduced maintenance hassles, schema integration tools, and accelerated time-to-market are additional bonuses of using ORM. Not to mention better moral of the developers and their managers as they are not suffering from sleepless nights due to low-level programming complexities and resulting project delays :) Damodar Periwal Software Tree, Inc. Simplify Data Integration.
  63. As described in this white paper, the use of a good ORM product can typically cut down the LOC for data integration logic significantly. The appendix of the following project report shows how the new DAL implementation with NJDX OR-Mapper shrunk the code size of the Order module from 234 lines to 68 lines (a 70% reduction). So the benefits of using OR-Mapping technology are many including: - Improved developer productivity due to elimination of endless lines of low-level SQL code - Simpler, intuitive, and object-oriented code - Less maintenance hassles as the code is easier to understand and evolve - More robust applications as the OR-Mapping engine is well tested in many different application scenarios - In many cases, better performing applications because of in-built optimizations of the OR-Mapping engine - Accelerated time-to-market driving up top-line revenues
    This white paper proves the point I have been trying to make in this blog. It distorts the truth in favor of ORM. The simple example in the white paper shows a 70% reduction in code when comparing inefficiently written code (the two queries can be combined into a single query) and doesn't show the mapping configuration (which arguably is also code) needed for JDX to create the proper query and map it to the appropriate classes. Taking all of that into consideration, you will probably see code reduction more realistically in the 30% - 50% range (probably closer to 30%). It also leaves out any costs associated with the tool (e.g. extra memory and CPU overhead, learning curve, etc.). Lastly, it doesn't address the challenges of ORM relative to tuning and troubleshooting. Even if ORM does provide value, you are doing the technology no good by intentionally slanting your view to sell a product. I'm all for ORM as long as it provides value. By leaving out any costs associated with ORM tells me that even you doubt its value. If that isn't the case, create a white paper that tells the whole story, not just the part that helps you sell your product. All of the points you make in this blog and in your white paper can be seen on the web site of any ORM product, claiming to save lines of code and reduce development time. I want to see both vendors and users of ORM products go beyond advertisement and look at the real value (TCO).
  64. Re: No ORM: The simplest solution?[ Go to top ]

    Been a way from this for a little while. I had some "real" work to do :-)
    Then I will get blue in the face, as I deny it. I deny that there is any overhead in ORM if you are going to do anything like that in your own framework - populating objects from a database, persisting them, perhaps cacheing a few.
    The overhead in ORM is inside the generic framework of mapping relational data to objects. ORM tools have to facilitate the idea of not knowing the relational model nor the object model. In doing so, there are many extra objects that are instantiated in order to create this abstraction. When I code my own, I know both my relational model and my object model, and simply assign the object attributes with the appropriate values obtained from the database. As I said before, this will always be far more efficient than any ORM tool can perform. Though many ORM tools include other features such as caching, They are separate from ORM. I agree that if a good caching tool exists one shouldn't be custom built. There are plenty of good caching tools that do not provide much overhead, independent of ORM.
    Providing the underlying code infrastructure to accomodate mapping of user defined objects to user defined database schemas will always require more object creation than manually mapping for a specific need.
    No, it won't. Either you map the objects, or the ORM does. The differences for a specific need are minimal, unless you are using perhaps just one or two objects. For a sufficient size, you will be simply replicating what the ORM does.
    I guarantee it is more effecient to do it yourself. The reason is for what I just stated above. When I map myself, I'm not building a generic ORM, I'm mapping specific relational data to specific objects, factoring out common code (e.g getting connections, releasing connections, exception handling, etc.). The only way this isn't true is if the custom code is very poorly written.
    Actually, you didn't say you measured it, you simply stated you didn't require more. That doesn't necessarily mean you measured it.It might mean it fit within the same memory configuration the original application fit within, which likely was more than the application acutally needed in order to accomodate growth.
    But a while back you were talking about the additional cost of memory, CPUs and CPU licences. This clearly demonstrates that that was incorrect.
    This statement indicates that you might not have a good understanding of Java memory management and how to actually measure memory utilization in Java. Let me give you an example. Let's say that my custom built application is utilizing 100MB of memory. I have set my JVM memory heap settings to be 200MB to accomodate for growth. I now change my application to use ORM, and it consumes 130MB of memory. I don't have to change anything relative to my application memory settings, even though I have consumed 30% more memory. It is still a real cost that must be considered. Building all my applications this way will lead me to more memory purchases for my servers than would have been required for custom built. It also leads to more CPU cycles to manage garbage collection (of the additional objects created by ORM). The only way to get a good measure of the difference is to create a well-built application with custom DAO code, then build it with ORM, then using a profiler measure the object creation, memory usage and GC frequency. You'll definitely see the difference (as I have).
  65. Re: No ORM: The simplest solution?[ Go to top ]

    and training on ORM
    and, of course, I don't need training. I am using ORM that conforms to a JSR standard. I and my co-developers have been using JDO for years. Switching to another JDO product requires minimal effort. Of course, when someone else comes across your home-grown solution, such years of experience is wasted - then they will need training to use your solution.
  66. Re: No ORM: The simplest solution?[ Go to top ]

    and, of course, I don't need training. I am using ORM that conforms to a JSR standard. I and my co-developers have been using JDO for years. Switching to another JDO product requires minimal effort.
    The training I'm referring to is the training to use ORM, not a specific tool. The ramp-up time I've seen for seasoned developers to begin using an ORM tool (pick one) is significant. It requires formal training as well as time to come up to speed on live projects. For example, in order to be flexible tools such as Hybernate have their own query language. There are configurations parameters that must be understood, etc. Even if already familiar with ORM, there is some training involved, and much more if not experienced on ORM.
    Of course, when someone else comes across your home-grown solution, such years of experience is wasted - then they will need training to use your solution.
    Really? You would have trouble understanding the concepts of a base class? Sure, it may take a few minutes, a couple of hours, maybe 1/2 day for a very slow person. But nothing compared with learning a full-blown ORM tool. I've experienced and observed both, and the learning time required for "home-grown" is essentially nothing compared with that required of an ORM tool. Again, it's another cost that ORM evangelists fail to recognize.
  67. Re: No ORM: The simplest solution?[ Go to top ]

    and, of course, I don't need training. I am using ORM that conforms to a JSR standard. I and my co-developers have been using JDO for years. Switching to another JDO product requires minimal effort.

    The training I'm referring to is the training to use ORM, not a specific tool. The ramp-up time I've seen for seasoned developers to begin using an ORM tool (pick one) is significant. It requires formal training as well as time to come up to speed on live projects. For example, in order to be flexible tools such as Hybernate have their own query language. There are configurations parameters that must be understood, etc. Even if already familiar with ORM, there is some training involved, and much more if not experienced on ORM.
    My experience is that this is certainly not the case - certainly not for formal training. The reason is that ORM is a pretty obvious approach. Query languages like those used by Hibernate and JDO are going to be very familiar to those who have used SQL and plain Java respectively.
    Of course, when someone else comes across your home-grown solution, such years of experience is wasted - then they will need training to use your solution.

    Really? You would have trouble understanding the concepts of a base class? Sure, it may take a few minutes, a couple of hours, maybe 1/2 day for a very slow person.
    This not the case. In this thread a whole range of wheel-reinventions has been discussed; furthermore it is stuff that is handled transparently by most ORMs. It is far more than just a base class. Things like home-grown mapping solutions, cacheing etc.
    But nothing compared with learning a full-blown ORM tool. I've experienced and observed both, and the learning time required for "home-grown" is essentially nothing compared with that required of an ORM tool. Again, it's another cost that ORM evangelists fail to recognize.
    No, it isn't. I have been using full-blown ORM for years and I simply don't recognise the situation you describe. As I keep mentioning, the documentation on the JPOX site is great, and should allow anyone to get going with JDO in a matter of a few hours. There may be tuning issues, and performance matters, but they can be addressed later. In fact if you purchase a commercial ORM of good quality, they will help address these issues for you, saving you considerable expense. It may take time learning things if you are going to use all the features of full-blown ORM, but who says we are? For the mapping of a simple object model and the persistence and retrieval of a moderate number of objects, things are easy.
  68. Completely agree :)
  69. I mean I completely agree with what Scot Bellamy has said. (TSS still does not allow me to edit my posts :(, but it should if no one has replied. As you probably know, we non-native English speakers make a lot of mistakes while posting :). Post preview is good feature, but sometimes fingers are faster :) )
  70. Re: No ORM: The simplest solution?[ Go to top ]

    I mean I completely agree with what Scot Bellamy has said.

    (TSS still does not allow me to edit my posts :(, but it should if no one has replied. As you probably know, we non-native English speakers make a lot of mistakes while posting :). Post preview is good feature, but sometimes fingers are faster :) )
    And I completely agree with you about this comment :) Even so, post preview is a good step forward from the way things were.
  71. Re: No ORM: The simplest solution?[ Go to top ]

    For example, as I am writting this, TheServerSide site is sooo slooow. Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)? When I am visitng, for example, PC hardware forums that are written in PHP, they are never so slow and they are much more feature rich. Why is that? Because they use simple tool for simple job and not complex tool for simple job. Though, sometimes we have to use complex tools to solve complex job and then we have to invest time in frameworks, but I believe this is rather an exception then the rule.
  72. Re: No ORM: The simplest solution?[ Go to top ]

    For example, as I am writting this, TheServerSide site is sooo slooow.

    Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?

    When I am visitng, for example, PC hardware forums that are written in PHP, they are never so slow and they are much more feature rich. Why is that? Because they use simple tool for simple job and not complex tool for simple job. Though, sometimes we have to use complex tools to solve complex job and then we have to invest time in frameworks, but I believe this is rather an exception then the rule.
    How much traffic do those PC hardware forums get? Do you really think that a re-write of TSS in PHP would cope? For some tasks, PHP is not a simple solution.
  73. Re: No ORM: The simplest solution?[ Go to top ]

    How much traffic do those PC hardware forums get? Do you really think that a re-write of TSS in PHP would cope?
    I can't talk about facts, but they probably have much more traffic then TheServerSide, becuse they have much more threads and much more posts then TheServerSide. Of course this tells nothing, but may be some indication. Regarding rewriting TSS, I am not familiar with PHP, have no idea about it. I just saw the forums that are feature ritcher, more performant, user friendlier, with more posts written in PHP. That is what I see as a forum user. However, I believe that rewriting TheServerSide to use pure JDBC with MVC and some templating engine (JSP with EL, Velocity or other) will give us much more performant site.
    For some tasks, PHP is not a simple solution.
    True. That's the point of the original post: Use right tool for the specific job.
  74. Re: No ORM: The simplest solution?[ Go to top ]

    How much traffic do those PC hardware forums get? Do you really think that a re-write of TSS in PHP would cope?


    I can't talk about facts, but they probably have much more traffic then TheServerSide, becuse they have much more threads and much more posts then TheServerSide.
    No, these aren't connected. Most of the bandwidth on sites like TSS is people reading articles and threads, not contributing. The number of threads and posts need not be any direct measure of the load placed on the site.
    Regarding rewriting TSS, I am not familiar with PHP, have no idea about it. I just saw the forums that are feature ritcher, more performant, user friendlier, with more posts written in PHP.
    As described above, you can't tell if they are more performant than alternatives - just that they are coping with the load on those sites.
    However, I believe that rewriting TheServerSide to use pure JDBC with MVC and some templating engine (JSP with EL, Velocity or other) will give us much more performant site.

    TSS uses an ORM (Kodo) which is widely recognised to give high performance, as it produces highly tuned SQL and has excellent cacheing. 'Pure JDBC' won't come close.
    True. That's the point of the original post: Use right tool for the specific job.
    Yes, but we are debating what the right tool is.
  75. Re: No ORM: The simplest solution?[ Go to top ]

    TSS uses an ORM (Kodo) which is widely recognised to give high performance, as it produces highly tuned SQL and has excellent cacheing. 'Pure JDBC' won't come close.
    My understanding is that TSS is written with Kodo and Tapestry as the proof-of-concept and for merketing reasons (for me, these are completely valid reasons). But a forum web app certainly does not have a need for ORM and component based web UI layer. Regarding pure JDBC: pure JDBC is by definition more performant then ORM as it is at lower level of abstraction. The other story is how you tune your JDBC and ORM, how you handle caching, etc... If your project is relatively simple, if relationships between your objects follow the same pattern, tuning SQL is easy. And regarding caching: two level caching that is common in modern ORM is two-edge-sword. Second level is great, improves performance but at the cost of increased memory usage. Hibernate, for example, serializes objects to attribute-value pairs and puts them in maps in second level cache! This imposes quite a memory overhead when compared how much memory would be used if objects are cached as they are (as object instances). Of course, this is not possible in ORM, because it has to be general and cache objects of all types! This is just another point where you have an unnecesary overhead in ORM.
  76. Re: No ORM: The simplest solution?[ Go to top ]

    TSS uses an ORM (Kodo) which is widely recognised to give high performance, as it produces highly tuned SQL and has excellent cacheing. 'Pure JDBC' won't come close.


    My understanding is that TSS is written with Kodo and Tapestry as the proof-of-concept and for merketing reasons (for me, these are completely valid reasons).
    That is irrelevant to the matter of Kodo being an extremely performant product (I know - I use it).
    But a forum web app certainly does not have a need for ORM and component based web UI layer.
    Of course not - I mean, they could write the whole thing in assembler - after all, it is at a lower level, and should be more performant....
    Regarding pure JDBC: pure JDBC is by definition more performant then ORM as it is at lower level of abstraction. The other story is how you tune your JDBC and ORM, how you handle caching, etc... If your project is relatively simple, if relationships between your objects follow the same pattern, tuning SQL is easy.
    You have contradicted yourself here. First you say JDBC is more performant because it is at a lower level, then you talk about cacheing (which isn't anything to do with low level JDBC). ORM is by definition potentially more performant than pure JDBC because it is at higher level of abstraction, which can transparently include cacheing. And yet again, you are re-inventing the wheel - why should I have to tune the SQL when my ORM products does that for me automatically?
    And regarding caching: two level caching that is common in modern ORM is two-edge-sword. Second level is great, improves performance but at the cost of increased memory usage. Hibernate, for example, serializes objects to attribute-value pairs and puts them in maps in second level cache! This imposes quite a memory overhead when compared how much memory would be used if objects are cached as they are (as object instances). Of course, this is not possible in ORM, because it has to be general and cache objects of all types! This is just another point where you have an unnecesary overhead in ORM.
    Firstly, ORM is a lot more than just Hibernate. There are a variety of ORMs and they have a variety of cacheing strategies. There are JDO products that have very memory-efficient cacheing, to the point where you can handle single transactions which involve hundreds of thousands or even millions of objects without major memory problems. (I know, because I have used such products for such transactions). Your argument here about unecessary overhead here is false.
  77. Re: No ORM: The simplest solution?[ Go to top ]

    Of course not - I mean, they could write the whole thing in assembler - after all, it is at a lower level, and should be more performant....
    When I said not needed, I meant not needed including productivity, maintainability and time-to-market reasons. I believe a forum site could be equaly fast build either using ORM or without it.
    You have contradicted yourself here. First you say JDBC is more performant because it is at a lower level, then you talk about cacheing (which isn't anything to do with low level JDBC). ORM is by definition potentially more performant than pure JDBC because it is at higher level of abstraction, which can transparently include cacheing.
    Transparency of caching has to do nothing with performance.
    And yet again, you are re-inventing the wheel - why should I have to tune the SQL when my ORM products does that for me automatically?
    Because you can do better job then the ORM tool.
    Firstly, ORM is a lot more than just Hibernate. There are a variety of ORMs and they have a variety of cacheing strategies. There are JDO products that have very memory-efficient cacheing, to the point where you can handle single transactions which involve hundreds of thousands or even millions of objects without major memory problems.
    I thought that in JDO, when dealing with single transaction, all caching is happening in the first level cache, which is transactional and store objects as instances. When you leave the transaction, first level cache is dropped. Thats, fine. What's not fine is the second level cache which must be concurrent too and handle requests by different Persistent Managers and also must be able to store objects of different types. So some sort of either serialization or deep object cloning is needed here, you can't keep original object instances in second level cache.
    (I know, because I have used such products for such transactions). Your argument here about unecessary overhead here is false.
    I believe that my argument about overhead in second level cache is actualy true. Of course, sometimes when you need clustered caching, you have no choice but to accept that overhead. The point is, that when you don't, and still use ORM, you are paying unnecesary penalties.
  78. Re: No ORM: The simplest solution?[ Go to top ]

    Of course not - I mean, they could write the whole thing in assembler - after all, it is at a lower level, and should be more performant....


    When I said not needed, I meant not needed including productivity, maintainability and time-to-market reasons.
    I believe a forum site could be equaly fast build either using ORM or without it.
    I strongly disagree. The point of ORM (well, why I use it) is to save time in development. It abstracts away details that would otherwise have to be hand written. This saves time.
    You have contradicted yourself here. First you say JDBC is more performant because it is at a lower level, then you talk about cacheing (which isn't anything to do with low level JDBC). ORM is by definition potentially more performant than pure JDBC because it is at higher level of abstraction, which can transparently include cacheing.


    Transparency of caching has to do nothing with performance.
    No, but cacheing does, and you don't get that with raw JDBC alone. Hence my argument that raw JDBC is not as performant as ORM. Once you start having to deal with cacheing you aren't at a low level anymore, and you might as well use a product that handles this for you.
    And yet again, you are re-inventing the wheel - why should I have to tune the SQL when my ORM products does that for me automatically?

    Because you can do better job then the ORM tool.
    Of course you can. But why bother hand-tuning a potentially substantial number of SQL statements when an ORM will do a reasonable job? If there are some statements that are critical, any decent ORM will allow you to express those in SQL and hand tune them, but to keep all queries in SQL in case they need tuning is a form of premature optimisation. If you are that interested in performance, and tuning, why bother with Java at all? The VM is an abstraction after all. We use Java because abstractions save us from having to deal with an unecessary level of details. ORM, in my view, does the same thing.
    Firstly, ORM is a lot more than just Hibernate. There are a variety of ORMs and they have a variety of cacheing strategies. There are JDO products that have very memory-efficient cacheing, to the point where you can handle single transactions which involve hundreds of thousands or even millions of objects without major memory problems.

    I thought that in JDO, when dealing with single transaction, all caching is happening in the first level cache, which is transactional and store objects as instances. When you leave the transaction, first level cache is dropped. Thats, fine. What's not fine is the second level cache which must be concurrent too and handle requests by different Persistent Managers and also must be able to store objects of different types. So some sort of either serialization or deep object cloning is needed here, you can't keep original object instances in second level cache.
    So don't use a second level cache. This is trivial to turn off (a single setting).
    (I know, because I have used such products for such transactions). Your argument here about unecessary overhead here is false.

    I believe that my argument about overhead in second level cache is actualy true. Of course, sometimes when you need clustered caching, you have no choice but to accept that overhead. The point is, that when you don't, and still use ORM, you are paying unnecesary penalties.
    As with so many features of ORM, if you don't need to use a feature, you simply turn it off. If you don't want to have a second level cache, because you consider it has overheads, just don't use it. No overheads, no penalties.
  79. Re: No ORM: The simplest solution?[ Go to top ]

    I strongly disagree. The point of ORM (well, why I use it) is to save time in development. It abstracts away details that would otherwise have to be hand written. This saves time.
    Yes, thats undoubtly true. But there are other ways of abstracting details then ORM: good design, code generators comes to mind. ORM is not silver bullet, especialy not that sometimes it is an overhead.
    No, but cacheing does, and you don't get that with raw JDBC alone. Hence my argument that raw JDBC is not as performant as ORM. Once you start having to deal with cacheing you aren't at a low level anymore, and you might as well use a product that handles this for you.
    I've implemented caching in several projects that used domain model and DAO/DatabaseGateway approaches. Is't very simple when you have a domain model. It's sure pain with record set approach. Couple of years ago I've also researched a little what is needed to build distributed cache infrastracture (there was no free and good distributed caches at that time) and even have built a prototype. Just to conclude that this is an extremely hard problem that I would better leave to Cameron and others to solve! I am talking about this just to show that sometimes problems that we think are hard are just easy to solve. One example is API for abstracting transactions with support only for sigle phase commit. The other example is implementing a container with declarative transactions on a top of existing Transaction manager. The third example is implementing in-process caching on a top of a domain model. The fourth example would be simple Object Request Broker (for Applet-Servlet communication for example, no RMI in M$ VM) on a top of existing serialisation. I've built these things myself and sometimes I was amased how easy they were to build, especialy when there are vendors talking how great these features are in their products. There are also hard problems to solve, like distributed caching, transparent clustering, full blown ORM. We have to distinguish these things, and use third-party tools when their usage pays off. That is the point of the original post. Sometimes (maybe not in your case), ORM is just an overhead.
    But why bother hand-tuning a potentially substantial number of SQL statements when an ORM will do a reasonable job? If there are some statements that are critical, any decent ORM will allow you to express those in SQL and hand tune them, but to keep all queries in SQL in case they need tuning is a form of premature optimisation. If you are that interested in performance, and tuning, why bother with Java at all? The VM is an abstraction after all. We use Java because abstractions save us from having to deal with an unecessary level of details. ORM, in my view, does the same thing.
    I agree with that. But with ORM it is much more likely that you will 'hit the performance wall' as ORM is on higher level of abstraction then JDBC and does not provide enough hooks to tune the system when needed. That is the problem with current state of the development. People are building higher levels of abstractions using 3GL languages (frameworks, containers...) for other people to use limiting their flexibility and control over resulting system with higher level black boxes with unsufficient hooks to tune them at lower levels. Also, frameworks are now often tested and good quality from framework user's perspecive. But when you try to hook down to implment some integration or some lower lever module for the framework, you encounter inconsistences, bugs, not so obvious and undocumented assumptions etc... That's normal, as primary audience for frameworks are framework users. But, now, as requiremets and expectations from us software developers are higher and higher, we cannot be satisfied with such black boxes. We need more power, more flexibility, more control. That is why I said frameworks are not so good at abstracting things. Currently I am very excited about Domain Specific Languages and code generation. This solves the above problem. DSLs do almost everything that frameworks do. They are risng the level of abstraction (even more then frameworks), provide more robust and coherent systems, lower maintenance, improve flexibility and all that without sacrifising control over the very bits at the low level of the system. You can tune generated code from DSL whatever you find feasible.
    So don't use a second level cache. This is trivial to turn off (a single setting).
    Turning off second level cache bogs the performance of ORM so down that I would never use ORM without second level cache. If ORM cannot cache my objects between transactions, it's useless for me. Second level cache is so vital for ORM performance. Without it, every update and delete is accompanied with a select to check for optimistic locking. Every lazy-load single object relationship (if not configured to use outer-join fetching) has to hit the DB for every transaction. Turning second level cache in my opinion is not acceptable.
    As with so many features of ORM, if you don't need to use a feature, you simply turn it off. If you don't want to have a second level cache, because you consider it has overheads, just don't use it. No overheads, no penalties.
    But I desperately want my objects to be cached between transactions :(
  80. Re: No ORM: The simplest solution?[ Go to top ]

    I am talking about this just to show that sometimes problems that we think are hard are just easy to solve.
    But that is still no reason to implement them yourself.
    We have to distinguish these things, and use third-party tools when their usage pays off. That is the point of the original post. Sometimes (maybe not in your case), ORM is just an overhead.

    But you have yet to give any convicing argument of why that should be the case.
    I agree with that. But with ORM it is much more likely that you will 'hit the performance wall' as ORM is on higher level of abstraction then JDBC and does not provide enough hooks to tune the system when needed.
    Again, what performance wall? I just don't believe you can look at a high-quality ORM product like, for example, Kodo, and claim that there aren't a huge number of hooks and techniques for tuning. Such companies have considerable number of commercial customers and have a good reputation because of such features.
    for other people to use limiting their flexibility and control over resulting system with higher level black boxes with unsufficient hooks to tune them at lower levels.
    But again, you have given no example of how or why flexibility is limited.
    Also, frameworks are now often tested and good quality from framework user's perspecive. But when you try to hook down to implment some integration or some lower lever module for the framework, you encounter inconsistences, bugs, not so obvious and undocumented assumptions etc... That's normal, as primary audience for frameworks are framework users. But, now, as requiremets and expectations from us software developers are higher and higher, we cannot be satisfied with such black boxes. We need more power, more flexibility, more control.
    I disagree, if the alternative is to implement things yourself, and generate your own bugs and assumptions. At least with a third-party product there can be thousands of users, so it is very unlikely that inconsistencies or bugs have not been discovered.
    Turning off second level cache bogs the performance of ORM so down that I would never use ORM without second level cache. If ORM cannot cache my objects between transactions, it's useless for me. Second level cache is so vital for ORM performance. Without it, every update and delete is accompanied with a select to check for optimistic locking.
    Every lazy-load single object relationship (if not configured to use outer-join fetching) has to hit the DB for every transaction. Turning second level cache in my opinion is not acceptable.
    ORM implementations aren't as simple as you imply here. Some JDO implementations will provide automatic cacheing of objects between transactions and even between different persistence managers from the same factory, This is all configurable.
    As with so many features of ORM, if you don't need to use a feature, you simply turn it off. If you don't want to have a second level cache, because you consider it has overheads, just don't use it. No overheads, no penalties.


    But I desperately want my objects to be cached between transactions :(
    You can do this. See above.
  81. I agree with that. But with ORM it is much more likely that you will 'hit the performance wall' as ORM is on higher level of abstraction then JDBC and does not provide enough hooks to tune the system when needed. That is the problem with current state of the development. People are building higher levels of abstractions using 3GL languages (frameworks, containers...) for other people to use limiting their flexibility and control over resulting system with higher level black boxes with unsufficient hooks to tune them at lower levels.
    In JPOX for instance all jars are "natevilly" OSGi bundles and JPOX has several extension points. You can hook basically everything in JPOX either using the JDO api (and in near future JPA) or using the SEVERAL JPOX extension points (OSGi Services). JPOX has a plugin engine that integrates to OSGi containers but also runs outside OSGi.


    Also, frameworks are now often tested and good quality from framework user's perspecive. But when you try to hook down to implment some integration or some lower lever module for the framework, you encounter inconsistences, bugs, not so obvious and undocumented assumptions etc...
    JPOX also has good (IMO) documentation on it's internals available. I also believe our approach is consistent with the premise of using OSGi.
    That's normal, as primary audience for frameworks are framework users. But, now, as requiremets and expectations from us software developers are higher and higher, we cannot be satisfied with such black boxes. We need more power, more flexibility, more control.

    That is why I said frameworks are not so good at abstracting things.
    I must admit that before implementing the plugin mechanism in JPOX, it was a black box, but now it has turn into a very extensible data access platform.
    Currently I am very excited about Domain Specific Languages and code generation. This solves the above problem. DSLs do almost everything that frameworks do. They are risng the level of abstraction (even more then frameworks), provide more robust and coherent systems, lower maintenance, improve flexibility and all that without sacrifising control over the very bits at the low level of the system. You can tune generated code from DSL whatever you find feasible.
    I dont know what DSL is, but frameworks currently have easy "developable" APIs and robust impls. However the ability to manage the framework in an enterprise pratform/business level is what mostly lacks in all them.
  82. For example, as I am writting this, TheServerSide site is sooo slooow. Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?
    Yes. It is a custom implementation, developed over half a dozen years, used by only one company, and it has some resulting issues. Is it a coincidence that a framework-heavy J2EE application that I recently looked at was serving 2,000,000,000 page impressions per week? Please don't take this as an argument for or against ORM. I personally suspect that on small projects, ORM could be more work than it's worth (and for that matter, so could Java ;-), but every project and every team is different. On all the projects that I see, I would always use ORM, but YMMV. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  83. Is it a coincidence that a framework-heavy J2EE application that I recently looked at was serving 2,000,000,000 page impressions per week?

    Please don't take this as an argument for or against ORM.
    For such a load one must use clustering. As I said, when clustering is needed then, of course, ORM should be seriosly considered.
    I personally suspect that on small projects, ORM could be more work than it's worth (and for that matter, so could Java ;-), but every project and every team is different.
    That is the point of the original post. And I could add that then for larger projects, when mapping objects to database tables is the only use of ORM (no need for unit-of-work, no need for clustering...) the productivity is about the same as without ORM but with well designed data access code.
  84. Re: No ORM: The simplest solution?[ Go to top ]

    For example, as I am writting this, TheServerSide site is sooo slooow.

    Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?


    Yes. It is a custom implementation, developed over half a dozen years, used by only one company, and it has some resulting issues.

    Is it a coincidence that a framework-heavy J2EE application that I recently looked at was serving 2,000,000,000 page impressions per week?

    Please don't take this as an argument for or against ORM. I personally suspect that on small projects, ORM could be more work than it's worth (and for that matter, so could Java ;-), but every project and every team is different. On all the projects that I see, I would always use ORM, but YMMV.

    Peace,

    Cameron Purdy
    Tangosol Coherence: The Java Data Grid
    I do. Even on apps as small as 2 pages. The value of consistency that I get by using Spring/Hibernate even on very small apps is worth more than the slight speed increase I would get by say embedding SQL in an jsp page. I've personally found that ORM makes my job easier and allows for more maintainable code. In fact, one of our guys, a non-developer was commenting on how much different our company is than it was a few years ago. How much more reliable our releases are and how much faster our turn around is. This occurred as our reliance on straight JDBC, internal factories(if people used them!), and jsp was replaced by Spring, Struts, and Hibernate. And his wasn't the only comment. The people who go out test, validate, or train on our apps have commented about how much better our software is.
  85. I do. Even on apps as small as 2 pages. The value of consistency that I get by using Spring/Hibernate even on very small apps is worth more than the slight speed increase I would get by say embedding SQL in an jsp page.

    I've personally found that ORM makes my job easier and allows for more maintainable code.

    In fact, one of our guys, a non-developer was commenting on how much different our company is than it was a few years ago. How much more reliable our releases are and how much faster our turn around is.

    This occurred as our reliance on straight JDBC, internal factories(if people used them!), and jsp was replaced by Spring, Struts, and Hibernate.

    And his wasn't the only comment. The people who go out test, validate, or train on our apps have commented about how much better our software is.
    You can sure have consistency and 'well-designness' without Spring/Hibernate, but if they do help your organization to achieve that then your choice in your case was right.
  86. I do. Even on apps as small as 2 pages. The value of consistency that I get by using Spring/Hibernate even on very small apps is worth more than the slight speed increase I would get by say embedding SQL in an jsp page.

    I've personally found that ORM makes my job easier and allows for more maintainable code.

    In fact, one of our guys, a non-developer was commenting on how much different our company is than it was a few years ago. How much more reliable our releases are and how much faster our turn around is.

    This occurred as our reliance on straight JDBC, internal factories(if people used them!), and jsp was replaced by Spring, Struts, and Hibernate.

    And his wasn't the only comment. The people who go out test, validate, or train on our apps have commented about how much better our software is.


    You can sure have consistency and 'well-designness' without Spring/Hibernate, but if they do help your organization to achieve that then your choice in your case was right.
    But see, I don't WANT to write all the code needed to do what Spring and Hibernate does. I also don't want to write a database or operating system.
  87. But see, I don't WANT to write all the code needed to do what Spring and Hibernate does. I also don't want to write a database or operating system.
    But you DO NOT have to write code for everything Spring and Hibernate does, of course, only the stuff you need on your project! That is the point: what YOU realy need on YOUR very project.
  88. I agree with the poster. For simple to medium web apps with no complex transactions this is better (lighterweigh, but sufficient) approach than full blown ORM framework.
    Yes, it may be easier, but thinking in enterprise applications, you should have a consistent approach for data access no matter the size of the application. That said, I think you either choose one and embrace it on all application sizes, or give freedom of choice to developers and face a mess in long term.
  89. If them you decide to go for an ORM, go for a mature solution, enterprise architecture aware with long term vision of data access and not simply ORM. A solution that fits in your organization and allows you to manage and monitor technical and functional aspects.
  90. I do agree too. I use the same kind of solution, including SQL in a separate file for later optimizations and maintenance. I have added a utility class for datasource, connection, try/catch convenience, and so on. The architecture stay simple, and relies on the DAO pattern. I have already used some ORM framework in the past but face some optimization problems with "lazy" loading. I have noticed that the framework were loading most of the data in the cache. When you have a lot of data, it sucks! The lazy loading was not effective with the compound key and the mapping used. I have found a solution in a forum but we would have to change the mapping. Not a good idea at the end of a project. I will not tell which framework I used in order to avoid a thread "my framework is better that X and won't meet this problem".
  91. I have already used some ORM framework in the past but face some optimization problems with "lazy" loading. I have noticed that the framework were loading most of the data in the cache. When you have a lot of data, it sucks!
    The lazy loading was not effective with the compound key and the mapping used. I have found a solution in a forum but we would have to change the mapping. Not a good idea at the end of a project.
    I will not tell which framework I used in order to avoid a thread "my framework is better that X and won't meet this problem".
    But this is where some of the more powerful commercial ORMs have most success - in the handling of large volumes of data, even within single transactions, and with tuning. Some of the more popular ORMs aren't tuned for that kind of processing (and indeed, in some cases, advise that they aren't used in that role). But this does not mean this is a general problem with ORM. I really do get the impression here that people are having bad experiences with certain ORM products which might not have the best performance. But to take that experience, and then claim (as some have here) that the hand-coded JDBC 'write-your-own-framework' approach is a broad solution is just plain wrong, and ignoring the facts: ORM is increasingly widely used because it works. The users of large commercial ORM products aren't being deliberately perverse - they are using these products because they deliver real value in terms of performance, scalability and development time. And also, I feel we should not be talking about frameworks. We should be talking about products that implement standards. That way, if there is a tuning issue, or other problem, you can switch vendor (or threaten to do so - often a useful strategy). This is one of the reasons I have stuck with JDO over the years rather than going for non-JSR solutions and I have made use of the ability to change supplier. But anyway... I guess that is as much as I can usefully contribute.
  92. Re: No ORM: The simplest solution?[ Go to top ]

    I used Hibernate last year for a couple of projects. It's a great product, but I often found it too cumbersome for simple projects. I tried Spring JDBC after that, and I like it a lot. Spring got it right by minimizing checked exceptions and eliminating micro-management of objects like the Statement and the ResultSet that distract you too much from what you're trying to accomplish. My experience with database programming is that most of the unique effort in a new project (as opposed to time or lines of code) goes into writing queries. The ORM tools don't really automate that. They give you different ways to write queries, but the amount of effort is the same whether you're writing SQL or HQL or building a query with Java objects. One thing I noticed with my Spring JDBC programming is that I was writing a lot of repetitive code to insert, update, delete and get (by primary key) objects from individual tables. So I factored that code up to a base class and made my own little framework augmentation. It might have been nice if the Spring people did something like this, but on the other hand, it would probably have been a lot more complicated than the solution I wrote to meet my direct needs.
  93. One thing I noticed with my Spring JDBC programming is that I was writing a lot of repetitive code to insert, update, delete and get (by primary key) objects from individual tables. So I factored that code up to a base class and made my own little framework augmentation. It might have been nice if the Spring people did something like this, but on the other hand, it would probably have been a lot more complicated than the solution I wrote to meet my direct needs.
    Please, do not hesitate to do things like this! Spring people din't do it as they felt they shouldn't probably. But with generics in jdk 1.5 writing generic DAOs is much more feasible. I am sick of hearing 'do not build yourself this, it's alredy build'. Sometimes, like in your case, the thing is NOT built as you need it. And sometimes it is easier to build something if it is simple, then finding, researching, learning already build solution. It is just more offten the case then people ususally think.
  94. Re: No ORM: The simplest solution?[ Go to top ]

    I used Hibernate last year for a couple of projects. It's a great product, but I often found it too cumbersome for simple projects.

    I tried Spring JDBC after that, and I like it a lot. Spring got it right by minimizing checked exceptions and eliminating micro-management of objects like the Statement and the ResultSet that distract you too much from what you're trying to accomplish.

    My experience with database programming is that most of the unique effort in a new project (as opposed to time or lines of code) goes into writing queries. The ORM tools don't really automate that. They give you different ways to write queries, but the amount of effort is the same whether you're writing SQL or HQL or building a query with Java objects.

    One thing I noticed with my Spring JDBC programming is that I was writing a lot of repetitive code to insert, update, delete and get (by primary key) objects from individual tables. So I factored that code up to a base class and made my own little framework augmentation. It might have been nice if the Spring people did something like this, but on the other hand, it would probably have been a lot more complicated than the solution I wrote to meet my direct needs.
    I had the opposite experience. Even simple I can very easy put things together. Things like Xdoclet have allowed me to put domain objects together quickly which excellent mapping. Spring/Hibernate has proven a very strong combination for me. That base class your wrote I got almost for free using the SpringHibernateTemplate.
  95. That base class your wrote I got almost for free using the SpringHibernateTemplate.
    I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :) I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?
  96. Re: No ORM: The simplest solution?[ Go to top ]

    I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

    I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?
    But let's remind ourselves of the code that you have implemented yourself (or suggested you would implement) in place of ORM: 1. Mapping 2. DAO objects 3. Callbacks 4. Cacheing 5. Connection pooling 6. Externalised SQL queries That is more than a few hundred lines of code. And anyone with any good of experience ORM will realise that the set-up code to use ORM in the simplest situation is only a few lines - a few lines in a config file, a few lines of mapping (the rest can be left for defaults), and a few more lines to obtain a persistence manager (or its equivalent). If you you really want to argue about how few lines it takes to set up an application to persist and retrieve objects (especially related objects in a data model), a good ORM can easily beat alternative solutions. ORM is the equivalent of 'System.out.println' - alternatives are the equivalent to obtaining the 'out' stream and pushing out bytes individually yourself - the fancy way :) Java provides good frameworks for stream handling (they may be built-in, but they are still frameworks). They are great abstractions, and they are efficient and save time.
  97. That base class your wrote I got almost for free using the SpringHibernateTemplate.


    I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

    I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?
    Right. If Frank supplied CRUD operations Declarative Transaction support Caching Lazy instantiation IOC AOP Dynamic SQL general Automatic loading of object graphs all in 50-100 lines of code, he should release a framework of his own! He'll put everyone else out of business.
  98. Re: No ORM: The simplest solution?[ Go to top ]

    That base class your wrote I got almost for free using the SpringHibernateTemplate.


    I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

    I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?


    Right. If Frank supplied
    CRUD operations
    Declarative Transaction support
    Caching
    Lazy instantiation
    IOC
    AOP
    Dynamic SQL general
    Automatic loading of object graphs

    all in 50-100 lines of code, he should release a framework of his own!

    He'll put everyone else out of business.
    It wasn't Frank. It was from here: http://www.theserverside.com/blogs/thread.tss?thread_id=41715#215595 But the point is the same.
  99. Re: No ORM: The simplest solution?[ Go to top ]

    That base class your wrote I got almost for free using the SpringHibernateTemplate.


    I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

    I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?


    Right. If Frank supplied
    CRUD operations
    Declarative Transaction support
    Caching
    Lazy instantiation
    IOC
    AOP
    Dynamic SQL general
    Automatic loading of object graphs

    all in 50-100 lines of code, he should release a framework of his own!

    He'll put everyone else out of business.


    It wasn't Frank.

    It was from here:
    http://www.theserverside.com/blogs/thread.tss?thread_id=41715#215595

    But the point is the same.
    It wasn't a shot a Frank(or even the other guy). I was just addressing Cekovic.
  100. I am happy to find a healthy discussion of my post - http://blog.taragana.com/index.php/archive/super-simple-java-framework-for-sql-happy-developers/ If you are interested, I have posted details of my simple database framework - http://blog.taragana.com/index.php/archive/java-database-framework-orm-replacement-in-80-lines-of-code/
  101. What are we paid for ?[ Go to top ]

    I work for a company making products (software) for a specific industry (I can't tell) lets say Peanut Butter. We have the following requirements : - support 5 to 6 different databases vendors. - support small to very big amount of data. - support fat clients and web clients - provide some very complex business features. - beat our competitors - make money As an architect (specialized in powerpoint and buzz words, of course) I have to provide my organization with processes, designs and architectures respecting those requirements. Everytime I choose to code "in house" infrastructure components (opposite to business component) I ask myself those questions : "Am I paid to develop a caching component, will my company make money out of this ?" "Do we have to maintain industry component or infrastructure components ?" "How easy will it be to hire somebody senior in our completely inhouse, undocumented, barely unit tested framework versus somebody knowing this well known, documented, unit tested, books-written-on open source framework ?" "What benefit will I add to my resume with this unknown inhouse framework in case I have to go back hunting for a job ?"
  102. I think it obviously depends on the app we develop, but we have a couple of factors that are not app related: - the time we have til the deathline - joy of learning - competition with other programmers it's all right that the learning curve is high, but that's a test for your competitiveness. Personally I think learning is the best thing of this job. once we get the knowlegde we can be more productive and we finish it up earlier a significant side of the app. on the other hand, I think orm apps are quite easier to mantain and, with no doubt, to port. my opinion is that jdbc code is messy, boring and hard to fix. so, if we are a this point, why to come back to caves? cheers, jaime
  103. Re: No ORM: The simplest solution?[ Go to top ]

    What do you think of this bare-bones approach? Are you bypassing or embracing ORM?
    I was quite enthusiastic using ORM months ago. It was hibernate to give it a name. I gave me some quick success after a hard start at the begining. But then it gave another hard time and then another and... Right now I got a bit skeptic with the experience I've gained up to now and it's not only that hibernate is now jbo**. One point I very much dislike is the intransparency that comes with hibernate's on the fly byte-code instrumentation. Try to debug hibernate stuff if you have to find the reason of some strange behaviour or failure. Hm, difficult... A failure might not be due to the ORM tool. It might be the programmer once familiar with jdbc now has to become familiar with hibernate. One has to learn the specific, sometimes subtle and very often 'well' documented peculiarites of such a tool. And beliefe me, such peculiarities will show up. And they will show up, if you don't need them...So you definitly have to learn a ORM tool in depth with all it's consequences. Extend scope and think of your project that applies let's say two frameworks which apply byte code instrumentation and spice up the soup with some annotation stuff. Who can tell what kind of soup this will be? The result is a intransparent mess by the power of x. I would fell to be at the Tools mercy. And I would not like the kind of feeling. And I would not like to be at the tool vendors mercy...jboss ;-) And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem. What about a distributed transaction, multi level cache ORM solution for a portlet that reads just some gibberish from the db to display it. What is your solution? What is the solution? I don't know. I have the task "investigate code generation" on my agenda. I think this would leave me with more transparency and control. What do you think about code generation? Any experience? Any framework?
  104. Re: No ORM: The simplest solution?[ Go to top ]

    A failure might not be due to the ORM tool. It might be the programmer once familiar with jdbc now has to become familiar with hibernate. One has to learn the specific, sometimes subtle and very often 'well' documented peculiarites of such a tool. And beliefe me, such peculiarities will show up. And they will show up, if you don't need them...So you definitly have to learn a ORM tool in depth with all it's consequences.
    To some degree, but the advantage of increasing use of JSR specifications for ORMs means that this is much less the case. JDO 2.0 gives you a very rich API and feature set that is present in all implementations - you don't so much need to learn one ORM tool - you learn the specification. My experience is that there are fewer vendor-specific pecularities with JDO 2.0 because so much of what you need is clearly stated in the specification. Of course, there may be some quirks in the tools provided to add functionality beyond the spec, such as schema generation, but for a simple case these aren't needed.
    Extend scope and think of your project that applies let's say two frameworks which apply byte code instrumentation and spice up the soup with some annotation stuff. Who can tell what kind of soup this will be? The result is a intransparent mess by the power of x.
    I have experience of this and I don't believe it need be messy. I have applied Aspect/J to JDO enhanced classes. The JDO enhancement did not appear to get in the way of my ability to trace through the aspects.
    I would fell to be at the Tools mercy. And I would not like the kind of feeling. And I would not like to be at the tool vendors mercy...jboss ;-)
    Then stick with standards - use JDO 2.0 or JPA. I have personal experience of transferring applications between different JDO vendors, and it is not a significant amount of work - you aren't left at the mercy of a vendor. This is one of the main reasons I have stuck with JSR standards over the years.
    And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem. What about a distributed transaction, multi level cache ORM solution for a portlet that reads just some gibberish from the db to display it. What is your solution?
    My solution is simple - turn off the distributed transactions and multi-level cache! Use the ORM as a simple and lightweight framework above your database. Just because an ORM implementation has all the features, does not mean you need to use them all.
  105. Re: No ORM: The simplest solution?[ Go to top ]

    And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem.
    We call it Keep It Simple and Straightforward. You can avoid most of the problems you mentioned by using JDX OR-Mapper. There is no byte code processing and no new query language to learn. Some of the KISS principles it follows are: - Solve the most important problem (object relational impedance mismatch) in the simplest possible way. - Don’t make the solution more complex than the original problem. - Be completely non-intrusive to the object model. - Make it easy to define, modify, comprehend, and share the mapping specification. - Avoid source code generation for data access. - No mind reading. - Avoid creating a new query language. - Expose small number of simple and consistent APIs. - Optimize data access logic automatically. - Stick to 90/90 rule about product features. - Offer intuitive tools to deal with object models, database schema, and the mapping. Free evaluation version of JDX is available at http://www.softwaretree.com. Hope you will give JDX a try. Thanks, -- Damodar Periwal Software Tree, Inc. Simplify Data Integration
  106. Re: No ORM: The simplest solution?[ Go to top ]

    And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem.

    We call it Keep It Simple and Straightforward. You can avoid most of the problems you mentioned by using JDX OR-Mapper. There is no byte code processing and no new query language to learn.
    Why should anyone care about byte code processing? This can be a fully automated step and totally transparent. The JPA specification does not require such processing anyway. Also, you don't need to learn a new query language with JDO 2.0. The query language is Java syntax. Also, you can use normal SQL for querying if you want to. The query language for JPA is so close to SQL that no-one familiar with SQL should have any trouble using it. I see little advantage in this proprietary product when there are very good implementations of JDO 2.0 and JPA that solve the same problem and are free. However, if your tools are so good, surely you could get a larger market by implementing JPA, rather that by encouraging developers to use a single-vendor solution?
  107. Re: No ORM: The simplest solution?[ Go to top ]

    Well! Mr chakrabarty has indeed done a fantastic job by simplifying things out.But, all depends on situations, in bare bone areas where you don't need all the powerful features of an ORM framework, where your application just need to fire some queries and get back the results, we can surely depend on this type of simple stuff.Infact during my initial days of training in my company, I was told to write a jdbc driver with all the logging facilities and caching of statements.So, the stuff we wrote as merely assignments can be of great use provided need be so.Angsuman's framework is just a wrapper over a very well developed api and with all respect,it will be hard for me to declare it a fully fledged framework. It would rather of great help if his framework also support transaction management and offcourse rollbacks. cheers, http://javaicillusion.blogspot.com/
  108. Maybe the problem is the O[ Go to top ]

    I've been following various ORM discussions with interest, as I'm trying to decide myself whether ORM is good idea (as opposed to just using SQL). So I can't speak from experience in using ORM, apart from dabbling with some tools to see how they work. I can speak from the standpoint of 20+ years experience in the industry though. I spent about half my programming years doing non-OO programming, rode the revolution and then the other half OO programming, mostly in C++ I'll admit, and of course SQL. I'm starting to come down on the side of using raw SQL with minimal interfacing to the object model. Like some comments I've seen, partly that is because SQL to me is already the definitive language for manipulating relational data. It does this so well, I don't really want to put anything in the way. But this idea I wanted to raise for discussion regards the assumption behind all of this, that an object model is essential. I guess my viewpoint might be a bit longer having started before OO was all the rage, but why do we assume that object model good/data model bad? This may seem heretical, but I guess I never really converted to the OO religion (although at the time I was pretty excited about it). To me it's just another tool to use where it is useful. Hence the discussion about ORM. Since we assume that we "must" have an object model as our base, then we must find a way to map from that to the relational model, which is really inconvenient because of this "impedence mismatch". So the assumption is we must have O, have to tolerate R therefore need M. Many people want R to disappear so they can focus on O. ORM seems like a means to that end, but what we seem to end up with is a lot of complexity to ram that square O peg into the R hole. Which personally I don't like so much. So the question I ask is, why do it? Why not use OO as a tool instead of using it to build a data model replacement? Then we can use our really useful RDBMS, use the best language to manipulate it (SQL) and use lose the need to map between two models. And as a bonus, have a common language with DBAs. We can still use OO programming techniques, just don't use an OO model as the centre of the application. Instead use a data model and implement the business rules using a language that may well be OO, but with the objects being used in a supporting role. Opinions?
  109. Re: Maybe the problem is the O[ Go to top ]

    but why do we assume that object model good/data model bad?
    IMHO, when using an OO-language like Java for application development, it’s natural, appropriate, and advantageous to represent domain data as instances of object model classes. Not using object model for domain data would be like having a horse but walking with it instead of riding on it. Using ORM makes the ride faster and easier.
    So the assumption is we must have O, have to tolerate R therefore need M.
    Well put. I would say that we may even want to use R for its own advantages. -- Damodar
  110. No ORM: VORUTA[ Go to top ]

    I think the Voruta (http://voruta.sourceforge.net/) complies with all requirements!
  111. we had a project which should support 3 databases, MySQL, Oracle, MS-SQL he told me he knows that ORM tools can not generate good SQL he claimed he is good dba and can write very optimized SQL so I gave him DAO and Abstract Factory pattern and asked him to use this technique instead. soon he reached that he should generate SQL code with a template engine! (this is a very simple ORM mapper!) I see no reason other than lazyness or low quality of OR mappers for not using them.
  112. OR make your life easy, use db4o[ Go to top ]

    ORM or no ORM, with db4o, you can forget about mapping files, DDLs, and SQL queries and use a single line of code to store your entire object graph: db.set(myObject); And write refactorable queries in the programming language: List contacts = db.query(new Predicate() { public boolean match(Contact contact) { return contact.getName().equals("some string"); } }); Now what's the simplest solution?
  113. No ORM for Me[ Go to top ]

    No way, I was using an ORM for years to my benefit, but then I read this blog post linked to from TheServerSide.com, and now I'm 100% ORM free. I mean who needs "frameworks", right? ORM's are for idiots who don't like to write thousands of lines of custom code that only they know how to debug and fix. Silly fools. Who needs to bother with frameworks that allow you to easily switch between different databases, I mean who ever needs to do that. Not me, all of my code always works with DB2, and, really, tell me who in the world would want to use another database. DB2 forever. Thanks for this blog post, it was really informative, I hope others will read it and adopt your rejection of ORM frameworks.
  114. Re: No ORM for Me[ Go to top ]

    No way, I was using an ORM for years to my benefit, but then I read this blog post linked to from TheServerSide.com, and now I'm 100% ORM free. I mean who needs "frameworks", right? ORM's are for idiots who don't like to write thousands of lines of custom code that only they know how to debug and fix. Silly fools. Who needs to bother with frameworks that allow you to easily switch between different databases, I mean who ever needs to do that. Not me, all of my code always works with DB2, and, really, tell me who in the world would want to use another database. DB2 forever. Thanks for this blog post, it was really informative, I hope others will read it and adopt your rejection of ORM frameworks.
    Before writing a useless rant, you probably should have read through at least some of the repsonses. There are very good arguements both for and against ORM. Many of the responses opposing ORM don't deny it's advertised benefits, but simply claim that the costs of ORM out way the benefits, therefore supporting the original post.
  115. Recently, I am developping a new project 'dbcoat' It seems to be suitable for your situation. Please refer to: http://sourceforge.net/projects/dbcoat
  116. Re: No ORM for Me - NOT![ Go to top ]

    Many of the responses opposing ORM don't deny it's advertised benefits, but simply claim that the costs of ORM out way the benefits, therefore supporting the original post.
    Sure, if you get a kick out of writing lots of boring, mechanical SQL glue code that can be much more quickly and efficiently be produced by an ORM then an ORM is going to rain on your parade.
    If, however, you like writing the POJOs that actually model the problem domain accurately and provide the foundation on which to provide fast to develop and easy to maintain solutions for your clients then you'll probably see that the benefits of using an ORM that allows truly transparent persistence of POJOs FAR OUTWEIGHT the costs of having to write roll your own SQL and 'hide' that ugliness in DAOs. I haven't written a DAO in years - use the 'exposed domain model pattern' and watch your productivity hit the roof. (http://expojo.com - shameless plug for an open source implementation of that pattern)
    I think these arguments are far more about some people being stuck in their old SQL ways and unwilling to embark on the learning curve that could take them to the next level of productivity rather than proper technical justifications for remaining in a 1970's relational database centric world. Move along lads, there's nothing to see here!
  117. The greatest problem on any project is complexity. The most used recommendation and always the missing requirement is: simplicity. Anyone can talk about KIS (Keep It Simple), no one really re-engineer a data model for sake of simplicity. The most complex and serious database I've seen have a common feature: don't try to enter into my world (relational database, DBA). The last frontier for a developer is a simple VIEW or a Store Procedure. It's curious because the two worlds are clearly delimited; nonetheless an ORM is very useful. Any wrapper for the JDBC is great, a silly developer will use it improperly, so it's better to have simple on-way-to-use interface that avoid any problem that can be automated or solved behind the scenes. My opinion is that ORM is OK, and must be used to interconnect to worlds: the Object Oriented model and the Relational Database model. Nonetheless, In My Humble Opinion, it is crazy to let any non-DBA to write SQL. One sample: they are laziness (I am too) and if you can write a query for accessing 4 tables instead of 4 queries ... you will do whatever you need to finish tasks earlier (when no time is available, nobody takes care about performance). The potential problem behind this common behavior is obvious. Another sample, I met two serious DBA that have one rule: it is strictly forbidden to access 2 or more tables in a query. Yes, this also surprised me. When you have several million records per table, any query accessing multiple tables could be fatal. The solution is also fatal (you don’t need a great database engine to do a simple query on a single table). Another DBA do not agree with me, but when he had to re-engineer 10 pages one-query (yes I’m not wrong) … then he really loves that “restriction”. It’s all about QA (Quality Assurance). To guarantee quality, you must control the performance. In other words you must set the minimum and maximum time a transaction must require, and also you must know the average response time of each one. When these boundaries are not set, all is a big mess; nothing can be done. Long time ago I remember some enterprise product limit a transaction time to a 0.5 seconds (which I really love). Once this threshold is reached, the transaction is aborted. That is the way to go. Regards, Jordi Ferran
  118. The most complex and serious database I've seen have a common feature: don't try to enter into my world (relational database, DBA). The last frontier for a developer is a simple VIEW or a Store Procedure. It's curious because the two worlds are clearly delimited; nonetheless an ORM is very useful. Any wrapper for the JDBC is great, a silly developer will use it improperly, so it's better to have simple on-way-to-use interface that avoid any problem that can be automated or solved behind the scenes.
    At my company we had similar issues. As I mentioned in a previous post, stored procedures are very poor application containers, leading to poor performing applications that do not scale well. In leading the application architecture group, we spent time working closely with the DBAs, and putting together Proof of Concept (POC) to show the benefits of embedded SQL. We also worked to address the concerns of the DBA group to ensure that security and other concerns were address appropriately. We then proceeded to eliminate all strored procedures from the key application (more than 200), and re-architected using embedded SQL, written primarily by developers. Both scalability and performance of the application were greatly improved, and the application is no less secure than it was before. You speak about the mistakes that developers can make writing SQL and using JDBC wrappers. My experience has been that developers with little or no experience with ORM tools make far more and costlier mistakes with ORM than without it. You suggest we need to keep it simple, and I couldn't agree with you more. One must consider the complexities introduced by an ORM tool as well. What is interesting about your assertion is that one of the major benefits claimed by ORM won't be realized if stored procedures are utilized. It seems that using stored procedures provides more arguements against ORM than for it.
    My opinion is that ORM is OK
    I agree, ORM is OK. However, my experience has been that the costs of ORM exceed its benefits.
    It’s all about QA (Quality Assurance). To guarantee quality, you must control the performance. In other words you must set the minimum and maximum time a transaction must require, and also you must know the average response time of each one. When these boundaries are not set, all is a big mess; nothing can be done. Long time ago I remember some enterprise product limit a transaction time to a 0.5 seconds (which I really love). Once this threshold is reached, the transaction is aborted. That is the way to go.
    I couldn't agree with you more. The best way to control performance is with embedded SQL, not with stored procedures or ORM. With a proper review process in place that involves experienced DBAs, you won't get better performance or quality with embedded SQL. You'll also find the overall solution is "simple".