No ORM: The simplest solution?

1. I needed a way to put all the SQL queries and DDL's in a separate file. This allows me or a DBA to later analyze the query with a fine tooth comb and optimize if necessary. It also allows me to easily change them without changing the code. Most of all cleanliness of the solution is appealing. At this point I am sure you are thinking of iBatis. I tried iBatis. Initially I liked it and thought I had my solution. However as I went down the lane I realized it too gave me features that I didn't need. Even this was more complicated than I needed. All I needed was a HashMap saved to a file in XML format. And my database class should support query execution by name (think key-value).

2. Secondly I needed connection pooling to prevent opening and closing too many connections and also running out of connections. I found a nice solution in Proxool. Additionally it supports having multiple connection profiles in a simple text file.

3. I needed to integrate these two capabilities in a simple database class along with utility methods like cleanly closing connection and logging the query data.

Re: No ORM: The simplest solution?

What do you think of this bare-bones approach? Are you bypassing or embracing ORM?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

I agree with the poster. For simple to medium web apps with no complex transactions this is better (lighterweigh, but sufficient) approach than full blown ORM framework.

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

And this approach is no less OO then using an ORM mapper. You can still (and should) have domain model, although it is not transactional.

If your transactions are simple, unit-of-work that ORM provides is just an overhead and major bloat.

Adding support for caching is also relatively easy with this approach.

Re: No ORM: The simplest solution?

For me, lightweight is having to write less code because an tried-and-tested ORM product does things for me.

The one thing I would say is not just a matter of experience and opinion is that the non-ORM approach is not as OO - ORMs usually allow easily use of full OOP features such as inheritance, in contrast to other approaches.

Re: No ORM: The simplest solution?

For me, lightweight is having to write less code because an tried-and-tested ORM product does things for me.

Major impact to productivity with full blown ORM with unit-of-work (especially in stateless environments) is that your object had to be 'connected' in some way to ORM framework. Either it is Hibernate Session, Persistence Manager or other. Disconnecting and reconnecting to for example Hibernate Session is rather pain in statless environments. No meter how authors of ORM framework try to make it easy, it is still pain, because of the nature of the problem (having to support unit-of-work, lazy load and such stuff in transparent way).

The one thing I would say is not just a matter of experience and opinion is that the non-ORM approach is not as OO - ORMs usually allow easily use of full OOP features such as inheritance, in contrast to other approaches.

This simply is not true. You can design your domain model without ORM even more freely and with less constraints then with ORM! Modern ORM does have constraints on your POJOs, small and not worthy constrains, but they have. Try declaring final getter for association property and see if it can be lazy loaded with Hibernate.

Re: No ORM: The simplest solution?

JDO 2.0 has very few constraints indeed. When there are such constraints (such as the requirement for a no-args constructor - which can be private) they can usually be implemented, if necessary, transparently by the ORM product.

Re: No ORM: The simplest solution?

Indeed "few" can be reduced to only one constraint, or no constraints at all since implementations do that for you.

JDO is a true POJO persistence API, not JPA, Hibernate and others.

Re: No ORM: The simplest solution?

Indeed "few" can be reduced to only one constraint, or no constraints at all since implementations do that for you.

JDO is a true POJO persistence API, not JPA, Hibernate and others.

Well, not quite. For example, in Collections and Maps where you want to persist the elements as separate objects rather than in a BLOB the elements have to be either persistent or immutable types such as String or Date. But this is far better than the 'entities only' restriction of JPA.

However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.

Re: No ORM: The simplest solution?

Col. elements do not have to be immutable neither persistent classes.

The only major common scenario not supported in JDO impls today is the capability of persisting collections of collections by not serializing them.

JPA should be enough for most users, however people should be aware of its limitations otherwise can get frustated with ORM in general

Re: No ORM: The simplest solution?

However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.

Re: No ORM: The simplest solution?

All of these features does not require more then couple of hundreds LOC to implement.

Re: No ORM: The simplest solution?

However, even with these restictions, it still seems to me to be far better than the bloat of having to hand-code things yourself with JDBC.

If JDBC code is well designed and structured, it's not bloat at all! Imagine this:
- you have Spring like data access API with callbacks
- you have transaction API that holds transactional resources (in this case JDBC connections) in thread local variables
- you have connection pooling,
- you have all your SQL in XML templates, from where you can fetch it, transform it (for dynamic SQL) and even make your SQL tamplates how swap when they change

All of these features does not require more then couple of hundreds LOC to implement.

With all this, your JDBC code is clean and concise, no bloat at all. If you add a bit of code generation, you will be more productive then with Hibernate if the project is medium to large in size.

The other story is if your organisation standardize on Hibernate and have tens of projects that useses it or similar thing. I was talking about pure technical productivity of the approach.

Re: No ORM: The simplest solution?

Writing a considerable number of extra lines to re-implement something that is already provided and available in existing packages via a concise API is the definition of bloat.

Modern ORMs may have a considerable number of features, but this no way means that code that uses them is bloated.

It is surely technically vastly more productive to use a product that has been well-designed and quality tested and has thousands of other users than to write all this yourself, aside from the possible future support issues that may arise when others attempt to deal with your 'home grown' solution. As your project grows there will be an increasing number of issues such as clustering and cacheing that you will need to deal with. There is no point at all in dealing with this yourself.

Re: No ORM: The simplest solution?

Writing a considerable number of extra lines to re-implement something that is already provided and available in existing packages via a concise API is the definition of bloat.

It is not big deal to wright all this stuff as you might think. For me, major bloat is seeing one hundred+ deep stack traces when using several popular frameworks together (ORM + container + web framework for example).

Modern ORMs may have a considerable number of features, but this no way means that code that uses them is bloated.

Sure they have, but these features are far beyond what average single project needs. That's my point, people end using something that they do not need for the specific project, adding complexity, dependency on third party feamework and surely pay performance price (sometimes insignificant, but sometimes mesurable).

It is surely technically vastly more productive to use a product that has been well-designed and quality tested and has thousands of other users than to write all this yourself, aside from the possible future support issues that may arise when others attempt to deal with your 'home grown' solution. As your project grows there will be an increasing number of issues such as clustering and cacheing that you will need to deal with. There is no point at all in dealing with this yourself.

Sure modern Java ORMs are high quality. Sure there are a lot of resources around about them. Sure a lot of people are familiar with them. But sometimes (more often then you might think), they are not needed at all and just can make trouble:

In one of the latest projects we used Hibernate. It was a medium size web app with a lot of need for state management. We used JSF to ease state management in the presentation layer. The result: we ended seeing Hibernate and JSF fighted each other as ORM unit-of-work was not 'compatible' with state management in JSF (connecting/disconnecting objects from ORM is a pain, at least with Hibnernate). Not that API is complex, on the contrary, but there is an 'impedance mismatch'. A lot of pepole had the same problem. At the end we had to build ourselves support for 'long session' and Seam like conversations (Seam was in early beta at that time). We could be much more productive in that project without using ORM...

And the irony was that I made a decision to use Hibernate :(
Hibernate is good. Hibernate is exelent. But for a simple web app (not small, not big, but simple) it is just an overhead.

Re: No ORM: The simplest solution?

In any web framework, you are dealing with deep stack frameworks. You get equal 'bloat' if you are dealing with traces into your own ORM implementation or if you are dealing with a standard ORM.

I still don't get why you think that reliance on third partly frameworks implies performance loss. When I use a third party framework, I am honest enough to accept that the writers of that framework might have written better, more finely tuned, code that I could have produced.

I am afraid that this suggests you have not sufficiently researched how ORMs can integrate with web frameworks, but I can understand your position - this is not a simple process.

Seam is a new solution, but there are others. I have used the JSF integration with Spring and JDO with excellent results. Now that I have this integration working, it certainly isn't an overhead, it is a re-useable approach that simplifies things for new projects.

You should investigate alternate JPA implementations such as TopLink, and you should look at the excellent JDO 2.0 specifications, and quality implementations - JPOX, Kodo, Xcalia.

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

In any web framework, you are dealing with deep stack frameworks. You get equal 'bloat' if you are dealing with traces into your own ORM implementation or if you are dealing with a standard ORM.

I and the original poster were talking not about home grown ORM implementations, but about simple home grown data access libraries, without unit-of-work, lazy load, complex mapping support etc... These libraries occupy no more then about 5 entries in the stack trace.

I still don't get why you think that reliance on third partly frameworks implies performance loss. When I use a third party framework, I am honest enough to accept that the writers of that framework might have written better, more finely tuned, code that I could have produced.

Sure, but the writers or third-party framework are building general ORM solution and home grown library is targeting specific project or group of projects and thus can be much simpler and more performant.

If we have 100+ deep stack traces, there must be are a lot of method invocation. Although it is very fast on modern JVMs, nothing is fast enough if it is repeating multiple times, passing various layers of abbstraction, which are, of course, required to build good designed general ORM solution, but not required at all for simple but usefull (in specific context) data acess library.

So, you have to build your own integration library to integrate JDO with Spring and JSF. Sorry but I have to ask how much time do you invested in it? Is your library usefull with other framework stacks, for example TopLink with Oracle EJB3 and ADF faces. I am sure it's not because these three are completely different beasts with thir own specific functionality, quirks and problems too, although some of them might be based on same JCP specifications.

That's my point, we Java programmers are lately very obsessed with frameworks (I was too) but sometimes simple solutions are far then enough!

You should investigate alternate JPA implementations such as TopLink, and you should look at the excellent JDO 2.0 specifications, and quality implementations - JPOX, Kodo, Xcalia.

The time needed to investigate all this is far beyond writting my own stuff that will fit my own project needs!

I believe that time is much better invested in the concepts of how these frameworks work, how they are designed, what patterns they use, and steal ideas from them, rather then investing time in how this framework integrates with that, what are the constraints of this framework and what are the quirks of n.mm version of that framework.

Re: No ORM: The simplest solution?

For example, as I am writting this, TheServerSide site is sooo slooow.

Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?

When I am visitng, for example, PC hardware forums that are written in PHP, they are never so slow and they are much more feature rich. Why is that? Because they use simple tool for simple job and not complex tool for simple job. Though, sometimes we have to use complex tools to solve complex job and then we have to invest time in frameworks, but I believe this is rather an exception then the rule.

Re: No ORM: The simplest solution?

Of course I haven't built my own integration library. I have simply used what is already available in Spring. Spring also supports the other frameworks you mention.

But you haven't used a simple solution. You have had to implement a range of things yourself. The simple solution is to use a product that provides these.

Surely it is part of the role of a professional developer to keep up-to-date with such things.

I think this is fundamentally mistaken. The result of this attitude would be that we would end up with endless re-inventions of the same thing, all with their own quirks and all with their own integration and support issues.

Re: No ORM: The simplest solution?

How much traffic do those PC hardware forums get? Do you really think that a re-write of TSS in PHP would cope?

For some tasks, PHP is not a simple solution.

Re: No ORM: The simplest solution?

Spring-JSF and Spring 2.0 have some of that but they are not production ready.

But you haven't used a simple solution. You have had to implement a range of things yourself. The simple solution is to use a product that provides these.

No, I have used simper solution :). Product that you use may have steep learning curve, thick documentation that talk about features your project does not need, has relatively complex setup, frequent maintenance release schedule which you have to be aware of and such things. There is a lot of time and work associated with that. If your project does not need advanced ORM features like unit-of-work, clustered caching or transparent lazy-loading then it is an overhead.

I can happily call Save methods on my DAOs instead of having unit-of-work, I can happily have simple caching of objects (easy to implement if you have domain model), I can happily code lazy-load in my domain model by hand or code generator.

Instead of writing mapping declaration in XML file or whatever is used, I can have two callback methods, one that binds object to the prepared statement and the other that reads it from the ResultSet and that's where my mapping is specified. If I choose to go with code generation, then my mapping is in metadata (or DSL if you want to call it that way). So, business code is not more complex then with full blown ORM, but the system is lightweighter.

Surely it is part of the role of a professional developer to keep up-to-date with such things.

OK, the part of the role of a professional developer is to keep up-to-date with such things, to use them when he/her needs them and not to use when can get job done without them :)

I think this is fundamentally mistaken. The result of this attitude would be that we would end up with endless re-inventions of the same thing, all with their own quirks and all with their own integration and support issues.

Yes, but 'my' quirks and 'my' integration and 'my' support is simpler as 'my' stuff is built for 'my' project needs!
I do not have to provide general solution. 'My' library is designed to meet 'my' project specific needs, no more, no less.

Re: No ORM: The simplest solution?

How much traffic do those PC hardware forums get? Do you really think that a re-write of TSS in PHP would cope?

I can't talk about facts, but they probably have much more traffic then TheServerSide, becuse they have much more threads and much more posts then TheServerSide.

Regarding rewriting TSS, I am not familiar with PHP, have no idea about it. I just saw the forums that are feature ritcher, more performant, user friendlier, with more posts written in PHP.

However, I believe that rewriting TheServerSide to use pure JDBC with MVC and some templating engine (JSP with EL, Velocity or other) will give us much more performant site.

True. That's the point of the original post: Use right tool for the specific job.

Re: No ORM: The simplest solution?

I am using Spring-JSF in production.

I really can't believe that there is more time involved in researching a relatively simple API like JDO, as against writing all this yourself.

Yes, your business code is certainly more complex that with 'full-blown' ORM. I don't have to have any callback methods that read things from result sets.

There is the assumption that your project is 'special' in some way,

that you are doing something unique that has not been done before.

And, of course, your work is wasted when you move to a new project which you assume has different needs....

Re: No ORM: The simplest solution?

TSS uses an ORM (Kodo) which is widely recognised to give high performance, as it produces highly tuned SQL and has excellent cacheing. 'Pure JDBC' won't come close.

Re: No ORM: The simplest solution?

I am just curious, how do you handle session state in your web app? Do you keep JDO Persistent Manager per HTTP session or you disconnect and reconnect object when storing them in HTTP session? Either way you have and overhead that is not necesary if your app does not need unit-of-work. And it does not need it if it has simple transactions involving at most couple of objects.

JDO API may be simple if you look at the JDO JavaDoc. But if you look at the JDO Spec and JDO Vendors Documentation it starts to be quite complex (I am not saying dificult). And I reapeat, someone does not need to reimplement

But you have to specify mappings.

Specifiying mapping in an XML file and writing for example just two callback methods described earlier in a generic DAO calss is merely the same task. Note that only callback methods have to be implemented, as DAO could be generic and inherit for example BaseDAO which could have CRUD methods and uses callback as strategies)

No, I'm not. But, I am doing the same thing in different context. I can reuse patterns and techniques, but reusing freameworks is somewhat harder. Patterns and techniques can be adapted to specific context easier, but frameworks usualy could not. So frameworks end up trying to be as general as they can, increasing overhead and complexity by definition.

No, it's not :). I've learned patterns, I've learned techniques. And projects could have simmilar, yet different needs. Similarity yet difference is much better handled with patterns and techniques then with frameworks.

We have a homegrown data access library written jn Java (I've described the features in previous posts). We recently started a project that uses .NET as this was requested so. We ported our data access library to .NET within a day.

We do not need unit-of-work and clustered caching in this project.

We need lazy load, but that is handled with code generation.

We are happy with our approach. We do not have to search forums when we hit the wall. We can tune our code to meet expected performance requirements.

We are not reinventing the wheel, we are just making the wheel as we have seen how others are doing that. And we made a wheel for our specific vehicle, to meet the best it's needs.

Re: No ORM: The simplest solution?

TSS uses an ORM (Kodo) which is widely recognised to give high performance, as it produces highly tuned SQL and has excellent cacheing. 'Pure JDBC' won't come close.

My understanding is that TSS is written with Kodo and Tapestry as the proof-of-concept and for merketing reasons (for me, these are completely valid reasons).

But a forum web app certainly does not have a need for ORM and component based web UI layer.

Regarding pure JDBC: pure JDBC is by definition more performant then ORM as it is at lower level of abstraction. The other story is how you tune your JDBC and ORM, how you handle caching, etc... If your project is relatively simple, if relationships between your objects follow the same pattern, tuning SQL is easy.

And regarding caching: two level caching that is common in modern ORM is two-edge-sword. Second level is great, improves performance but at the cost of increased memory usage. Hibernate, for example, serializes objects to attribute-value pairs and puts them in maps in second level cache! This imposes quite a memory overhead when compared how much memory would be used if objects are cached as they are (as object instances). Of course, this is not possible in ORM, because it has to be general and cache objects of all types! This is just another point where you have an unnecesary overhead in ORM.

Re: No ORM: The simplest solution?

Maintaining persistence managers for such a length of time is not a good idea. I allow Spring to generate persistence managers for me as I need them. It does this transparently.

Actually, for simple projects, you don't, or at least only minimally. You can let the vendors tools generate schemas or reverse-map them for you.

But there you go again - mentioning things that you have to do that are already provided by a product! Why should I have to generate a BaseDAO, implement the CRUD methods, and the callbacks? I really can't see the point.

Because you are avoiding dependency on third party product, you have better control on your system, better flexibility, better performance.
I've tried ORM framework and tried no ORM framework approach. The productivty is about the same, especialy on bigger projects, where investment in custom infrastructure pays out more. What you get with automatic mapping you loose on hitting the wall, here and there, like we did trying to use ORM with statefull web UI.

Completely other story is when you need ORM features that are hard to build:
- transparent unit-of-work
- transparent lazy-load
- transparent clustered caching

If your project need this, then you must go with ORM that provides this.

And what was the point, when products like NHibernate are already there, and will handle far more use cases that your code?

NHibernate is port of Java Hibernate 2.0 with uncertain future. I wouldn't bet my company's product on something like that!

And now you have to handle support issues with your home-grown framework on two platforms.

I do not have complex support issues. My library is just couple of hundred lines of code.

So what happens when you do? Are you going to write that yourself as well?

Then, but only then, when there is a hard-to-build feature that I need, then I would go to ORM. Sure, why not :) ?

You haven't re-invented the wheel - you have re-invented a whole pile of them:

DAO, callbacks, lazy loading, transaction handling, performance tuning, support for multiple platforms...

Yes, why not? You have to do mappings, write quiries in yet another query language, configure your ORM, performance tune (lazy-load or outer join fetching, whether you will use proxies or not, the size of batch fetcing etc..) These are similar things in the amount of time needed.

and if you ever have to deliver this code to a third party, they will have to learn your framework.

I do no have a framework, just plain stupid and simple data access library based on some well-known patterns! And some simple DAO layer above that uses the library for which I hope is good designed and easily understood. At least by members of my team. That is enough in my context.

Re: No ORM: The simplest solution?

Of course not - I mean, they could write the whole thing in assembler - after all, it is at a lower level, and should be more performant....

You have contradicted yourself here. First you say JDBC is more performant because it is at a lower level, then you talk about cacheing (which isn't anything to do with low level JDBC). ORM is by definition potentially more performant than pure JDBC because it is at higher level of abstraction, which can transparently include cacheing.

And yet again, you are re-inventing the wheel - why should I have to tune the SQL when my ORM products does that for me automatically?

Firstly, ORM is a lot more than just Hibernate. There are a variety of ORMs and they have a variety of cacheing strategies. There are JDO products that have very memory-efficient cacheing, to the point where you can handle single transactions which involve hundreds of thousands or even millions of objects without major memory problems.

(I know, because I have used such products for such transactions). Your argument here about unecessary overhead here is false.

Re: No ORM: The simplest solution?

I do not have complex support issues. My library is just couple of hundred lines of code.

Re: No ORM: The simplest solution?

Of course not - I mean, they could write the whole thing in assembler - after all, it is at a lower level, and should be more performant....

When I said not needed, I meant not needed including productivity, maintainability and time-to-market reasons.
I believe a forum site could be equaly fast build either using ORM or without it.

You have contradicted yourself here. First you say JDBC is more performant because it is at a lower level, then you talk about cacheing (which isn't anything to do with low level JDBC). ORM is by definition potentially more performant than pure JDBC because it is at higher level of abstraction, which can transparently include cacheing.

Transparency of caching has to do nothing with performance.

And yet again, you are re-inventing the wheel - why should I have to tune the SQL when my ORM products does that for me automatically?

Because you can do better job then the ORM tool.

Firstly, ORM is a lot more than just Hibernate. There are a variety of ORMs and they have a variety of cacheing strategies. There are JDO products that have very memory-efficient cacheing, to the point where you can handle single transactions which involve hundreds of thousands or even millions of objects without major memory problems.

I thought that in JDO, when dealing with single transaction, all caching is happening in the first level cache, which is transactional and store objects as instances. When you leave the transaction, first level cache is dropped. Thats, fine. What's not fine is the second level cache which must be concurrent too and handle requests by different Persistent Managers and also must be able to store objects of different types. So some sort of either serialization or deep object cloning is needed here, you can't keep original object instances in second level cache.

(I know, because I have used such products for such transactions). Your argument here about unecessary overhead here is false.

I believe that my argument about overhead in second level cache is actualy true. Of course, sometimes when you need clustered caching, you have no choice but to accept that overhead. The point is, that when you don't, and still use ORM, you are paying unnecesary penalties.

Re: No ORM: The simplest solution?

Maintaining persistence managers for such a length of time is not a good idea. I allow Spring to generate persistence managers for me as I need them. It does this transparently.

Sure it is not. But if you want your UI to be statefull, you need either 'long session' or connection/disconnection of objects. If you are using ORM in a web app and do not need either, then your UI is probably stateless. If your UI is statless, then you probably do not need unit-of-work either. You then end up using ORM only to bind objects to tables (OK, there is caching too). You could do that much simpler way. If you want to use a third-party solution there is iBatis for that.

Actually, for simple projects, you don't, or at least only minimally. You can let the vendors tools generate schemas or reverse-map them for you.

As you said, that can work for simple projects. I would add that could work for prototyping and even not even for simple projects.

But there you go again - mentioning things that you have to do that are already provided by a product! Why should I have to generate a BaseDAO, implement the CRUD methods, and the callbacks? I really can't see the point.

Because you are avoiding dependency on third party product, you have better control on your system, better flexibility, better performance.

What you get with automatic mapping you loose on hitting the wall, here and there, like we did trying to use ORM with statefull web UI.

And what was the point, when products like NHibernate are already there, and will handle far more use cases that your code?

NHibernate is port of Java Hibernate 2.0 with uncertain future. I wouldn't bet my company's product on something like that!

You haven't re-invented the wheel - you have re-invented a whole pile of them:

DAO, callbacks, lazy loading, transaction handling, performance tuning, support for multiple platforms...

Yes, why not? You have to do mappings, write quiries in yet another query language, configure your ORM, performance tune (lazy-load or outer join fetching, whether you will use proxies or not, the size of batch fetcing etc..) These are similar things in the amount of time needed.

and if you ever have to deliver this code to a third party, they will have to learn your framework.

I do no have a framework, just plain stupid and simple data access library based on some well-known patterns! And some simple DAO layer above that uses the library for which I hope is good designed and easily understood. At least by members of my team. That is enough in my context.

Re: No ORM: The simplest solution?

I strongly disagree. The point of ORM (well, why I use it) is to save time in development. It abstracts away details that would otherwise have to be hand written. This saves time.

No, but cacheing does, and you don't get that with raw JDBC alone. Hence my argument that raw JDBC is not as performant as ORM. Once you start having to deal with cacheing you aren't at a low level anymore, and you might as well use a product that handles this for you.

But why bother hand-tuning a potentially substantial number of SQL statements when an ORM will do a reasonable job? If there are some statements that are critical, any decent ORM will allow you to express those in SQL and hand tune them, but to keep all queries in SQL in case they need tuning is a form of premature optimisation. If you are that interested in performance, and tuning, why bother with Java at all? The VM is an abstraction after all. We use Java because abstractions save us from having to deal with an unecessary level of details. ORM, in my view, does the same thing.

So don't use a second level cache. This is trivial to turn off (a single setting).

As with so many features of ORM, if you don't need to use a feature, you simply turn it off. If you don't want to have a second level cache, because you consider it has overheads, just don't use it. No overheads, no penalties.

Re: No ORM: The simplest solution?

ORM is being used very successfully with stateful web UIs. TSS is an example.

There are approaches like Seam that show that ORM and stateful web UIs are a good fit.

Yes, but at what stage does this stop? JDBC drivers have different qualities, and are third-party - do we bypass them and write our own socket-level interfaces to the database?

Re: No ORM: The simplest solution?

Yes, but you need Seam for that. Yet another framework in the framework soup. Yet another dependency. Yet another framework to learn. Yet another rope to hang myself. Seam is also fresh framework I would think twice before using it in a production app.

Please do not get me wrong. Seam is great! The idea of having type of state like conversation and process is essencial in building rich UIs and having it on statless web is tremendous benfit. But Seam's area of potential usage is rather limited to building web apps with rich statefull UIs. That's not bad, actually it is good, as frameworks should focus on one particual thing and abstract it right. We, developers, have to conclude when and how to use them.

Re: No ORM: The simplest solution?

I am talking about this just to show that sometimes problems that we think are hard are just easy to solve.

We have to distinguish these things, and use third-party tools when their usage pays off. That is the point of the original post. Sometimes (maybe not in your case), ORM is just an overhead.

I agree with that. But with ORM it is much more likely that you will 'hit the performance wall' as ORM is on higher level of abstraction then JDBC and does not provide enough hooks to tune the system when needed.

for other people to use limiting their flexibility and control over resulting system with higher level black boxes with unsufficient hooks to tune them at lower levels.

Also, frameworks are now often tested and good quality from framework user's perspecive. But when you try to hook down to implment some integration or some lower lever module for the framework, you encounter inconsistences, bugs, not so obvious and undocumented assumptions etc... That's normal, as primary audience for frameworks are framework users. But, now, as requiremets and expectations from us software developers are higher and higher, we cannot be satisfied with such black boxes. We need more power, more flexibility, more control.

Turning off second level cache bogs the performance of ORM so down that I would never use ORM without second level cache. If ORM cannot cache my objects between transactions, it's useless for me. Second level cache is so vital for ORM performance. Without it, every update and delete is accompanied with a select to check for optimistic locking.
Every lazy-load single object relationship (if not configured to use outer-join fetching) has to hit the DB for every transaction. Turning second level cache in my opinion is not acceptable.

As with so many features of ORM, if you don't need to use a feature, you simply turn it off. If you don't want to have a second level cache, because you consider it has overheads, just don't use it. No overheads, no penalties.

But I desperately want my objects to be cached between transactions :(

Re: No ORM: The simplest solution?

ORM is being used very successfully with stateful web UIs. TSS is an example.

Please do not quote TSS site as a good example of ORM usage. There are a lot of better forum software out there that do not use ORM. Post preview has been implemented only recently. No post editing still. These are basic forum features.

There are approaches like Seam that show that ORM and stateful web UIs are a good fit.

Yes, but you need Seam for that. Yet another framework in the framework soup. Yet another dependency. Yet another framework to learn. Yet another rope to hang myself. Seam is also fresh framework I would think twice before using it in a production app.

Yes, but at what stage does this stop? JDBC drivers have different qualities, and are third-party - do we bypass them and write our own socket-level interfaces to the database?

JDBC is fine. Actually it is great, it is one of the most well designed and successfull Java APIs (along with Servlet API)! It is at low level enough, yet easy to use. Though you need a thin layer above to reduce the coding. But if you ever have to deal with ADO.NET for example, you will learn to appreciate JDBC even more! (ADO.NET is realy, realy very bad for anything but the grid on the form ;) )

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

This was interesting discussion indeed, maybe we should be back at work :)

One question though, are you working for a company that is software vendor or in an IT department of a company from other industry? (My company is software vendor, but I also worked in an IT department) Maybe that is the cause why we cannot agree?

Because, when we are selling a product, dependency on third-party is not at all wellcome (additional maintenance (in the eyes of the customer), possible costs (Customer: 'Oh, I have to pay for this license too!')).

For an internal business app produced by company's IT depeartnemt, reliance on standardised third-party tools is, of course, very wellcome.

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

I believe the message that began this blog describes the correct approach for all applications. Though progress has been made, ORM continues to struggle in areas of complexity and processing overhead, just as it did 10 years ago. ORM's recent surge has nothing to do with the progress of ORM, but the progress of computing power. However, computing power still comes at a cost, does the benefit of ORM out way it?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

At the moment my coding is mainly for internal use. However, I see see good potential for use of third party frameworks even for products. This gives the customer choice. Not only would ORM allow my customer the choice of persistence product, because I ensure I conform to JSRs, I can potentially give them the choice of ORM as well.

I can say 'for general use, you may wish to use this open source product.

Re: No ORM: The simplest solution?

I mean I completely agree with what Scot Bellamy has said.

(TSS still does not allow me to edit my posts :(, but it should if no one has replied. As you probably know, we non-native English speakers make a lot of mistakes while posting :). Post preview is good feature, but sometimes fingers are faster :) )

Re: No ORM: The simplest solution?

Well, customers usualy do not care about ORMs, JRSs and such stuff, they usualy care about working software that solves their business problem, the lower cost, the better.

Sorry, just kidding :).

It is interesting that the higher profile and education is the customer, he/she usually values open source more.

For example, we had a project for ECB (European Central Bank) and they valued Open Source a lot, we used Apache, Tomcat, Open LDAP, Perl and such stuff. I mean, they got the money, but they like open source, they wanted Java, used Oracle and they will to pay less for the software.

The other customers of our company are from bookmaking (betting) industry (mostly from UK but also from other parts of the world). They have less money, prefer Microsoft solutions and are mostly agnostic to open source.

But this is a problem for the sales stuff, not us anyway.

Re: No ORM: The simplest solution?

For example, as I am writting this, TheServerSide site is sooo slooow.

Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

I have already used some ORM framework in the past but face some optimization problems with "lazy" loading. I have noticed that the framework were loading most of the data in the cache. When you have a lot of data, it sucks!
The lazy loading was not effective with the compound key and the mapping used. I have found a solution in a forum but we would have to change the mapping. Not a good idea at the end of a project.
I will not tell which framework I used in order to avoid a thread "my framework is better that X and won't meet this problem".

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

I am afraid I disagree with this premise. Yet again, I would like to emphasise that ORM != Hibernate, or rather it is considerably more than Hibernate. Having used both JDBC and ORM for substantial projects, my conclusion is that ORM is a far better way to manage complexity, and I found that for a large data model, my reduction in data access code was considerably more than 25% or 50%.

You can't judge the benefits or otherwise of Java ORM in general by investigating one product. Hibernate has its use cases, and other ORM products have theirs. Many ORM products have an excellent reputations for scalability and performance, and I know personally of projects that have required such performance and ORM has delivered, so I just can't agree that ORM is only growing in use because of increased computing power. Many companies have invested considerable resources to produce high-performance products. In all honesty, I don't believe that if you use a good ORM product, there is performance overhead that has to be set against other benefits.

Re: No ORM: The simplest solution?

Is it a coincidence that a framework-heavy J2EE application that I recently looked at was serving 2,000,000,000 page impressions per week?

Please don't take this as an argument for or against ORM.

I personally suspect that on small projects, ORM could be more work than it's worth (and for that matter, so could Java ;-), but every project and every team is different.

Re: No ORM: The simplest solution?

One thing I noticed with my Spring JDBC programming is that I was writing a lot of repetitive code to insert, update, delete and get (by primary key) objects from individual tables. So I factored that code up to a base class and made my own little framework augmentation. It might have been nice if the Spring people did something like this, but on the other hand, it would probably have been a lot more complicated than the solution I wrote to meet my direct needs.

Re: No ORM: The simplest solution?

When you say you have experienced code reduction more than 25% or 50%, do you have measureable examples, or is it based on estimates? What I found is that those who are proponents of ORM often tend to quote the ORM advertising as to the savings they get. If you are really experiencing more than 50% savings, I would suggest the original JDBC code was very badly written to begin with (just as it was in our case).

I chose Hybernate as an example because it requires no licensing cost, and because it is arguably the most popular ORM tool. If the cost/benefit analysis doesn't work favorably for Hybernate, then it certainly won't work favorably for expensive commercial products.

Re: No ORM: The simplest solution?

For example, as I am writting this, TheServerSide site is sooo slooow.

Is it a coincidence that it is using a web app 'framework trio' (container + ORM + component based web framework)?

Yes. It is a custom implementation, developed over half a dozen years, used by only one company, and it has some resulting issues.

Is it a coincidence that a framework-heavy J2EE application that I recently looked at was serving 2,000,000,000 page impressions per week?

Please don't take this as an argument for or against ORM. I personally suspect that on small projects, ORM could be more work than it's worth (and for that matter, so could Java ;-), but every project and every team is different. On all the projects that I see, I would always use ORM, but YMMV.

Peace,

Cameron Purdy
Tangosol Coherence: The Java Data Grid

Re: No ORM: The simplest solution?

I used Hibernate last year for a couple of projects. It's a great product, but I often found it too cumbersome for simple projects.

I tried Spring JDBC after that, and I like it a lot. Spring got it right by minimizing checked exceptions and eliminating micro-management of objects like the Statement and the ResultSet that distract you too much from what you're trying to accomplish.

My experience with database programming is that most of the unique effort in a new project (as opposed to time or lines of code) goes into writing queries. The ORM tools don't really automate that. They give you different ways to write queries, but the amount of effort is the same whether you're writing SQL or HQL or building a query with Java objects.

One thing I noticed with my Spring JDBC programming is that I was writing a lot of repetitive code to insert, update, delete and get (by primary key) objects from individual tables. So I factored that code up to a base class and made my own little framework augmentation. It might have been nice if the Spring people did something like this, but on the other hand, it would probably have been a lot more complicated than the solution I wrote to meet my direct needs.

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

That base class your wrote I got almost for free using the SpringHibernateTemplate.

Re: No ORM: The simplest solution?

I do. Even on apps as small as 2 pages. The value of consistency that I get by using Spring/Hibernate even on very small apps is worth more than the slight speed increase I would get by say embedding SQL in an jsp page.

I've personally found that ORM makes my job easier and allows for more maintainable code.

In fact, one of our guys, a non-developer was commenting on how much different our company is than it was a few years ago. How much more reliable our releases are and how much faster our turn around is.

This occurred as our reliance on straight JDBC, internal factories(if people used them!), and jsp was replaced by Spring, Struts, and Hibernate.

And his wasn't the only comment. The people who go out test, validate, or train on our apps have commented about how much better our software is.

Re: No ORM: The simplest solution?

I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?

Re: No ORM: The simplest solution?

That base class your wrote I got almost for free using the SpringHibernateTemplate.

I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?

Re: No ORM: The simplest solution?

I do. Even on apps as small as 2 pages. The value of consistency that I get by using Spring/Hibernate even on very small apps is worth more than the slight speed increase I would get by say embedding SQL in an jsp page.

I've personally found that ORM makes my job easier and allows for more maintainable code.

In fact, one of our guys, a non-developer was commenting on how much different our company is than it was a few years ago. How much more reliable our releases are and how much faster our turn around is.

This occurred as our reliance on straight JDBC, internal factories(if people used them!), and jsp was replaced by Spring, Struts, and Hibernate.

And his wasn't the only comment. The people who go out test, validate, or train on our apps have commented about how much better our software is.

You can sure have consistency and 'well-designness' without Spring/Hibernate, but if they do help your organization to achieve that then your choice in your case was right.

Re: No ORM: The simplest solution?

That base class your wrote I got almost for free using the SpringHibernateTemplate.

I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?

Right. If Frank supplied
CRUD operations
Declarative Transaction support
Caching
Lazy instantiation
IOC
AOP
Dynamic SQL general
Automatic loading of object graphs

all in 50-100 lines of code, he should release a framework of his own!

He'll put everyone else out of business.

Re: No ORM: The simplest solution?

This is actual experience. It was explicit JDBC. It may have been badly written because it was so explicit, but I then spend some time devising my own framework, my own method of handling DAOs. And I then realised I was effectively writing my own ORM, which was a waste of time.

I am pretty surprised that posters here are able to put so many anti-ORM case studies and so many anti-ORM opinions relatively unchallenged (except by me). I know from personal experience, and dealings with others, that ORM can be simple and very productive. ORM obviously needs better marketing, as a combination of experiences with poor quality ORMs (or certain ORMs used in situations they weren't designed for) with (occasionally) a Not Invented Here attitude obviously seems to be inhibiting its acceptance by some developers.

Re: No ORM: The simplest solution?

This is exactly the point I'm trying to make. Much of the touted benefit of ORM is saving programming time. The fact is that much of the savings is comparing bad programming to good implementation with ORM. Not really a good comparison. It doesn't take building an elaborate framework for efficient data access. In our case, we utilized an open-source connection pool and built a BaseDAO class that handled much of the mundane pieces. The remainder was simply good programming, factoring out common code, which is a practice not just for data access, but for coding in general.

I am pretty surprised that posters here are able to put so many anti-ORM case studies and so many anti-ORM opinions relatively unchallenged (except by me). I know from personal experience, and dealings with others, that ORM can be simple and very productive. ORM obviously needs better marketing, as a combination of experiences with poor quality ORMs (or certain ORMs used in situations they weren't designed for) with (occasionally) a Not Invented Here attitude obviously seems to be inhibiting its acceptance by some developers.

I think what you are finding on this forum are many

who see through the hype, and/or have truly evaluated the TCO of ORM.

What I find surprising is the number of people I have encountered over the years who have failed to look at the total cost of ORM, but simply get caught up in the hype. If you are not measuring the additional cost required to achieve scalability and performance using ORM, then you really have no idea how much it costs, and are simply buying in on the hype of saving lines of code, and even the code savings are hyped based on improper comparison.

Re: No ORM: The simplest solution?

I agree with that. But with ORM it is much more likely that you will 'hit the performance wall' as ORM is on higher level of abstraction then JDBC and does not provide enough hooks to tune the system when needed. That is the problem with current state of the development. People are building higher levels of abstractions using 3GL languages (frameworks, containers...) for other people to use limiting their flexibility and control over resulting system with higher level black boxes with unsufficient hooks to tune them at lower levels.

Also, frameworks are now often tested and good quality from framework user's perspecive. But when you try to hook down to implment some integration or some lower lever module for the framework, you encounter inconsistences, bugs, not so obvious and undocumented assumptions etc...

That's normal, as primary audience for frameworks are framework users. But, now, as requiremets and expectations from us software developers are higher and higher, we cannot be satisfied with such black boxes. We need more power, more flexibility, more control.

That is why I said frameworks are not so good at abstracting things.

Currently I am very excited about Domain Specific Languages and code generation. This solves the above problem. DSLs do almost everything that frameworks do. They are risng the level of abstraction (even more then frameworks), provide more robust and coherent systems, lower maintenance, improve flexibility and all that without sacrifising control over the very bits at the low level of the system. You can tune generated code from DSL whatever you find feasible.

Re: No ORM: The simplest solution?

But see, I don't WANT to write all the code needed to do what Spring and Hibernate does. I also don't want to write a database or operating system.

Re: No ORM: The simplest solution?

That base class your wrote I got almost for free using the SpringHibernateTemplate.

I am sure that the base class Frank wrote didn't have more then 50-100 lines of code! So he probably didn't payed big cost neither :)

I am wondering when we will be starting to use a framework that does System.out.println in a fancy way :( ?

Right. If Frank supplied
CRUD operations
Declarative Transaction support
Caching
Lazy instantiation
IOC
AOP
Dynamic SQL general
Automatic loading of object graphs

all in 50-100 lines of code, he should release a framework of his own!

He'll put everyone else out of business.

It wasn't Frank.

It was from here:
http://www.theserverside.com/blogs/thread.tss?thread_id=41715#215595

But the point is the same.

Java Database Framework (ORM Replacement) in 80 Lines of Code

Re: No ORM: The simplest solution?

My point is that apart from the very smallest of projects, even good programming is largely a waste of time compared with ORM. Why bother even writing a BaseDAO?

Have you? So far you have only reported experiments with Hibernate.

Hibernate, good thought it is, does not reflect the full range and capabilities of ORM on Java.

If you have evaluated other products, and can report on this, I would be interested.

Of course I am measuring the additional cost required to achieve scalability and performance!

Re: No ORM: The simplest solution?

I have been looking at ORM products for more 10 years. I've investigated products such as TopLink, prior to the days of Java.

The problems are essentially the same. What I have experienced, rega1rdless of product, is increased memory usage by 20%-30%, increased CPU usage by 20%-30%, and increased database CPU usage by 10%-20%, all while seeing performance decrease by 10%-50%.

The cost of ORM has decreased over the past 10 years, not because ORM has improved, but because memory and CPU costs have decreased.

Taking the low end of the ranges mentioned above will increase costs for high volume applications in the 10s of thousands of dollars (and likely higher when considering CPU-based licensing costs of applications running on servers which require additional CPU). These are the costs that proponents of ORM never consider (that I've seen).

Of course I am measuring the additional cost required to achieve scalability and performance!

I would like to see those numbers. How much is it costing for the addtional memory, CPU, CPU-based licenses of products, commercial ORM product, and training on ORM compared with how much you are saving?

Re: No ORM: The simplest solution?

and training on ORM

Re: No ORM: The simplest solution?

What other products? So far you have mentioned Hibernate, and pre-Java TopLink.

Major organsations must be somewhat perverse in choosing to use products that are so disadvantageous. You might wish to suggest to Ebay that they abandon their use of ORM for their site.... or perhaps they might know what they are doing and you are mistaken?

That is false. I have also been using ORM on and off for over a decade (I evaluated Gemstone with Smalltalk). The quality of ORM has improved beyond recognition. There was nothing like the flexible and powerful query languages we have today, for example.

That is because costs aren't on that scale. I am using Kodo, and you are wildly out (in the high end) in terms of your pricing. CPU-based licensing costs are ancient history in my experience.

Of course I don't need additional memory or CPU - that is just FUD - in fact ORM is saving me memory because of its good cache management - I am running transactions with hundreds of thousands of objects on a laptop PC.

As for costs, I am using Kodo JDO. You can find the price from BEA. Your estimates are wildly exaggerated.
I have no idea where you are getting your ideas of what ORM requires, but it isn't anything I recognise.

Re: No ORM: The simplest solution?

and, of course, I don't need training. I am using ORM that conforms to a JSR standard. I and my co-developers have been using JDO for years. Switching to another JDO product requires minimal effort.

Of course, when someone else comes across your home-grown solution, such years of experience is wasted - then they will need training to use your solution.

Re: No ORM: The simplest solution?

and, of course, I don't need training. I am using ORM that conforms to a JSR standard. I and my co-developers have been using JDO for years. Switching to another JDO product requires minimal effort.

The training I'm referring to is the training to use ORM, not a specific tool. The ramp-up time I've seen for seasoned developers to begin using an ORM tool (pick one) is significant. It requires formal training as well as time to come up to speed on live projects. For example, in order to be flexible tools such as Hybernate have their own query language. There are configurations parameters that must be understood, etc. Even if already familiar with ORM, there is some training involved, and much more if not experienced on ORM.

Of course, when someone else comes across your home-grown solution, such years of experience is wasted - then they will need training to use your solution.

Really? You would have trouble understanding the concepts of a base class? Sure, it may take a few minutes, a couple of hours, maybe 1/2 day for a very slow person.

But nothing compared with learning a full-blown ORM tool. I've experienced and observed both, and the learning time required for "home-grown" is essentially nothing compared with that required of an ORM tool. Again, it's another cost that ORM evangelists fail to recognize.

Re: No ORM: The simplest solution?

What other products? So far you have mentioned Hibernate, and pre-Java TopLink.

At the time I used Toplink, it was the leader in the market for ORM. Now Hybernate is arguably the leader in the Java space. I don't see where wasting time on non-market leaders is going to change anything.

Major organsations must be somewhat perverse in choosing to use products that are so disadvantageous. You might wish to suggest to Ebay that they abandon their use of ORM for their site.... or perhaps they might know what they are doing and you are mistaken?

Plain and simple, they are not evaluating the total cost of the solution. As such, they have no idea how much the solution is costing them, the very point I'm trying to make. And as you are dancing around the topic, I suggest you have no idea how much the solution is costing you either. You still haven't produced the numbers of you costs vs. your savings. I gave you numbers based on my experience over the years, both with a non-java product early in ORM and with a Java product that is a current product.

That is false. I have also been using ORM on and off for over a decade (I evaluated Gemstone with Smalltalk). The quality of ORM has improved beyond recognition. There was nothing like the flexible and powerful query languages we have today, for example.

Sorry, I didn't word this very well. I agree that ORM has made signficant improvements over the years. What I meant to say is that the reduced cost of ORM has little to do with those improvements as compared with the lower cost of CPU and memory.

That is because costs aren't on that scale. I am using Kodo, and you are wildly out (in the high end) in terms of your pricing. CPU-based licensing costs are ancient history in my experience.

I provided numbers based on my experience. They are absolutley correct. The fact that you can't provide numbers tells me that you really haven't measured it objectively.

Of course I don't need additional memory or CPU - that is just FUD - in fact ORM is saving me memory because of its good cache management - I am running transactions with hundreds of thousands of objects on a laptop PC.

As for costs, I am using Kodo JDO. You can find the price from BEA. Your estimates are wildly exaggerated.
I have no idea where you are getting your ideas of what ORM requires, but it isn't anything I recognise.

I hope you can some day look at this objectively, and not ignore the facts.

Caching has been around much longer than ORM. It just so happens that ORM tools incorporate it. You don't have to use an ORM tool for caching. That's a completely separate topic.

Re: No ORM: The simplest solution?

Because market leaders aren't always the market leaders because of quality in all areas, and to make any claims based on a sample of two products in a market with so many is statistically of little or no use.

By that argument you would believe Windows was the ultimate OS for all situations.....

Sorry, but given the choice to believe your figures (especially based on what you admit is a limited experience of the area), or my own experience of a wider range of ORMs and the major success of projects based on ORM, I go for the latter.

I can't give you pricing because this has probably changed since BEA has taken over Solarmetric. But I can give you the fact that your price estimates are way, way too high.

I have certainly measured things objectively. I am working in areas which involve relatively high volume batch processing - loading and transforming hundreds of thousands or even millions of records into databases. Years ago I had been doing this by JDBC+SQL for portability (although it would have been even faster with proprietary loading tools). Benchmarking showed that the database inserts and processing were the major factor in the time taken for the processes to run. I switched to a commercial ORM (at the time it was called JDOGenie). What was the time difference? None. No need for extra CPU, no need for extra memory, even for processing very large transactions. I don't make claims without evidence, which is why I don't accept that your figures for increased CPU or memory are in any way applicable to ORM in general, and certainly not for the products I have used.

Re: No ORM: The simplest solution?

Because market leaders aren't always the market leaders because of quality in all areas, and to make any claims based on a sample of two products in a market with so many is statistically of little or no use.

By that argument you would believe Windows was the ultimate OS for all situations.....

Now you are reaching. The fundamentals of ORM are essentially the same, no matter what product you might be using. Though the overhead of object creation and additional CPU may vary from product to product, it does exist in every ORM product.

Comparing it to Windows is REALLY reaching. Windows is the undisputed leader in desktop OS. It isn't the leader in hosting high-end computing environemnts. Enough said.

I choose to measure TCO, not buy into product hype. The numbers I have quoted are from experience, not made up. As long as you refuse to measure TCO, we really have nothing else to discuss here.

I can't give you pricing because this has probably changed since BEA has taken over Solarmetric. But I can give you the fact that your price estimates are way, way too high.

Let me give an example based on the numbers I mentioned before. If I have a high-volume, database intensive application built on Weblogic Server, which requires 4 CPU's using straight JDBC, it will require 1 additional CPU if using an ORM tool, such as Hybernate. Weblogic Server lists at more than $20,000/CPU + the hardware cost of the CPU + the labor cost for adding the CPU (and typically CPU's are added in pairs). If you look at Oracle licenses due to additional DB resource usage, the cost can go even higher.

I have certainly measured things objectively. I am working in areas which involve relatively high volume batch processing - loading and transforming hundreds of thousands or even millions of records into databases. Years ago I had been doing this by JDBC+SQL for portability (although it would have been even faster with proprietary loading tools). Benchmarking showed that the database inserts and processing were the major factor in the time taken for the processes to run. I switched to a commercial ORM (at the time it was called JDOGenie). What was the time difference? None. No need for extra CPU, no need for extra memory, even for processing very large transactions. I don't make claims without evidence, which is why I don't accept that your figures for increased CPU or memory are in any way applicable to ORM in general, and certainly not for the products I have used.

The results you mention here tell me two things; 1) The original JDBC code was poor to begin with or the SQL was very simple and 2) you didn't measure the memory and CPU utilization of the application. ORM has overhead and there is a cost to using it.

Architects and developers often choose to use it without measuring the real cost.

I choose to understand the full cost and benefit of a solution before making a choice.

As I have done over the past decade+, I will continue looking at and evaluating data access methods and tools.

But value is measured in total benefit vs. total cost. Ignoring some of the costs will do my company a great disservice.

As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.

Re: No ORM: The simplest solution?

Comparing it to Windows was not reaching - just illustrating the pointlessness of taking just the market leader as representative. And regarding the fundamentals of ORM, you are wrong, as you would know if you had used more products. There can be different approaches like proxying or enhancement, and different types of object lifecycle. These can affect memory use and performance, especially when dealing with large numbers of objects. Different approaches require different amounts of object creation, for example.

Why should use of an ORM require an additional CPU? If you research this, you will find that for years developers have been using ORM to improve the performance of app servers by substituting the ORM for EJB persistence. This is has been a common use case for JDO. Far from requiring additional CPU resources, it reduces this requirement.

(1) It was not poor, but it was relatively simple, but then who cares? ORM saved me writing the SQL, and gave me the same performance.

(2) I certainly did measure the memory and CPU utilisation, and I told you I did. The database was the primary user of CPU in both cases. I was extremely concerned about memory, due to the size of the transaction. The memory use was not excessive - no extra CPU or memory was required as you implied would be the case.

By implication, others don't do this, which is nonsense.

Costs need to be investigation by looking at a wide range of solutions; you can't just pick one product and assume you are providing anything like a balanced judgement.

As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.

No, it really isn't. This might only be the case if you pick one product that might not be the best suited to your requirements, and assume if that fails that using your own code is the answer. There can often be better ways.

Re: No ORM: The simplest solution?

This discussion probably has long since lost its fruitfullness. I don't think we are going to agree. I contend that you are still not looking at total cost, and I believe that has clouded your judgement.

The reason I said it is stretching is because I would apply the same reasoning to selecting an OS for my organization as I have been mentioning for ORM. I would evaluate total cost. For example, Linux is becoming a viable option for desktop computing, and has benefits of better security and less vulnerability to viruses than Windows. However, the costs of using it (even if it were free) outweigh those benefits.

I'm fully aware of optimization techniques in coding. The fact remains that, no matter how optimized, there is still overhead in an ORM tool. You can deny that until you are blue in the face, and it will still be true.

Providing the underlying code infrastructure to accomodate mapping of user defined objects to user defined database schemas will always require more object creation than manually mapping for a specific need.

If you have found something that somehow magically does this with no extra overhead, then please show me the JDBC and respective ORM examples to prove it.

Why should use of an ORM require an additional CPU? If you research this, you will find that for years developers have been using ORM to improve the performance of app servers by substituting the ORM for EJB persistence. This is has been a common use case for JDO. Far from requiring additional CPU resources, it reduces this requirement.

That's assuming you use CMP. I don't. My custom JDBC will always be more efficient, requiring less memory and CPU than ORM.

(1) It was not poor, but it was relatively simple, but then who cares? ORM saved me writing the SQL, and gave me the same performance.

My experience has been that simple is what ORM does best. The more complex the data access transactions, the more ORM costs (in memory and CPU). So it saved you from writing simple SQL. What did it cost? There is always a cost! Denying the cost aspect doesn't make it untrue, just simply implies you don't understand it.

(2) I certainly did measure the memory and CPU utilisation, and I told you I did. The database was the primary user of CPU in both cases. I was extremely concerned about memory, due to the size of the transaction. The memory use was not excessive - no extra CPU or memory was required as you implied would be the case.

Actually, you didn't say you measured it, you simply stated you didn't require more. That doesn't necessarily mean you measured it.It might mean it fit within the same memory configuration the original application fit within, which likely was more than the application acutally needed in order to accomodate growth.

The fact is, if you actually measure it, you will see extra objects created, utilizing more memory, and requiring more CPU for garbage collection.

If you truly have found a tool that has absolutely no overhead, send some examples so I can try it out myself. Maybe you have stumbled on to something!

Costs need to be investigation by looking at a wide range of solutions; you can't just pick one product and assume you are providing anything like a balanced judgement.

As for now, the most cost effective means of data access was mentioned in the original post on this blog, keeping it simple and lean with your own code, using common sense code factoring techniques.

No, it really isn't. This might only be the case if you pick one product that might not be the best suited to your requirements, and assume if that fails that using your own code is the answer. There can often be better ways.

There are many factors that go into making a market leader. Though market share is a significant factor, there are many others. Most people don't have time to evaluate every product that is released. If there is a truly inovative and beneficial product, it will make a lot of noise and evolve itself as a market leader (maybe the product you are using will do so). You don't have to look at many products to understand a technology and its benefit to an organization. You need to have a good understanding of what the technology is attempting to accomplish, the experience to understand the implications, and for the few products that are truly market leaders, a solid evaluation plan that measures the complete cost and benefit.

Re: No ORM: The simplest solution?

Yes, you really do have to look at many products to understand a technology and its benefits, certain in this case. There are major differences in technology, targetted use cases and features of Java ORMs; even within those that target the same JSR, be it JDO or JPA

What are we paid for ?

Re: No ORM: The simplest solution?

An ORM layer does bring some extra overhead in terms of performance and memory-usage.

Re: No ORM: The simplest solution?

An ORM layer does bring some extra overhead in terms of performance and memory-usage.

But that is one of the issues I disagree with. I don't believe it does. I think this is often a myth based on the false idea that having additional layers of abstraction necessarily imposes performance or resource penalties.

Re: No ORM: The simplest solution?

An ORM layer does bring some extra overhead in terms of performance and memory-usage.

But that is one of the issues I disagree with. I don't believe it does. I think this is often a myth based on the false idea that having additional layers of abstraction necessarily imposes performance or resource penalties.

Steve, I think we are more-or-less on the same page in terms of advantages of ORM. In some cases, the performance of some poorly designed JDBC applications can even improve significantly with optimal use of a good ORM framework. Most of my comments were meant as a response to other posts citing drawbacks of ORM solutions.

-- Damodar

Re: No ORM: The simplest solution?

much better in the caves

Re: No ORM: The simplest solution?

Steve,

Speaking as someone who has no axe to grind either for or against ORM, I would like to say that you do your viewpoint no favours by repeatedly stating that ORM products impose no overhead.

When people point out that an ORM must impose some extra overhead, it would behoove you to accept that as a simple and self-evident truth, because that's what it is. For sure, the overhead might be minimal; it might even be insignificant: but it definitely exists. The far more important point is that the use of the ORM might well mean that the overhead is more than compensated for.

To repeatedly argue that an extra layer of (quite complex) software imposes no overhead makes you sound like a zealot, and it detracts from the point you're trying to make.

Well, it does to me, anyway - I hate to see intelligent people arguing like politicians...

Re: No ORM: The simplest solution?

What do you think of this bare-bones approach? Are you bypassing or embracing ORM?

Re: No ORM: The simplest solution?

A failure might not be due to the ORM tool. It might be the programmer once familiar with jdbc now has to become familiar with hibernate. One has to learn the specific, sometimes subtle and very often 'well' documented peculiarites of such a tool. And beliefe me, such peculiarities will show up. And they will show up, if you don't need them...So you definitly have to learn a ORM tool in depth with all it's consequences.

Extend scope and think of your project that applies let's say two frameworks which apply byte code instrumentation and spice up the soup with some annotation stuff. Who can tell what kind of soup this will be? The result is a intransparent mess by the power of x.

I would fell to be at the Tools mercy. And I would not like the kind of feeling. And I would not like to be at the tool vendors mercy...jboss ;-)

And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem. What about a distributed transaction, multi level cache ORM solution for a portlet that reads just some gibberish from the db to display it. What is your solution?

Re: No ORM: The simplest solution?

And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem.

Re: No ORM: The simplest solution?

And then what about "kiss" - keep it stupid simple, meaning: use the solution that best fits your problem.

We call it Keep It Simple and Straightforward. You can avoid most of the problems you mentioned by using JDX OR-Mapper. There is no byte code processing and no new query language to learn.

Re: No ORM: The simplest solution?

Re: No ORM: The simplest solution?

Then I will get blue in the face, as I deny it. I deny that there is any overhead in ORM if you are going to do anything like that in your own framework - populating objects from a database, persisting them, perhaps cacheing a few.

Providing the underlying code infrastructure to accomodate mapping of user defined objects to user defined database schemas will always require more object creation than manually mapping for a specific need.

No, it won't. Either you map the objects, or the ORM does. The differences for a specific need are minimal, unless you are using perhaps just one or two objects. For a sufficient size, you will be simply replicating what the ORM does.

Actually, you didn't say you measured it, you simply stated you didn't require more. That doesn't necessarily mean you measured it.It might mean it fit within the same memory configuration the original application fit within, which likely was more than the application acutally needed in order to accomodate growth.

But a while back you were talking about the additional cost of memory, CPUs and CPU licences. This clearly demonstrates that that was incorrect.

Re: No ORM: The simplest solution?

I am tempted to put actual figures on a blog article to justify this; the problem is that it may be futile, as some may claim that any overhead is a justification to write home-grown solutions, and one can always tune the ORM little more, or optimise the SQL a little more to tweak things one way or the other. Still, it might be an interesting exercise.

I guess I do see this as political, which is perhaps why I have been pushing this point.... my view is that the time when we write our own frameworks should be long past. And what has been suggested be written in-house in some posts here certainly justifies the use of the word 'framework'!

Maybe the problem is the O

Re: No ORM: The simplest solution?

My point hasn't just been overhead, it has been TCO. I have yet to see a post in favor of ORM tools describe the benefit in terms of TCO.

Re: Maybe the problem is the O

but why do we assume that object model good/data model bad?

So the assumption is we must have O, have to tolerate R therefore need M.

No ORM: VORUTA

ORM's are a overkill