|
Sponsored Links
Resources
Enterprise Java
Research Library
Get Java white papers, product information, case studies and webcasts
|
News
News
News
|
Messages: 74
Messages: 74
Messages: 74
Printer friendly
Printer friendly
Printer friendly
Post reply
Post reply
Post reply
XML
XML
XML
|
 |
Object Persistence with the db4o Object Database
There is always a lot of talk about the data tier, and especially ORM solutions. This article takes a look at simple object persistence with an embedded object database, and how in certain situations it could be a good choice.
Intro
Many Java applications need to deal with persistent data. In most cases, this means interfacing with a relational database, possibly a legacy database or an industry standard Database Management System (DBMS). The JDBC API and drivers for most database systems provide a standard way of using SQL to execute database queries. However, the interface is complicated by the "impedance mismatch" between the domain object model of the application and the relational model of the database. The object model is based on software engineering principles and models the objects in the problem domain, while the relational model is based on mathematical principles and organizes data for efficient storage and retrieval. Neither model is particularly better than the other, but the problem is that they are different and do not always sit together comfortably in the same application.
Some solutions to this problem, such as Hibernate and Java Data Objects, are designed to provide the developer with transparent persistence: the application deals with persistent objects using an object-oriented API without the need for SQL code to be embedded in the Java code. Container Managed Persistence (CMP) does a similar job for EJB containers, but it is not a general persistence facility for the Java platform. In any of these solutions, the objects are mapped to tables in a Relational DBMS (RDBMS) by the underlying framework, which generates the SQL required to store and retrieve object attributes. The more complex the object model, the more difficult the mapping. Descriptors, usually XML files, need to be created to define the mappings. Inheritance and many-to-many relationships, in particular, add complexity as these relationships cannot be directly represented in the relational model. Inheritance hierarchies can be mapped onto a set of tables in different ways; the choice results in a tradeoff between storage efficiency and query complexity, as a separate join table is required to implement a many-to-many relationship.
Storing objects in a database, which itself uses an object model, offers another solution. A variety of Object Oriented DBMS (OODBMS) products was developed, particularly during the 1990s, but such tools can be complex to configure and can require the use of an object definition language. The objects are stored as objects, but they are not native to the application language. These products have not had a major impact on the market outside some niche areas, and effort appears now to be mainly concentrated on object-oriented APIs for relational databases as well as on hybrid object-relational databases.
Object Persistence with the db4o Object Database
|
|
|
Message #148219
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Neither model is particularly better than the other, but the problem is that they are different and do not always sit together comfortably in the same application.
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard, but saying that a data model based on logic and mathematical principles is not better than a model based on "best practices by software engineers" is really something new.
So, besides selling us old stuff in new bags, what is your substitute for logic and set theory? Why do you still try to sell us something that was replaced more than 30 years ago? Why don't you stop the handwaving and tell us what is so great about your product? Why should we use procedural integrity, weird graph navigation and fixed access paths to our data instead of a much more flexible logical model?
|
|
|
Message #148234
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Seems to me that this type of database works best when either all apps that require a view of the database are written in the same language, java, or when there is only one app. That said, I'm guessing there are many domains where this kind of specialized database would make sense.
I don't know if a separate api exists for querying information out of the db40 database but in many environments separate systems all use the database as the collaboration-point. For example, batch mainframes would update tables that a java middle-tier would use. Or some 3rd party reporting software written in C would also access these tables.
I guess a web-services or corba wrapper could be thrown around the db40 api so that queried object structures could be passed to different platforms.
Anyway, more choice is more choice. If I was working on a stand-alone app I wouldn't mind something easy like db4o for persistence just because it seems much easier than writing a bunch of DAO's to force an object structure into a relational database.
|
|
|
Message #148252
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
However, the interface is complicated by the "impedance mismatch" between the domain object model of the application and the relational model of the database. It is very strange, I have never saw this "impedance mismatch". Probably domain model is wrong if some "impedance mismatch" exits, it is not RDBMS fault if you produce "impedance mismatch" in your model.
|
|
|
Message #148284
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
It is very strange, I have never saw this "impedance mismatch".
So what you're saying is you've never actually written an enterprise application for a Global 5000 or Fortune 500 company. If you think that all enterprise development (with projects of say, 20 or more developers) get complete control over the object model and database schema, you couldn't be more wrong.
- Don
|
|
|
Message #148285
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
So what you're saying is you've never actually written an enterprise application for a Global 5000 or Fortune 500 company. So "impedance mismatch" is related to "Global 5000 or Fortune 500" ?
BTW I am very glad to talk with experts.
|
|
|
Message #148287
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
It is very strange, I have never saw this "impedance mismatch". So what you're saying is you've never actually written an enterprise application for a Global 5000 or Fortune 500 company. If you think that all enterprise development (with projects of say, 20 or more developers) get complete control over the object model and database schema, you couldn't be more wrong. - Don
You can have Oracle act as an object database if you want to, no impedance mismatch here. But may be not the best performance for a near-real-time heavy loaded application !
I work in the Telecom industry where performance AND reliability are both equaly important. Usually for data internal to the application (no other application will even touch it, very rarely if not never transactionnal behaviour requiered as no concurrency access to the data) it would be suicide to use Oracle or any other RDBMS. We directly write to the file system, or use low-level APIs.
Then, this internal data can be batch exported to some RDBMS, for reporting, billing, ...
So i see absolutely no competition between OODBs and RDBMS from my point-of-view. Each has their market which don't overlap. Of course this is a pure engineering point-of-view. Than, you have to deal with your manager who just went out of a nice diner with his prefered RDBMS company's sales manager, got brainwhased (champagne and a gorgeous lady can help brain-washing...), and is now absolutely certain (...well, "looks" certain) that :
-you are a risk for the project, even though you are building it
-you might be good developper but your code will be unmaintainable
-no customer will want an application with custom code, they only want BasicForDummies that ensures no bugs and that anybody can maintain
-only big $$$ triple-secured servers + big $$$ RDBMS can efficiently and reliably store a handful of HTML pages
Then you know it is time to look for a different company before this one crashes (but your manager will have his anonymous bank account somewhere in a fiscal Paradise...).
My 2 cents :o)
|
|
|
Message #148289
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
It looks like COBOL soves all "impedance mismatch" problems for "Global 5000 or Fortune 500".
Agreed, this is kind of my point (sorry I was terse in the last post). I dusted off an old whitepaper I wrote a few years ago and posted it on my blog last month -- see it here.
The point is that, like it or not (and clearly none of us like it), corporate culture greatly impacts how software is developed. And I'm not even talking politics here, I'm just talking about dealing with legacy systems, integrating with partner applications, security considerations in many domains, managing large scale projects with 100's of developers, time to market considerations, developer skills, internationalization of development, etc.
It's just not fair to say that "I never have impedance mismatch, you must be dumb if you do" because all it proves is that you've never had to write software in the enterprise where many decisions are out of your control. Just as it's not fair to say "your technology sucks and trying to compare yours to mine means you're stuck in the 1960's". It's all about perspective.
- Don
|
|
|
Message #148290
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
So how "impedance mismatch" is related to "many decisions are out of your control" ? Are you talking about some technical problem or about "corporate culture" ?
Goole says "impedance mismatch" is not related to political problems
http://en.wikipedia.org/wiki/Impedance_mismatch, if it is a technical problem then it is not a problem for me.
|
|
|
Message #148291
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard,
There were hierarchical storage systems in the 60s, but certainly not object databases. Object database aren't 60s ideas - they have been around and used successfully in major projects for a long time - just ask Gemstone.
Why should we use procedural integrity, weird graph navigation and fixed access paths to our data instead of a much more flexible logical model?
The object databases I have used with Smalltalk years ago certainly had no weird navigation or fixed paths. Queries were no different to normal collection searches:
result := list select: [:item| item.amount == 10].
There we no fixed paths - there was retrieval by reachability from any connected object, or all instances of an Class could be accessed using (of course) the Smalltalk allInstances method (something I really miss in Java).
Compare this to the code and configuration required to use an ORM!
I would also disagree that the relational model is more flexible. It's robust and reliable, and has served the IT industry very well for decades, but can be a very bad match for many data structures and models.
|
|
|
Message #148294
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard, but saying that a data model based on logic and mathematical principles is not better than a model based on "best practices by software engineers" is really something new.
OO/network DBMS can be very useful when you need to handle large hierarchies - simple graph navigation can be times more fast and simple way to fetch data then SQL. Even ORACLE START WITH/CONNECT BY statement works VERY slow for hierarchies.
Maxim Kramarenko
TrackStudio - Hierarchical bug tracking software.
|
|
|
Message #148295
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
It's robust and reliable, and has served the IT industry very well for decades, but can be a very bad match for many data structures and models. Yes, relational database is not designed for object persistence, serialization is. I see no reason to use relational database for object persistence if serialization can be used to solve the same thing. Persistence is a very trivial thing, but I need more from database and I prefer to drop mismatching data structures and models for this reason.
|
|
|
Message #148296
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard,There were hierarchical storage systems in the 60s, but certainly not object databases. Object database aren't 60s ideas - they have been around and used successfully in major projects for a long time - just ask Gemstone.
Define "object"; I don't understand your point. Are you talking about a variable, a value, or a type? What is the difference between a network and an object database?
There we no fixed paths - there was retrieval by reachability from any connected object, or all instances of an Class could be accessed using (of course) the Smalltalk allInstances method (something I really miss in Java).Compare this to the code and configuration required to use an ORM!
So you have had the freedom to arbitrarily join and aggregate your information, right? No fixed access path.
would also disagree that the relational model is more flexible. It's robust and reliable, and has served the IT industry very well for decades, but can be a very bad match for many data structures and models.
Then, describe to us what is so inflexible about the relational model (and don't say "tree operations", because that is more than trivial) and how you would like to improve it to be a good match for "data structures and models". I think a statement couldn't be more vague than that, but lets see where we go with that.
|
|
|
Message #148297
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
OO/network DBMS can be very useful when you need to handle large hierarchies - simple graph navigation can be times more fast and simple way to fetch data then SQL. Even ORACLE START WITH/CONNECT BY statement works VERY slow for hierarchies.
ROTFL
|
|
|
Message #148299
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Everything as an object, 'Table' or 'View' is an 'object' too, is not it ?
|
|
|
Message #148300
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
OO/network DBMS can be very useful when you need to handle large hierarchies - simple graph navigation can be times more fast and simple way to fetch data then SQL. Even ORACLE START WITH/CONNECT BY statement works VERY slow for hierarchies. ROTFL
Possible, you have anything to add ? For example, you know a very fast and simple way to handle hierarchies with RDBMS :-)
Maxim Kramarenko
TrackStudio - Hierarchical bug tracking software
|
|
|
Message #148301
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
I see you do not have problems to handle hierarchies with RDBMS :
"The high performance rates and scalability, support for nearly all servlet containers and DBMS"
BTW I like your site, it was a good idea to use PHP.
|
|
|
Message #148303
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard,There were hierarchical storage systems in the 60s, but certainly not object databases. Object database aren't 60s ideas - they have been around and used successfully in major projects for a long time - just ask Gemstone. Define "object"; I don't understand your point. Are you talking about a variable, a value, or a type?
If you understood Smalltalk (which I believe was the main language using object database in the early 90s) there is no difference - everything is an object (even Classes are objects).
What is the difference between a network and an object database?
A network database is a database in which record type can have multiple owners. An object database is a persistence system which is ... designed to store and retrieve objects! These are definitions at different levels of API. Object databases may, at lower levels, use all sorts of storage methods - direct to filesystems, or even a network (or relational!) database.
So you have had the freedom to arbitrarily join and aggregate your information, right? No fixed access path.
yes, absolutely. Any relationship or that could be expressed in the Smalltalk language (complex field selections, sets, lists, etc.) would all operate efficiently within the database. In some systems, a full Smalltalk implementation as the native language of the database.
would also disagree that the relational model is more flexible. It's robust and reliable, and has served the IT industry very well for decades, but can be a very bad match for many data structures and models. Then, describe to us what is so inflexible about the relational model (and don't say "tree operations", because that is more than trivial) and how you would like to improve it to be a good match for "data structures and models". I think a statement couldn't be more vague than that, but lets see where we go with that.
I did not say it was inflexible. I said it was a bad match for some models and data structures. Implementing things like inheritance can be messy, for example, or polymorphic collections.. or multimedia. Relational systems are also bad at coping with change: the developer alters schemas on live systems at their peril. In good object databases, new classes can be defined, and instances of those classes stored, in active applications.
As for improving it: Well, there have been many better ways to store information in some cases. I was pretty impressed by Prolog in the 80s and liked the idea of a persistent storage of arbitrary facts which could be retrieved via an inference engine. I have come across many business situations where this kind of system would be more effective than a relational database. This is certainly not practical in most cases, but the relational model is just one model (a good model in most cases, certainly), and is not always the 'best fit' to real data.
|
|
|
Message #148304
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
A network database is a database in which record type can have multiple owners. An object database is a persistence system which is ... designed to store and retrieve objects!
So, an object database can store "values, variables, and types" (everything is an object?). Are you sure about this? As far as I know only values is what we store in a database, certainly using variables and data types to do so. If we like to communicate you'll have to define the terms you are using, and I'm not sure we agree on "object'. (Thats why I'm not using the word when talking about data management, were there is no room for ambiguities.)
Object databases may, at lower levels, use all sorts of storage methods - direct to filesystems, or even a network (or relational!) database.
Well, of course the physical implementation should never influence the logical model! So I don't get the point here, how a particular database system is implemented is pretty much irrelevant for the pure logical and conceptual model and perspective on your information. It should of course be the fastest physical implementation, no doubt about that. But frankly, I don't care about the implementation details and as an application developer, I should leave that to my DBA.
I don't know what to say about "multimedia" or your other points.
|
|
|
Message #148305
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
There are many good ideas in theory. I like deductive databases as idea too, but it looks like rule engines use relational databases in practice, the same is about object databases. It looks like relational model matches logical and object models if it is possible to use logical or object view on top of it. So I fail to see "independence mismatch".
|
|
|
Message #148314
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
There are many good ideas in theory. I like deductive databases as idea too, but it looks like rule engines use relational databases in practice, the same is about object databases. It looks like relational model matches logical and object models if it is possible to use logical or object view on top of it. So I fail to see "independence mismatch". As i said earlier, you can direcltly flush your java object graph in your preferred RDBMS instance. Just design a single table with two columns, first one being the the memory address of the object, the second being the serialisation of it (including its type, so it is really object oriented, and not only data, LOL !). You're done, 100%, instant out-of-the box object to relational bridge. Let me put a patent on it, sell it to Kodak and be a billionaire !
But for some usages it might just not be the most efficient implementation in terms of cost of hardware, disk usage, CPU, memory, ...
Everybody agrees that a vehicle is not necessarly a car, it can also be a bus, a ship, a plane... Each is appropriate for some specific usage.
Why should ALL data always go through a RDBMS ? Depending on the application, all data does not need transactions, concurrent access so complex locking, roll-back, complex joins, user-defined reporting...
I saw such a waste of harware and money on RDBMS, where everything would be running fine on a Pentium... Money has to change hands :o)))
|
|
|
Message #148318
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Why should ALL data always go through a RDBMS ? Depending on the application, all data does not need transactions, concurrent access so complex locking, roll-back, complex joins, user-defined reporting... Yes, some data does not need RDBMS, some data does not need any persistent cache too, is it "impedance mismatch" between the domain object model of the application and the relational model of the database ? Object database is just a trivial way to solve trivial problem, persistence. It is not any kind of alternative for RDBMS.
The ralational model is based on software engineering principles and models the entities a.k.a objects in the problem domain, this model is based on mathematical principles and organizes data for efficient storage and retrieval.
|
|
|
Message #148326
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Yes, some data does not need RDBMS, some data does not need any persistent cache too, is it "impedance mismatch" between the domain object model of the application and the relational model of the database ? Object database is just a trivial way to solve trivial problem, persistence. It is not any kind of alternative for RDBMS. The ralational model is based on software engineering principles and models the entities a.k.a objects in the problem domain, this model is based on mathematical principles and organizes data for efficient storage and retrieval.
Object database solving trivial problems ? I can not believe you are serious. You must be joking ! I can tell you that the oodb model is also based on software engineering principle ! Would you embbed an RDBMS in a smart-card chip ? Good luck my friend for a single application accessing it's own data ?!?
For sure the relational theory of data (which RDBMS "implements", and which is certainly not the only one) is a perfect "abstract" one. Have you ever noticed how every astract thing is perfect ? But then you have the implementation, that does not fits ALL needs of EVERYBODY.
The model does not organise the data for efficient storage and retrieval by itself : the model only exists on a piece of paper, in the mind of the reader. But it might be implemented, implementation which can be very well thought and tuned. No doubt about that.
All i want to say is that one-size-fits-all is a marketting trick. Do you agree with that, or will you jump back to the theory of the relational model, which i absolutely don't criticise here ? Hopefully after a good sleep you will provide real arguments.
Chris
PS : I just beat another team on my current project. They were implementing everything within Oracle with stored procedures (very experienced PL/SQL coders and Oracle administrators). I came up with java custom code, writing directly to the file system (yep i can do it, it does not break unless you hit the computer with a hammer, it is fault-tolerant in case of hardware failure, and it scales smoothly by just running another instance of it on another server) and using the RDBMS only for reporting purpose (see, i use it sometimes, only whn it is needed).
Guess what ? My solution is using 1/10th of their cpu and disk, is easier to maintain, has more features, and i will not speak about overall cost as i would like to stay on the technical level for this one.
By the way, be ensure that i do use and appreciate RDMS like Oracle, and even MSSQLServer on a daily basis.
Use the right tool for th right job, is my advise. But some people are simply religious about their preferred tool.
|
|
|
Message #148327
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
The ralational model is based on software engineering principles and models the entities a.k.a objects in the problem domain, this model is based on mathematical principles and organizes data for efficient storage and retrieval.
Again, mixing logical and physical concerns.
The relational data model foremost is a way to guarantee data integrity at a conceptual level (thus preserving the semantics of the information), and for avoiding redundancy through normalization! If we wouldn't have these integrity and normalization features our data would be in danger, and if we use ad-hoc models instead of one based on logic, data access operations could not guarantee the safety of the information.
Fast access to data is a physical concern and an implementation detail. What does concurrency control have to do with normalization? The relational model does not organize data for efficient storage and retrieval, in a strict sense. How could it, its at a completely different level. Our database _implementations_ have to take care of efficient storage and retrieval. If an implementation is too slow or inefficient, how could it be the fault of the logical model? It's the fault of the developer who implemented it or the vendor of the DBMS that was used. Granted, data models might exist that are difficult to implement in efficiently, but thats certainly not true for the relational data model.
Please don't make it harder for everyone to see the real problems. We still have many of them but reviving old network database ideas as our logical model is not the way to a solution. We need better implementations of the relational model. So far the big DBMS vendors got away with their SQL products because we don't ask them the right questions and let them sell anything they come up with. Getting confused and tricked by another group of vendors (the network database guys) is not making this much better.
|
|
|
Message #148329
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
I see you do not have problems to handle hierarchies with RDBMS :"The high performance rates and scalability, support for nearly all servlet containers and DBMS".
It was major pain for us. The very old version of our software (that supports ORACLE only because we really needed START WITH/CONNECT BY) works very slow on even 2000-3000 tasks. Last (beta) version works fine with 50.000 tasks and require just 200-300 MB RAM.
But now:
1) We store full object tree in the memory - Hibernate object/query cache is not enough here. For example, if we add or move a single task - we shouldn't invalidate all tasks.
2) We can't use SQL queries to locate objects in tree, its too slow way.
3) We can't use RBMS transaction isolation levels or locks because we need to synchronize RDBMS changes with cache.
4) RDBMS can't help us with clustering support, we need to implement distributed cache (swarmcache-based) and r/w locks manually. In the better world RDBMS can cache data and implement distributed locks for us.
5) Session replication in application servers is pain to configure, in 3.1 we'll store sessions and replicate them between cluster nodes manually. I know the Coherence approach, but for single application our way seems more easy :-)
6) We can't use SP, triggers, sub-selects, self-correlated queries, etc to support all those databases. Actually only simple DELETE/INSERT/UPDATE/SELECT used.
7) Yes, we use referential integrity, but it is not enough for hierarchies. We need a lot of additional code to avoid and check for hierarchy-specific problems.
We start several years ago with slow application and huge use of all RDBMS features. I remember SP, triggers and very complex queries; most complex of them contains 10 joins, 4 subselects and 2 START WITH/CONNECT BY in different directions :-) Now its much faster, but we can't use most RDBMS features.
it was a good idea to use PHP . It works good and we know it :-)
|
|
|
Message #148331
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
I use Lotus Notes for document management including bug tracking. It is hierachical database, is not it ?
Hierachical databases like file system are very usefull too.
This model (directed acyclic graph) is based on mathematical principles, does not it ?
Probably it is possible to find use case for object databases too, but I am afraid it is crap if it is based on something else than mathematical principles and logic.
|
|
|
Message #148332
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
So, an object database can store "values, variables, and types" (everything is an object?). Are you sure about this? As far as I know only values is what we store in a database, certainly using variables and data types to do so.
Sorry - I was talking about Smalltalk the language, not the database. The object databases I used had smalltalk as the native language. In Smalltalk, everything is an Object and new classes can be easily defined (and re-defined) at run-time. This made Smalltalk a particularly bad fit to relational systems, as the limited range of column types available in an RDBMS, and with those column types being very hard to change once set up.
Well, of course the physical implementation should never influence the logical model! So I don't get the point here, how a particular database system is implemented is pretty much irrelevant for the pure logical and conceptual model and perspective on your information. It should of course be the fastest physical implementation, no doubt about that. But frankly, I don't care about the implementation details and as an application developer, I should leave that to my DBA.
Absolutely! You are right. However, you asked me what the difference was between a Network database and an Object database. I was pointing out that, in my opinion, talk of a 'difference' is not appropriate, as a network storage model is just one way of implementing a Object database.
I don't know what to say about "multimedia" or your other points.
I'll try and be clearer: my point is that relational storage is superb for reasonably regular data. The mapping between relational databases and developer's code worked really well a couple of decades ago, before the widespread use of OOP: C structs and Pascal Records could be coded that looked very much like the database records.
However, when developing in an object-oriented manner, it's common to end up with very irregular data: collections which may hold more than one class of object (and with the different classes possibly being of very different structure). We also may have to work with very variable sized and often large data: multimedia or images for example. I know that relational systems can handle this kind of thing, and with the right coding relational systems can be very fast at this. However, I would argue that relational databases aren't a natural way to handle this variety of data: there is too much that has to be done to set up additional tables and columns to represent these linked objects and the connections between them. Relational databases can handle very large values through BLOB or LONG types, but the implementations of these often seem to be a kludge - something bolted awkwardly on: look at the constraints with LONG in Oracle for example.
|
|
|
Message #148333
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Persistence is trivial
I see no reason to use relational database for object persistence if serialization can be used to solve the same thing. For persistence, I personally advocate lots of RAM and a big, mean UPS. If anyone thinks that this won't work, then he probably needs more than persistence...
|
|
|
Message #148334
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Persistence is trivial
>For persistence, I personally advocate lots of
>RAM and a big, mean UPS. If anyone thinks that
>this won't work, then he probably needs more
>than persistence...
Do you think your router should work this way?
If adding the extra memory reduced a companies
margins by 5% do you think they should do that?
If the UPS fails and your router is down, is
that ok with you? If i can't afford a big
mean UPS for all my boxen what shall I do?
If i am buying redundant machine to handle
HA why not through in some $100 disks so
i can handle a power failure too?
|
|
|
Message #148335
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
> I came up with java custom code, writing directly
> to the file system
Your database has grown to a 10,000 records. Not very big. Does your file system handle that well? Not many do. Now you have to do a query against all those records to match an attribute value. You are screwed. It takes forever to search all the files on the disk.
|
|
|
Message #148337
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
http://blog.hibernate.org/cgi-bin/blosxom.cgi/2004/12/04#gettricked
Thank you for the answer, I can say the following:
I'm using the PREDICT operator provided by my search engine if I'd like to predict future. If no such operator is available or if I find its performance characteristics unacceptable I call my search engine vendor and complain until they fix it.
Name your database vendor. If nobody implements good hierarchical processing for RDBMS, why you think that is is possible ? I needn't to call DBMS vendor, we already have implemented such logic over RDBMS and it is not a minor improvement or additional operator. Its another way to handle data, cache and lock them. For example, when we set write lock for node, DBMS should set write lock to its descendants.
PS. I remember a lot of products that require hierarchies and avoid RDBMS to store them. Some examples: most file systems, MS Outlook, MS Exchange Server, Lotus Notes, Active Directory are hierarchies at the low level.
Maxim Kramarenko
TrackStudio - Hierarchical bug tracking software.
|
|
|
Message #148339
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
natural way to handle this variety of data: there is too much that has to be done to set up additional tables and columns to represent these linked objects and the connections between them. Relational databases can handle very large values through BLOB or LONG types, but the implementations of these often seem to be a kludge - something bolted awkwardly on: look at the constraints with LONG in Oracle for example.
If your values are "large" or a "binary" (thats multimedia, right?) or if your values are forming rather complex (I hate that term) structures (data without structure doesn't exist, that would be only noise), the relational model is just right. I can't see any limitations in the model that haven't either been resolved or at least discussed three times over in the last 30 years. I've personally only spent a much shorter time on this but the available documentation is pretty good.
If your DBMS doesn't handle large binary values properly don't buy it next time. Make sure you tell your boss why so he doesn't do it as well. Even if that only works 1 out of 10 it's already getting better.
|
|
|
Message #148340
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Name your database vendor. If nobody implements good hierarchical processing for RDBMS, why you think that is is possible ?
And if I do name a database "vendor" or an implementation that is good, would you accept it if its name isn't Oracle, DB2, Sybase, IBM, or MySQL? Then we would probably talk about "no marketshare" and then I'd give up :) I don't think thats a good end.
|
|
|
Message #148342
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
> I came up with java custom code, writing directly > to the file systemYour database has grown to a 10,000 records. Not very big. Does your file system handle that well? Not many do. Now you have to do a query against all those records to match an attribute value. You are screwed. It takes forever to search all the files on the disk. Only one file, Sorry ! LOOOOOOOOOOL ! :o)))
|
|
|
Message #148345
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Probably we all use plain file to store records, I use it for logging, it matches all data structures and models I need for persistent logging too.
|
|
|
Message #148346
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Name your database vendor. If nobody implements good hierarchical processing for RDBMS, why you think that is is possible ? And if I do name a database "vendor" or an implementation that is good, would you accept it if its name isn't Oracle, DB2, Sybase, IBM, or MySQL? Then we would probably talk about "no marketshare" and then I'd give up :) I don't think thats a good end.
Sure, I needn't marketshare, just the solution/idea :-)
I don't know what "good" implementation for you, but for me it should
1) Execute EXPLODE as fast as access by primary key
2) Tree modification should be fast. Move/insert/delete single node (with descendants) should be as fast as update single row by known primary key.
|
|
|
Message #148348
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Neither model is particularly better than the other, but the problem is that they are different and do not always sit together comfortably in the same application. This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard, but saying that a data model based on logic and mathematical principles is not better than a model based on "best practices by software engineers" is really something new.So, besides selling us old stuff in new bags, what is your substitute for logic and set theory? Why do you still try to sell us something that was replaced more than 30 years ago? Why don't you stop the handwaving and tell us what is so great about your product? Why should we use procedural integrity, weird graph navigation and fixed access paths to our data instead of a much more flexible logical model?
Christian!
I am not sure if I undersand your point here..
What exactly are you criticizing?
When EJB 1.0 was out it was sometimes promoted as a plaform which delivers Object Database which usees RDBMS as the storage.
This is something which Dion refered to as "hybrid solution".
If somebody is choosing to work with JDO, EJB and other O/R framwork he is often using the same level of abstraction as he would use in case of pure Object Database.
Are you trying to say here that your Hibernate model should not be used as the "logical model" (whatever it means) as
it is a bad one? Do you also think that if it ever was possible to use different storage in Hibernate then Relational Database in a transparent way for end user and even if most (all) of the functionaliy would be preserved that it will be definitly and unconditinally always worst then Hibernate + RDBMS combo?
Michal
|
|
|
Message #148350
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
If your values are "large" or a "binary" (thats multimedia, right?) or if your values are forming rather complex (I hate that term) structures (data without structure doesn't exist, that would be only noise), the relational model is just right.
I think that there is good evidence that the basic relational model is not an easy fit for object storage (as an example of rather 'complex' structures!): for example, many relational database vendors provide extensions to the database engine to handle objects. Why would they bother if the relational model was a good fit?
I can't see any limitations in the model that haven't either been resolved or at least discussed three times over in the last 30 years. I've personally only spent a much shorter time on this but the available documentation is pretty good.
Well, I have been developing for nearly 30 years, and have seen the same discussions over and over! One obvious limitation is that the relational system does not have the ability to express the semantics of object-orientation. Sure, you can use a relational system to store objects, and you can have columns that contain some kind of class id, but the interpretation of that id has to be at a different level. Another limitation is that there is no idea of intrinsic order. Sure, you can put in an auto-increment number and use that to order a table, but there is nothing that would indicate to anyone else who came across the database that this column has the meaning of being the order. These are just two...
You may mention vendor-specific extensions here, but they are exactly that - extensions, and not part of the relational model.
If your DBMS doesn't handle large binary values properly don't buy it next time. Make sure you tell your boss why so he doesn't do it as well. Even if that only works 1 out of 10 it's already getting better.
I didn't say it doesn't handle large binary values properly - I said that the handling of large binary values is a 'kludge' - something 'bolted on' to the relational model. It's not in principle a vendor-specific issue (although I have a vendor-specific example) - it's a general point.
Also, I would not suspect that going to a boss or decision maker (although that is me in most cases) and telling them not to use Oracle because of the LONG field restrictions would get a positive reaction......
I must state I share your enthusiasm for relational systems, and I the ORM systems I use work well for me. I think my only argument with you is about their flexibility and fit for certain uses and data. Personally, I would still use relational systems for these situations, even with the bad fit, as I require their robustness and guarantee of data integrity.
|
|
|
Message #148351
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
I think that there is good evidence that the basic relational model is not an easy fit for object storage (as an example of rather 'complex' structures!): for example, many relational database vendors provide extensions to the database engine to handle objects. Why would they bother if the relational model was a good fit?
I'm very direct now: There is no difference between an object and a value. Both have a type, from a predefined domain. They are assigned to variables. You work with operators that are related to types. Thats as easy as I can explain it and I hate that it is so much closer to a simple lie.
A good starting point is Introduction to Database Systems, Chris Date.
One obvious limitation is that the relational system does not have the ability to express the semantics of object-orientation.
That's really not true. As for the rest of your statements, they are very very far away from the "truth" (yeah, I know). I think the basics are fairly easy to learn (lets say one weekend with the right material).
I'm teaching training courses about good data management practices. Not for a long time, but already I can see confusion when it comes to good practices and some theory in my class. It takes only a few hours to get everyone to the same level, and so far, I've had excellent responses after the course. Yes, I think that Java developer (and it seems people with a broader background as well) need to know more about the basics. I don't think we talk about the same thing.
End of rant...
|
|
|
Message #148352
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Are you trying to say here that your Hibernate model should not be used as the "logical model" (whatever it means) as
it is a bad one? Do you also think that if it ever was possible to use different storage in Hibernate then Relational Database in a transparent way for end user and even if most (all) of the functionaliy would be preserved that it will be definitly and unconditinally always worst then Hibernate + RDBMS combo?
No! I'm saying that we should think about the future as well. Our current tools take care of most of the daily issues, just enough to make it acceptable. I think we are not at the end of what we can achieve .Considering the 5 ORM "mismatch" problems again (Hibernate in Action...), they are almost all easily solvable if we could just get our industry moving (which is just so much more difficult).
I think that ORM tools might be a chance for the next step, towards relational type systems in the database that are also native to the application language (that means Java classes in the DB, just like in SQLJ). At some point this bridge will allow us to execute procedural Java code in the database and even later we will get portable polymorphic systems. But each is a huge step for the size of the market we are in.
Talking about SQLJ, the right idea at the wrong time. Oracle has announced EOL for support, IBM will have to keep some up. We should probably try that again and get it right. If you look at why it failed, you can clearly see where it went wrong: the implementations just weren't usable. Nothing much wrong with the concept, really.
Often a second round can make it better. My goal here is to tell everyone that Hibernate3, EJB3, etc. is not the last thing that will have to be learned by Java developers. However, it only gets better if everyone knows what to ask for.
|
|
|
Message #148353
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
One obvious limitation is that the relational system does not have the ability to express the semantics of object-orientation. That's really not true. As for the rest of your statements, they are very very far away from the "truth" (yeah, I know).
Well, I'm afraid that's exactly what I feel about your statements. I guess we could move into the fine detail of these matters, but I don't see any us finding any common ground.
Yes, I think that Java developer (and it seems people with a broader background as well) need to know more about the basics. I don't think we talk about the same thing.End of rant...
I assumed that this was a respectful discussion about relational databases. I certainly would not assume that anyone who disagreed with me was simply in need of taking the right course, and lacking in knowledge of 'the basics'.
Anyway, thanks for the debate.
|
|
|
Message #148354
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
No! I'm saying that we should think about the future as well. Our current tools take care of most of the daily issues, just enough to make it acceptable. I think we are not at the end of what we can achieve .Considering the 5 ORM "mismatch" problems again (Hibernate in Action...), they are almost all easily solvable if we could just get our industry moving (which is just so much more difficult). Definitly when it comes to datbases this is the last domain when revolution should be ever exeptected.
The most important part of it: Data was/is often living much longer then any plaftorms in the past.
I think that ORM tools might be a chance for the next step, towards relational type systems in the database that are also native to the application language (that means Java classes in the DB, just like in SQLJ). At some point this bridge will allow us to execute procedural Java code in the database and even later we will get portable polymorphic systems. But each is a huge step for the size of the market we are in.Talking about SQLJ, the right idea at the wrong time. Oracle has announced EOL for support, IBM will have to keep some up. We should probably try that again and get it right. If you look at why it failed, you can clearly see where it went wrong: the implementations just weren't usable. Nothing much wrong with the concept, really.Often a second round can make it better. My goal here is to tell everyone that Hibernate3, EJB3, etc. is not the last thing that will have to be learned by Java developers. However, it only gets better if everyone knows what to ask for.
But I feel that this leads us towards the usable implementation of the idea Object Database. Isn't it? Relational Database /SQL, stored procedures still will be there. But objects will be logically closer to the data which is kept in the database. And relational databases will be more friedly for objects.
Michal
|
|
|
Message #148355
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
But I feel that this leads us towards the usable implementation of the idea Object Database. Isn't it? Relational Database /SQL, stored procedures still will be there. But objects will be logically closer to the data which is kept in the database. And relational databases will be more friedly for objects.
Guys, could you just stop using the word "object" for a second? So far it couldn't be even defined in this thread, so clearly its inadequate for any further discussion.
|
|
|
Message #148356
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
I assumed that this was a respectful discussion about relational databases. I certainly would not assume that anyone who disagreed with me was simply in need of taking the right course, and lacking in knowledge of 'the basics'.
Man, you have to take a step back. It's not really a disagreement here, I'm basically reciting database literature. If you don't want to hear that I'm going somewhere else.
|
|
|
Message #148357
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
But I feel that this leads us towards the usable implementation of the idea Object Database. Isn't it? Relational Database /SQL, stored procedures still will be there. But objects will be logically closer to the data which is kept in the database. And relational databases will be more friedly for objects. Guys, could you just stop using the word "object" for a second? So far it couldn't be even defined in this thread, so clearly its inadequate for any further discussion.
Well, havn't I made myself clear - for me framworks like your hibernate + any relational database provides "hybrid" implementation of object database. The API which you are using with Hiberante is the same kind of API which you might use to talk to pure object datbase.
So the meaning of the term object is the same as you are using in hibernate. Object database interally for storing data may use any theoretical/phicilcal model they wish including the relational one. Only differnce is that
even if you wish you may not see it directly as you can still do it with Hibernate when you are using relational databases. But when number of features provided by Hibernate will grow the number of reasons to by pass Hibernate Query Language will be descending.
michal
|
|
|
Message #148359
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Back 2 Reality
Some solutions to this problem, such as Hibernate and Java Data Objects, are designed to provide the developer with transparent persistence: the application deals with persistent objects using an object-oriented API without the need for SQL code to be embedded in the Java code.
JDO was designed to be independent of data store - Sun even put out a File store for 1.0.1 that although not fully implemented proved the concept.
Now reality is setting in (DB has Data, Java has Objects), and market forces drive JDO to be more SQL-like. This is not bad, as now we can focus on optimizing a solution that works using sound DB principles, sound Object principles, and a sound architectural principles (seperation of concerns, mapping in between).
You want to be productive and earn money in business (not in lab or university), you accept this and move forward.
Meanwhile, it's really funny to see the OO-weenies argue in circles with the data-heads.
|
|
|
Message #148368
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Back 2 Reality
ORM is nothing more than an adapter to map SQL and programming language types. RDBMS can be accessed by many languages and I see no reason to add all concepts from programming languages to data, it does not need to depend on programming language a.k.a "software engineering principles". I think it is a good idea to have language neutral model in database and to avoid
"inpedence mismatch".
|
|
|
Message #148384
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Probably we all use plain file to store records, I use it for logging, it matches all data structures and models I need for persistent logging too.
I do not use it for logging. I use it to store millions (yes, millions) of records of non-transactional data which are accessed 50% read and 50% write. I was storing structured data with direct acess to file system API 18 years ago, as most developers had to do by the way.
I helped developped/maintain a C program (with helper assembly code) to index content for CD-ROMs 18 years ago that was sold to a fortune 100 company, for long live legal storage of documents. 650MBytes 18 years ago was real large data for most PCs. We beat an DBMS based solution because we had taylor code that fitted the needs 10 times better.
I know what i can do by myself with not that many lines of code, and what is better delegated to a product. And i don't deny that the major RDBMS are good roducts by the way, and never did.
Read back the thread when you have a good sleep.
Adios.
|
|
|
Message #148387
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Lightweight persistence
I can't see any FUD in this article.
I think it is very assuming to discredit non-relational persistence solution in such general terms. And frankly, it doesn't give you much credibility.
I have no experience with db4o but I love prevayler (http://www.prevayler.org) (the lead developer of prevayler is working for db4o). As I understand it, Prevayler is simpler than db4o but not so suitable for embedded devices.
Obviously there are many usecases where RDBMS are the best fit. But there are also many usecase where RDBMS are unnecessary and there are much more lightweight approaches to object persistence than RDBMS. Particularly I would like to point out to prevayler.
In many, many cases this is a fanatstic solution
* No external container
* One stays completely in the Java world
* Extremely fast and reliable
* Very easy to test and refactor
* Significantly less development time and less debugging time
Disadvantages:
* Data has to fit into the RAM (often no Problem, sometimes impossible)
* No distributed transactions
* Data is not so easily accessible for data warehouses and stuff like this.
Some people recently trying to spread FUD about solutions like prevayler. Those people I have in mind, have a personal interest to denounce this technology. Sadly this interest is more important to them than technological decency.
I recommend to have a look at prevayler. It's not for nothing that people like Kent Beck and Ward Cunningham have contributed to this project.
And I'd like to point out to the prevayler manifesto which is great fun to read.
|
|
|
Message #148389
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Back 2 Reality
ORM is nothing more than an adapter to map SQL and programming language types. RDBMS can be accessed by many languages and I see no reason to add all concepts from programming languages to data, it does not need to depend on programming language a.k.a "software engineering principles". I think it is a good idea to have language neutral model in database and to avoid"inpedence mismatch".
In database you have both the data and models. Looking from the certain perspective - you have set of records on your disk(s), b-trees, indicies and stuff like that.
Models are one level up then that. I don't really see how
you we can at all avoid "adding all concepts from programming languages to data". SQL and relational model is all about that and it is really a good thing. Thanks to that have a sound and decent model for accessing our data (records/files) and actually it is what makes databases useful. But SQL is already the programming language and logical organization of data in the database is also driven by the needs of that model (e.g. the concept of "table"). There are some other orthogonal subjects to that like data integrity, transactions etc which are not so important here.
So are you trying to say that there is a place for single model only in the databases and that there can be nothing more useful and simpler then relational model? I don't think so.
Database like Oracle give you already far more then just that ...
You have special things for dealing with XML (this is actually a horrible thing :)), graphs/trees and so on. So the idea is: you can have multiple possible views (models) over the same data in the same database engine and you are able to choose the best one for your needs.
Michal
|
|
|
Message #148390
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Back 2 Reality
ORM is nothing more than an adapter to map SQL and programming language types. RDBMS can be accessed by many languages and I see no reason to add all concepts from programming languages to data, it does not need to depend on programming language a.k.a "software engineering principles". I think it is a good idea to have language neutral model in database and to avoid"inpedence mismatch". In database you have both the data and models. Looking from the certain perspective - you have set of records on your disk(s), b-trees, indicies and stuff like that. Models are one level up then that. I don't really see howyou we can at all avoid "adding all concepts from programming languages to data". SQL and relational model is all about that and it is really a good thing. Thanks to that have a sound and decent model for accessing our data (records/files) and actually it is what makes databases useful. But SQL is already the programming language and logical organization of data in the database is also driven by the needs of that model (e.g. the concept of "table"). There are some other orthogonal subjects to that like data integrity, transactions etc which are not so important here.So are you trying to say that there is a place for single model only in the databases and that there can be nothing more useful and simpler then relational model? I don't think so.Database like Oracle give you already far more then just that ...You have special things for dealing with XML (this is actually a horrible thing :)), graphs/trees and so on. So the idea is: you can have multiple possible views (models) over the same data in the same database engine and you are able to choose the best one for your needs.Michal
Michal,
+1000. Very well said an demonstrated, i keep this "Oracle XML special datatype" as a smasher to the next "RDBMS are the one and only way to handle data storage" argument i will obsiously get form customers/collegues.
Have a nice day !
Christian
|
|
|
Message #148394
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Back 2 Reality
So are you trying to say that there is a place for single model only in the databases and that there can be nothing more useful and simpler then relational model? No, I am saying there is no mismatch and ralational model does not conflict with "good OOP practice".
|
|
|
Message #148423
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
RDBMS is slow too
> I came up with java custom code, writing directly > to the file systemYour database has grown to a 10,000 records. Not very big. Does your file system handle that well? Not many do. Now you have to do a query against all those records to match an attribute value. You are screwed. It takes forever to search all the files on the disk.
An RBMS would be slow at this too... unless you indexed it. Likewise, he could develop an index into his 10,000 files.
|
|
|
Message #148426
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
I think that there is good evidence that the basic relational model is not an easy fit for object storage (as an example of rather 'complex' structures!): for example, many relational database vendors provide extensions to the database engine to handle objects. Why would they bother if the relational model was a good fit? I'm very direct now: There is no difference between an object and a value. Both have a type, from a predefined domain. They are assigned to variables. You work with operators that are related to types. Thats as easy as I can explain it and I hate that it is so much closer to a simple lie
One of the points of object-orietned programming is data encapsulation. The data cannot be modified directly, but only through messages passed to an object in the form of method calls.
An RDBMS exposes raw, naked data. You can query any data, any time, anywhere. The data is unprotected. If you have an "age" column in a person table, there is nothing to stop you from putting a "-1" in there. Of course, you could use triggers or other RDBMS-implementation-specific thinsg to stop such nonsense, but then you are duplicating business logic.
A OODBMS should prevent that form ever happening. You define your Person object with a setAge method, and that method ensures that the age is always non-negative. Now if someone wants to try to call setAge with a negative value, it will fail.
The ability to arbitrarily manipulate data in an RDBMS is both its power and its a achilles heel. I tmakes it great for ad-hoc reporting, searching, diagniostics, quick-fix updates, etc. But the constraints they put on data to fit into their model causes a huge problem with aspects of OOP, like encapsulation, inheritence, etc.
|
|
|
Message #148427
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
RDBMS is slow too
> I came up with java custom code, writing directly > to the file systemYour database has grown to a 10,000 records. Not very big. Does your file system handle that well? Not many do. Now you have to do a query against all those records to match an attribute value. You are screwed. It takes forever to search all the files on the disk. An RBMS would be slow at this too... unless you indexed it. Likewise, he could develop an index into his 10,000 files.
I don't have 10.000 files. I repeat : i don't have 10.000 files... :o)))
|
|
|
Message #148430
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
One of the points of object-orietned programming is data encapsulation. The data cannot be modified directly, but only through messages passed to an object in the form of method calls.
There is no problems to "hide" data if you must do it, use get/set pattern from beans API.
An RDBMS exposes raw, naked data. You can query any data, any time, anywhere. The data is unprotected. If you have an "age" column in a person table, there is nothing to stop you from putting a "-1" in there. Of course, you could use triggers or other RDBMS-implementation-specific thinsg to stop such nonsense, but then you are duplicating business logic.A OODBMS should prevent that form ever happening. You define your Person object with a setAge method, and that method ensures that the age is always non-negative. Now if someone wants to try to call setAge with a negative value, it will fail. CHECK( age >= 0), I see nothing evil in this definition. It is more evil to hardcode it in application.The ability to arbitrarily manipulate data in an RDBMS is both its power and its a achilles heel. I tmakes it great for ad-hoc reporting, searching, diagniostics, quick-fix updates, etc. But the constraints they put on data to fit into their model causes a huge problem with aspects of OOP, like encapsulation, inheritence, etc. It doe's not cause any problems, there are many design and mapping patterns to solve it, relational model matches any correct ( logical ) model, you can generate adapter code from model and you will have all cool RDBMS features you like and adapters (application objects) at the same time.
|
|
|
Message #148451
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
CHECK( age >= 0), I see nothing evil in this definition. It is more evil to hardcode it in application.
Well, now it seems we're back to the old problem of where to code it. Do you hard-code it in the application and let the database be dumb? Or do you hard-code it in the database (which is what I interpret your proposal as) and make the application dumb?
Now, if you hard-code it exclusively in the database, I assert that this precludes any Object-Oriented Programming because the object is no longer in control of its data. Someone else could write a new app that also modifies the same database, and insert data that violates the expected rules for my application reading that data.
If you hard-code it in both, then you've got double-maintinence.
If you hard-code it only in the application, then anyone could modify your data if its in an RDBMS. But, if you instead store objects in an object database, then tehy can't get to your data -- only your objects. And your objects, if designed properly, will protect your data.
I'm not saying not to use an RDBMS. Just acknowledge that doing so violates OO encapsulation. Its a risk, and all risks can be managed. But it is a risk you wouldn't have with a proer object database.
|
|
|
Message #148452
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
I'm not saying not to use an RDBMS. Just acknowledge that doing so violates OO encapsulation. Its a risk, and all risks can be managed. But it is a risk you wouldn't have with a proer object database.
Where can we find this definition of "object encapsulation"?
|
|
|
Message #148453
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Lightweight persistence - Prevayler
...Particularly I would like to point out to prevayler.In many, many cases this is a fanatstic solution
* No external container
* One stays completely in the Java world
* Extremely fast and reliable
* Very easy to test and refactor
* Significantly less development time and less debugging time
Disadvantages:
* Data has to fit into the RAM (often no Problem, sometimes impossible)
* No distributed transactions
* Data is not so easily accessible for data warehouses and stuff like this.
I read about Prevayler some time ago, but never used it. One disadvantage I recall about it that you forgot to mention is that you can no longer use your typical time/date classes. You have to use their special classes so that they can be replayed back. This is an extra step when adapting an existing system.
Another thing I'd like to point out for people who think there are no tools for arbitrarily querying/reporting from an ODBMS. JXPath (http://jakarta.apache.org/commons/jxpath/) is one tool I think has promise. It uses an XPath-like syntax to allow you to navigate an object-graph. Thus, you could get all addresses in your contact list with the 90210 zip code like so:
locations[address/zipCode='90210']/address
I'd like to see such a syntax married with an ODBMS.
|
|
|
Message #148457
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
Ah, so "object encapsulation" for this person means hiding the internal representation and present the client of your type (class) with a public interface of operators (methods). I don't see anything about business rules, constraints or integrity there.
Ok, so why can't you have scalar types (thats what they are, if they don't present internal components, but all manipulation of values of that type have to be done through operators) in the relational data model? Some (including the inventor of the model) would argue that tuples hold scalar values, so that should work just fine, right?
|
|
|
Message #148464
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
object != value
If you hard-code it in both, then you've got double-maintinence. Yes, this is one of reasons to generate adapters. I am not expert, but I hope you can define model in programming language neutral way for OODBMS too, it causes the same "problem", you need to generate adapter for some "action" language like JAVA.
|
|
|
Message #148465
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
This is the kind of FUD we have heard for more than 10 years from network/object-database vendors. I know, selling ideas from the 60's can be quite hard, but saying that a data model based on logic and mathematical principles is not better than a model based on "best practices by software engineers" is really something new. OO/network DBMS can be very useful when you need to handle large hierarchies - simple graph navigation can be times more fast and simple way to fetch data then SQL. Even ORACLE START WITH/CONNECT BY statement works VERY slow for hierarchies.Maxim KramarenkoTrackStudio - Hierarchical bug tracking software.
Suggest reading Joe Celko's SQL for Smarties: Advanced SQL Programminghttp://www.amazon.com/exec/obidos/tg/detail/-/1558605762/002-2004642-6599214?v=glance
and try using nested sets model instead of usual adjacency list model.
|
|
|
Message #148498
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Suggest reading Joe Celko's SQL for Smarties: Advanced SQL Programminghttp://www.amazon.com/exec/obidos/tg/detail/-/1558605762/002-2004642-6599214?v=glanceand try using nested sets model instead of usual adjacency list model.
Yes, I know this idea - please check out
http://www.intelligententerprise.com/001020/celko.jhtml?_requestid=29530
Its very slow, I'll try to describe our requirements.
Our application implements task hierarchy (like projectA->moduleB->taskC). Access control is used to determine which level of the task tree a user's status (and thus the permission set associated with it) is applicable on. For example, status 'developer' for projectA will be inherited for 'taskC' automatically.
Moreover, we supports field-level security. Each get/set method should check permissions before set/get field value. For example, Task.getHandler() should
1) Gather user permissions for task (require START WITH/CONNECT BY). This can be cached in the constructor.
2) Check that user has permissions to view task handlers for this task.
3) Check that user has permissions to view that specific handler (require START WITH/CONNECT BY)
To show task list (that contains 1000 tasks, for example), we need to know user status for each task and permissions for each value. This means that we should execute START WITH/CONNECT BY query (or any other syntax) 3000-5000 times times in a second or even faster. Even query by primary key can't be so fast :-)
This is very simple example, but it shows that hierarchical processing often require not just fast START WITH/CONNECT BY, but another way to navigate, store and cache data.
Maxim Kramarenko
TrackStudio - Hierarchical bug tracking software.
|
|
|
Message #148507
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Hierarchy example
This means that we should execute START WITH/CONNECT BY query (or any other syntax) 3000-5000 times times in a second or even faster. Even query by primary key can't be so fast :-)This is very simple example, but it shows that hierarchical processing often require not just fast START WITH/CONNECT BY, but another way to navigate, store and cache data. No, this shows that you should make a query to check permissions for the whole list of items. Making a separate check for each item almost the same mistake as the dreaded 1+N loading in BMP entity beans.
|
|
|
Message #148544
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Hierarchy example
No, this shows that you should make a query to check permissions for the whole list of items. Making a separate check for each item almost the same mistake as the dreaded 1+N loading in BMP entity beans.
Yes, we has such one-in-one query several years ago: it was dynamically-generated query that contains about 10 joins, 4 subselects and 2 START WITH/CONNECT BY in different directions :-) It was very slow even on small database (2000-3000 tasks) and the application model itself was much simpler than current one (no ability to override permissions for tasks, no field-level security, etc).
I remember that it was very funny for us - 1 GB RAM, 500 MB database server, 10 MB of data and 20 seconds to execute query. I am agree that another tree implemenation can be faster, but not 500 times faster.
Note, that I don't mean that complex hierarchical logic can't be implemented using RDBMS. Really, it can be implemented and can work fast. But RDBMS (or application above it) should work as hieararchical DBMS, it should know that some table represents a tree, know how to cache it, how to lock data, etc.
PS. Even early we uses CMP, it was really nighmare for our application :-)
Maxim Kramarenko,
TrackStudio - Hierarchical Bug Tracking Software.
|
|
|
Message #148551
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Object Persistence with the db4o Object Database
Hibernate second level cache will probably do the trick for you.
Yes, we use both object and query Hibernate caches, both help a lot. But application internally require 2 additional cache levels to store task/user tree with permissions and initialized objects.
|
|
|
Message #148752
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Three O-R mismatches, not one
I think it's sad that someone deeply involved in a successful and neat Java product like Hibernate has retarded to citing DBDebunk FUD.
Java and .NET are mainstream, right? It's only natural that people simply want to store objects, right? Now how can this be done efficiently and fast with existing tools?
It's clear that there is a mismatch between Java and SQL, otherwise O-R mappers like Hibernate would have absolutely no reason to exist.
However there is not only one mismatch, there are three:
(1) The O-R language mismatch
Java doesn't directly support storing objects to SQL databases so you need to go through an intermediate layer, a different language, SQL. If you do this by hand, refactoring will kill you. If you use a tool to do all the
SQL for you, you will be fighting all kinds of issues with the tool and with performance.
(2) The O-R model mismatch
Commonly used SQL databases do not support inheritance. Inheritance can be somewhat simulated using joins but approaches to do so will quickly experience queries to hit the complexity and performance wall.
(3) The O-R performance mismatch
Object databases use direct pointers for navigation. Access to objects does not have to go through a single index lookup. I can easily prove the performance difference with a simple code example. I challenge any O-R
mapper vendor to provide an equivalent solution to store and load a deep tree and I bet that noone will get close to 1/10th of the speed my db4o solution offers.
How does all this compare to the "true set theory", DBDebunk FUD always comes up with? I have absolutely no idea. Some people tend to step back to theory when they have been beaten in practice: "But my project is better in theory."
Here is my tree code:
import java.io.*;
import com.db4o.*;
public class Tree {
static final String FILE = "tree.yap";
static final int DEPTH = 15;
Tree preceding;
Tree subsequent;
public static void main(String[] args) {
new File(FILE).delete();
Tree tree = createTree(DEPTH);
ObjectContainer objectContainer = Db4o.openFile(FILE);
long start = System.currentTimeMillis();
objectContainer.set(tree);
long id = objectContainer.ext().getID(tree);
objectContainer.close();
long stop = System.currentTimeMillis();
System.out.println("Time to store tree: " +
(stop - start) + "ms");
objectContainer = Db4o.openFile(FILE);
start = System.currentTimeMillis();
tree = (Tree)objectContainer.ext().getByID(id);
objectContainer.activate(tree, Integer.MAX_VALUE);
stop = System.currentTimeMillis();
System.out.println("Time to load tree: " +
(stop - start) + "ms");
objectContainer.close();
}
public static Tree createTree(int depth){
if(depth < 1){
return null;
}
Tree tree = new Tree();
tree.preceding = createTree(depth - 1);
tree.subsequent = createTree(depth - 1);
return tree;
}
}
|
|
|
Message #148753
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
db4o is not old stuff in new bags
db4o is not old stuff ...
- Persistence is non-intrusive. You simply store any object without having to run an enhancer or a special classloader.
- True object querying is provided using query-by-example.
- It's native Java.
- Simple refactorings like adding and removing fields are applied automatically on-the-fly without having to reorganise a database file.
- Object identities are watched and preserved.
- All Java constructs are supported.
- Installation is as simple as it can be. Deploy one 350k Jar along with your application.
...but I agree with respect to the "new bags":
- open source
|
|
|
Message #148773
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Fear
I think it's sad that someone deeply involved in a successful and neat Java product like Hibernate has retarded to citing DBDebunk FUD.
If I had a product prolonging the morbid days of the relational database by putting object makeup on it, I would also be scared to death of object databases finally being made viable by the opensource movement.
Hibernate folks are in a most uncomfortable position where they have to defend the relational model and still justify hiding it behind an OO veil.
Why not use SQL in the first place? Why go through the hassle of writing an object-relational mapper? Why not use the "wonderful" relational model directly?
Because it sucks. They know that.
Nobody in their sane mind would start writing an object-relational mapper if they didn't think so.
See you, Klaus.
db4o - The Opensource Object Database
Prevayler - Databases in Memoriam
|
|
|
Message #148778
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Relational DBMSs Considered Harmful
The burden of proving relational databases are better than object databases:
"The relational model is mathematically correct."
That is important but it is not enough. The Chinese Remainder Theorem is also mathematically correct but it is useless as database technology.
"The object model is not mathematically proven."
Such a statement might have been relevant 30 years ago when OO languages were still seminal. Today, it is just like saying: "Your bank transactions rely on the unproven conjecture that NP-Complete problems can be solved in exponential time only.". It is true but it is just silly. It is just FUD.
Actually, OO is a superset of the relational model. I could restrict my OO programs to using only object classes such as Table, Row, Column, Relation, Tuple (whatever you want to call them) if I really was that sick.
I can implement a relational database using an OO language, but a relational database is pretty much useless without an OO or even a procedural language.
So the relational subset of OO is mathematically proven. If amidst the whining and finger-pointing, anyone ever finds a flaw in the extensions OO provides, we can all "happily" fallback to relational.
"You can represent any data structure and all possible integrity constraints using the relational model"
You can also represent any data structure and all possible integrity constraints using an infinite array of bits and bitwise boolean operations. That still sucks.
So the burden really is to prove that the relational model is a non-sucking way to represent data, not merely a sufficient one.
Since that is a matter of opinion, this discussion is pretty much pointless.
Should we have an opinion poll or is the fact that nobody uses an SQL engine in RAM, when persistence is not required, enough to settle the issue?
"OO databases are no more than network databases"
OO databases allow for behavior. Network databases do not. So the burden here is to prove that behavior does not exist.
"Separation of data and behavior is a good thing"
The term "separation of concerns" is often abused to refer to this. Separation of concerns is a good thing and OO is all about separation of concerns: cohesion, encapsulation, modularity (concepts foreign to the relational model).
Separating data and behavior is something completely different. It is the same as programming using only tables kept in global variables. You can even have a fancy language such as SQL to make accessing those global variable "easier", but it still only accessing global variables.
So the burden here is to prove that programming only with global variables does not suck.
Do we really need to discuss this?
See you, Klaus.
db4o - Objects are Here to Stay
Prevayler - Do You Still Use a Database?
|
|
|
Message #148828
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Serialization does not work for longterm persistence
I see no reason to use relational database for object persistence if serialization can be used to solve the same thing.
Serialization should not be used for longterm persistence for the following reasons:
(1) Serialization will break if the class model is modified.
(2) Serialization provides no mechanisms for querying.
(3) Serialization is not transactional.
(4) Serialization does not provide a mechanism to update individual objects. It requires the complete object tree to be stored and loaded every time and that makes it slow and it requires lots of memory.
(5) Serialization does not provide multi-user or remote-connection features.
|
|
|
| New content on TheServerSide.comNew content on TheServerSide.comNew content on TheServerSide.com |
|
|
Reza Rahman explores the features of the proposed JSR 299, Contexts and Dependency Injection for Java EE (CDI). When approved, it promises to be a key feature of Java EE 6.
(November 2, Article)
SAML is an XML-based standard for exchanging authentication and authorization data between security domains. The single most important problem that SAML was created to solve is the Web browser Single Sign-On problem. Many organizations are debating whether to stay with version 1.1 or move to 2.0. This article makes observations about both options.
(September 28, Article)
Joe Ottinger takes a look at how people learn, and applies it to the practice of programming. He notes that understanding how people learn is an essential part of working in a programming team.
(September 22, Article)
Stephen Maryka gave us an article about the Asynchronous Web and posed a number of questions that get examined like an approach to delivering Asynchronous Web capabilities through extensions to existing Java EE technologies.
(July 14, Article)
JavaServer Faces Flex goal is to provide users capability in creating standard Flex components, part of flexSDK which is open sourced through MPL license, as normal JSF components. This article by Ji Hoon Kim will provide an overview of creating a simple multilingual JSF page consisting of JSF Flex tags.
(June 29, Article)
In this session Jeff explores the key characteristics of successful SOA projects. He covers some of the patterns, and anti-patterns, tool sets, and strategies that he himself learned the hard way. Last, he provides a strategy and blueprint for achieving a high likelihood of success in your SOA project.
(June 23, Tech Talk)
Ari Zilka, CTO of Terracotta, Inc., talks about the new features in Terracotta 3.1, announced during JavaOne and available now.
(June 15, Tech Talk)
In this Tech Talk, Josh Long explores an integration challenge using Spring Integration and walks through the implementation, employing and expanding on the basic patterns of Enterprise Application Integration to tie together components into a function integration solution, and then demonstrates how Spring Integration helps address the integration requirements.
(June 15, Tech Talk)
In this Tech Talk, David Geary teaches you: The basics of Google Web Toolkit; How to implement Ajax-enabled applications in Java; Internationalization; Hooking into the browser history mechanism; Remote procedure calls.
(June 4, Tech Talk)
Jon Kern discusses the best architecture/technical solutions and ensure that they are repeated by all developers. By tackling the architecture up-front in a serial manner, subsequent parallel development will be much more manageable and predictable.
(May 28, Tech Talk)
This keynote describes the frustrations of modern knowledge workers in their quest to actually get some work done, and solutions for how to guard yourself against all those distractions. Neal Ford talks about environments, coding, acceleration, automation, and avoiding repetition as ways to defeat the misguided attempts to sap your ability to produce good work.
(May 26, Tech Talk)
Gil demonstrates how new, aggressive uses of already abundant compute capacity by common applications offer competitive value for application designers.
(May 21, Tech Talk)
Chris Keene introduces WaveMaker as a new way to automate the ability to generate Hibernate classes in order to more quickly bring OR mapping into an application.
(May 19, Article)
In this session Nati Shalom demonstrates how to take a standard Java EE web application and scale it out or down dynamically without changes to the application code. Seeing as most web applications are over-provisioned to meet infrequent peak loads, this is a dramatic change because it enables growing your application as needed, when needed, without paying for unutilized resources.
(May 19, Tech Talk)
Mastering EJB was one of the original and most influential EJB books in the industry. Mastering EJB III now returns with two new expert co-authors, updated for EJB 2.1 and 30% new chapters including security, integration, best practices, open source, and more.
(Book PDF Download)
The Application Server Matrix is a detailed listing of J2EE vendors and their application server products, with information on latest version numbers, J2EE spec support and licensing, pricing, platform support, and links to product downloads and reviews.
(Application Server Comparison Matrix)
|
|
