db4o 5.0 with Native Queries, a new way of expressing queries

Discussions

News: db4o 5.0 with Native Queries, a new way of expressing queries

  1. db4objects Inc. has announced the availability of version 5.0 of the open source object database db4o.

    db4o version 5.0 comes with Native queries, a new approach to express database queries with Java syntax, completely without the use of strings. Native Queries are fully checked at compile time, they can be written using IDE autocompletion and they can be automatically refactored by the IDE. Under the hood, Native Queries are optimized to use database indexes by analyzing Java bytecode.

    An example of the native query facility from the project site:
    List students = database.query( new Predicate(){
          public boolean match(Student student){
            return student.getAge() < 20
              && student.getGrade().equals(gradeA);}})
    By providing full compliance with the Java Language Specification, Native Queries eliminate the need to learn other non-mainstream database APIs besides Java.

    What do you think?

    Threaded Messages (94)

  2. Lots of typing.[ Go to top ]

    Seems like a neat trick, but do I really want to write all that as apposed to:

    FROM Student s WHERE s.age < 20 AND s.grade = "GradeA"
  3. Criteria API[ Go to top ]

    If you use persisten layer which is partialy generated with Hibernate and you generate extension above Hibernate Criteria API then you can achieve following:
    - Pure object oriented queries
    - Type and name safe query interface
    - Great performance (translated to native SQL queries)
    - Well know technology behind (SQL)

    With hibernate behind I don't feel big difference between object database or relational database. Are object database still important for java developers?
  4. Criteria API[ Go to top ]

    If you use persisten layer which is partialy generated with Hibernate and you generate extension above Hibernate Criteria API then you can achieve following:- Pure object oriented queries- Type and name safe query interface- Great performance (translated to native SQL queries)- Well know technology behind (SQL)With hibernate behind I don't feel big difference between object database or relational database. Are object database still important for java developers?
    Of course, you also get [R]DBA's and DM's along with anyone who knows how to use WinSQL (or Toad or ...).
  5. Criteria API[ Go to top ]

    I'm a big fan of the Hibernate Criteria API. IMO it's the only reason to use Hibernate. The rest of the stuff in Hibernate wasn't all that thrilling or time saving. In Hibernate 2.x the API was quite limited, but it has come into it's own in 3.x.

    I'm not the biggest fan of Hibernate (If you have asked for help on the forum, you'll find yourself wanting to go to Atlanta will very ill intentions). But the Criteria API has saved me loads of time dealing with a UI that has upwards of 8 zero-to-many criteria boxes. If I tried to do that in any form of SQL coding (JDBC, HQL, etc) it would be a complete disaster.

    John Murray
    Sobetech
  6. In fact they are...[ Go to top ]

    Are object database still important for java developers?

    They are very important... Take operations on huge object trees for an example. An object database is simply the better matching model for this kind data. SQL imposes a huge overhead on those operations - dozens if not hundreds of joins in the background to assemble the tree are just not the perfect solution.
  7. In fact they are...[ Go to top ]

    Take operations on huge object trees for an example. An object database is simply the better matching model for this kind data.

    If I need to work with huge trees, maybe I should consider something rather specific for huge trees.

    For example, Berkeley DB XML
    http://www.sleepycat.com/products/bdbxml.html
  8. I for one prefer SQL syntax to the example. I'm bias, but sql seems less error proned to me than using a pure Java syntax for defining a query.

    peter
  9. I for one prefer SQL syntax to the example. I'm bias, but sql seems less error proned to me than using a pure Java syntax for defining a query.
    I wonder how Java could be more error prone than SQL. I assume that Java's static checking is "less error prone" by definition. You're wrong to say that safe queries are "supposed to be easier than SQL". They aren't supposed to be easier. Safe queries are supposed to be safer. Get it?
  10. I for one prefer SQL syntax to the example. I'm bias, but sql seems less error proned to me than using a pure Java syntax for defining a query.

    I wonder how Java could be more error prone than SQL. I assume that Java's static checking is "less error prone" by definition. You're wrong to say that safe queries are "supposed to be easier than SQL". They aren't supposed to be easier. Safe queries are supposed to be safer. Get it?

    I'll clarify. To me it's less error proned because I have a few years of experience writing sql queries and it feels more "natural" to me. Plus, it's simpler to read and therefore less error proned to me. To someone else, that probably isn't true. It doesn't matter if the model is Object Oriented or Relational, the developer "has" to know the model.

    Writing a query with a ton of curly brackets to my eyes is harder to read than sql. Plus, I'm not convinced that most developers have a great grasp of modeling theory and understand all the subtleties. Not that I do, since there are plenty of things in UML and modeling theory I find confusing. Having written my own schema compiler, modeling theory is rather complex and most people only learn enough to "get the job done."

    Object databases are genuinely useful, but there is a significant learning curve. To me, there isn't a great advantage to Object databases. It's just different. At the end of the day, it's the problems one chooses to live with.

    peter
  11. Plus, it's [SQL is] simpler to read and therefore less error proned to me.
    Your anecdote doesn't jive with the post immediately above yours:
    "If I tried to do that in any form of SQL coding (JDBC, HQL, etc) it would be a complete disaster."
    If you're right about SQL being easy to read, then why would SQL be dismissed as "a complete disaster"? Could it be that SQL isn't as readily hand coded as you suggest?
    Writing a query with a ton of curly brackets to my eyes is harder to read than sql.
    Safe queries enjoy all the power of Eclipse, the code completion, the refactoring, the syntax coloring, the cross referencing. SQL gets none of this, and there's no static checking. SQL can't be aspected, annotated, or otherwise enhanced, and this surely has implications for your hand coded SQL. Development with SQL might be slower. Maintenance with SQL might be harder. SQL can't tell you that a column is never used. Eclipse can tell that a safe-queriable field is never used.
  12. Safe queries enjoy all the power of Eclipse, the code completion, the refactoring, the syntax coloring, the cross referencing. SQL gets none of this, and there's no static checking. SQL can't be aspected, annotated, or otherwise enhanced, and this surely has implications for your hand coded SQL. Development with SQL might be slower. Maintenance with SQL might be harder. SQL can't tell you that a column is never used. Eclipse can tell that a safe-queriable field is never used.

    I don't believe that is true. The only way an IDE would know a specific field is not used is to execute a query. The same thing can be done for RDBMS and OODBMS. For small databases, that's probably fine, but fact is, the size and content of the database during development time most likely will be different than the production database. Or are you suggesting developers query against the production database while working on an application?

    Sure it's nice to have fancy IDE's, but is that substitution for knowing the database and really understanding what one is building? When I'm working on data related tasks, I usually have a sql console up so I can run adhoc queries. Some people like having everything be in one tool, but I don't feel that is necessarily always good or even good.

    Syntax is a human issue to me. Arguing that Native Query is superior is a not a valid argument to me. It's equivalent to saying "english is the best language."

    peter
  13. I like SQL[ Go to top ]

    First of all, this is interesting discussion. Thanks for the different threads of the debate.

    Nothing against db4o, just like to second peter lin's point that there are some programmers like SQL. I am one the those one. He feel comfortable with SQL, so do I. I used the Hibernate HSQL and still like the raw SQL's power.

    SQL is easy to read and it's provides me the expressiveness power to do most of the thing I want to do. Using java syntax (we have our own java syntax, may not be a good one) does make it harder to understand what's the original query trying to do. We have something have to put the original intented SQL in the comments along side with the Java syntax to make it clear.

    There are tools (IDE) for SQL that can helps with code completion, syntax coloring et. For example, TOAD for Oracle on windows does the colors and syntax check etc. TORA for Oracle on linux does the code completion.

    SQL can be directly checked, analyszed (EXPLAIN PLAN) and optimized (run statics, in MS SQL Server 7 or later, this is simply one click) within these IDEs (TOAD for Oracle, Query Analyser Tool for SQL server, DB2 Console for DB2 etc.)

    I feel SQL is very easy to me, but that might be just me. Maintenance SQL would not be any harder than any other pieces of code if one has the SQL skill.

    Now, that has been said, I am also open mind for OODBMS, I would like to try db4o sometime. But after read all the postings, I am still a bit unclear about this: if I am not developing PDA, nor desktop application, if I am only developing an enterprise application, why should I using OODBMS instead of the RDBMS ? What's the chances for enterprise to adapt my choice of technology if I am using OODBMS ?

    My 2 cents and questions.

    Chester Chen
  14. The only way an IDE would know a specific field is not used is to execute a query. The same thing can be done for RDBMS and OODBMS.
    Uh, no. Eclipse can detect that a member is unused (ie, Find References returns none). Eclipse allows safe queries to be examined and refactored in ways that SQL can't. There's no refactoring editor for SQL.
    Or are you suggesting developers query against the production database while working on an application?
    I'm talking about kinds of static analysis applied to safe queries. Static analysis is done without a database connection. SQL lacks tooling for static analysis. Safe queries are usable with lots of commodity tooling that does static analysis. Eg, javac does tons of static analysis on safe queries and catches whole categories of bugs that aren't caught with SQL tools until runtime. Since it's more productive to catch bugs at build time, therefor safe queries are more productive than SQL.
  15. Unused doesn't mean not useful data[ Go to top ]

    The only way an IDE would know a specific field is not used is to execute a query. The same thing can be done for RDBMS and OODBMS.

    Uh, no. Eclipse can detect that a member is unused (ie, Find References returns none). Eclipse allows safe queries to be examined and refactored in ways that SQL can't. There's no refactoring editor for SQL.

    thanks for explaining what you meant. It wasn't clear to me. I agree there are nice benefits with static type checking and analyzing the model to determine if a field is used. But just because a field or object is not used, does not mean it's not useful or won't be used in the future. To get a complete picture of whether an object or field is used, one would have to have all the applcations loaded in eclipse and search every single class. For small projects, that's probably fine. But what about a large company that has dozens of development groups, each with it's own source control. If you don't have access to their source code or jar files, how can you know for sure a field is not used? Just to be clear, the ability to do static type checking is nice and can prevent some errors, but I think its useful depends on the environment.
    Or are you suggesting developers query against the production database while working on an application?

    I'm talking about kinds of static analysis applied to safe queries. Static analysis is done without a database connection. SQL lacks tooling for static analysis. Safe queries are usable with lots of commodity tooling that does static analysis. Eg, javac does tons of static analysis on safe queries and catches whole categories of bugs that aren't caught with SQL tools until runtime. Since it's more productive to catch bugs at build time, therefor safe queries are more productive than SQL.

    Again, I disagree. Visual studios can make a connection to the database and generate type information. That information is then used for a variety things like generating XML Schema. The resulting schema can then be used for static type checking. To the best of my knowledge, there isn't an IDE that does this level of integration between the database and IDE, but it can be done.

    For you NQ are more productive, and that's completely valid. To argue someone else is less productive in SQL doesn't make any sense to me. My original compliant is the format of db4objects query is hard for me to read compared to SQL or HSQL. Whether something is "harder/easier" from a human perspective has very little to do with whether it is technically better.

    peter
  16. performance[ Go to top ]

    Having never used an Object-oriented database, does anyone have any experience as to the performance in relation to any of the "common" relational DBs like Oracle, DB2, PostGres, etc?

    I've seen some pretty large differences in metrics, but would like to get some "real world" feedback. Furthermore, (and this is where I declare that I haven't read the entire product docs) does db4o support transactions (normal and XA), stored procs, etc?

    At the very least, it certainly looks like something that would be interesting to play with.

    Cheers
  17. performance[ Go to top ]

    One thing about object databases is that the various vendors have radically different architectures. So, some are very scalable and high performance and others are dogs.

    I can't say anything regarding db40 performance or scalability, but The Versant Object Database is much faster than relational dbms as the models become more complex and the data volumes increase.

    We were recently selected by the European Space Agency for exactly that reason. The following are the results of their testing that clearly show that OODBMS's ( Versant in particular ) are much faster then relational. In this case, comparing to Oracle.

    ftp.versant.com/pub/outgoing/benchmark/planck_bench_results.pdf

    With the standardization of Java API's ( JSR220 and JSR243 ) that work well with both relational and object databases, OODB's will have a stronger presence in future application design.

    Robert Greene
    Versant Corporation
    www.versant.com
  18. performance[ Go to top ]

    Hi,I am interested in benchmarks for OODBs, do you have the source code for this benchmark ?

    Diamond G.
  19. performance[ Go to top ]

    I will inquire on this, but I think ESA owns the rights to it.

    Cheers,

    Robert Greene
    Versant Corporation
    www.versant.com
  20. Native queries eliminate the need of trying to get the industry behind a JSR or other organization's backed database API by using what's already there: Java semantics. So you start with a standard rather than trying to establish standards which, in the past, have always remained nichy and are, IMO, one of the biggest reasons why ODBMS did not get adopted in the first place.

    Versant and other of the old-style ODBMS vendors have participated in the pushs to establish OQL (disbanded) and JDO 1 (disbanded).

    Apart from this, NQs are also entirely native and use no strings. This makes NQs 100% typesafe, 100% refactorable, and 100% object-oriented, making developers much more productive.

    Also, non-native APIs, non-native OODBMS, and O/R mapping are much less performant than a native, embeddable object database like db4o, which has been fastest in most cases in recent open source database benchmarks: http://www.polepos.org.

    More on the benefits and design goals of NQs in the Cook/Rosenberger whitepaper: http://www.db4o.com/about/productinformation/whitepapers/

    Or just download db4o V5.0 for free: http://www.db4o.com/community/

    Christof Wittig
    db4objects
    www.db4o.com
  21. Over simplification[ Go to top ]

    Native queries eliminate the need of trying to get the industry behind a JSR or other organization's backed database API by using what's already there: Java semantics. So you start with a standard rather than trying to establish standards which, in the past, have always remained nichy and are, IMO, one of the biggest reasons why ODBMS did not get adopted in the first place.

    Versant and other of the old-style ODBMS vendors have participated in the pushs to establish OQL (disbanded) and JDO 1 (disbanded).

    Apart from this, NQs are also entirely native and use no strings. This makes NQs 100% typesafe, 100% refactorable, and 100% object-oriented, making developers much more productive.

    Also, non-native APIs, non-native OODBMS, and O/R mapping are much less performant than a native, embeddable object database like db4o, which has been fastest in most cases in recent open source database benchmarks: http://www.polepos.org.More on the benefits and design goals of NQs in the Cook/Rosenberger whitepaper: http://www.db4o.com/about/productinformation/whitepapers/Or just download db4o V5.0 for free: http://www.db4o.com/community/
    Christof Wittig
    db4objectswww.db4o.com

    I think that argument is not valid for a couple of reasons.

    1. existing systems which use RDBMS still require integration and some sort of mapping beween objects and tables

    2. Unless Db2objects can capture every detail of the model like unidirection, bidirectional, one-to-many, one-to-one, many-to-many and inheritance, native queries will not provide significant benefit that is repeatable across a wide variety of projects

    3. very large terabyte databases are used in a variety of situations. has db4objects proven it is efficient, scalable and maintainable for these cases?

    4. no clear integratio/migration path is given, so who is going to switch from RDBMS to db4objects?

    5. are there any Extract Transform Load features to assist customer who want to switch from RDBMS?

    object databases have certain strengths and are valuable tools. reducing the argument to standards is a red herring in my book. If db4objects wants to win mindshare, it's going to have to provide clear answers to these and other critical questions.

    peter
  22. 1. existing systems which use RDBMS still require integration and some sort of mapping beween objects and tables
    ...
    3. very large terabyte databases are used in a variety of situations. has db4objects proven it is efficient, scalable and maintainable for these cases?
    4. no clear integratio/migration path is given, so who is going to switch from RDBMS to db4objects?
    ...
    If db4objects wants to win mindshare, it's going to have to provide clear answers to these and other critical questions
    I'm happy to do this:

    db4objects' team and the fast growing community of now 7,500 users believes that object databases are best at use in distributed, zero-admin environments, such as on devices, in packaged software, and realtime control systems. Users typically don't target to replace Oracle etc. but give developers a choice that look for persistence solutions beyond RDBMS in client-centric environments: VDC research has shown, that more than 50% of developers in that space still write their own database; clearly there's something wrong with the incumbent offers.

    As a result, db4objects provides a clear answer to the above: There is no migration, but there are new areas of persistence, where conventional non-native technology falls short and where people with 250,000 downloads clearly demonstrate a need for alternatives over RYO, file-based, serialization, or complex RDBMS with O/R mapping that degrades performance and drives operational cost.

    Last not least, it is as easy as it is wrong to desribe an easy-to-use object database like db4o as not scalable or performant. db4o has been selected as the object cache for the Spanish high speed bullet train AVE because it was the only system that could reliably cache 200,000 objects per second in a safety-critical environment: http://www.db4o.com/about/customers/success/indra.aspx

    Also companies like Boeing builds db4o into their new generation of P8-A aircrafts in a zero-admin environment:
    http://www.db4o.com/about/customers/industries/transportation.aspx#boeing

    All of these are not toys, but sound economic choices after extensive evaluations and stress tests.

    Oh, above all: db4o is open source and free under the GPL...
    I think that argument is not valid for a couple of reasonsYou reply with this under the headline "over simplification" to 3 different arguments I have made. DOn't you think you oversimplify yourself.
  23. Hi everybody,

      All of us want to have some references: benchmarks, news and so on about the new products we'd like to use. Ok, this is a my real experience with db4o.

      I met Christof and Carl (hi guys!!) when I was working in my last company, Indra Sistemas in Spain. We were developing a very huge complex project for the high speed train. I was in charge of the integration department and one of my duties was to implement a "real time" database for caching the state of thousands of physical elements (at that time over 100.000 elements) deployed along the railway track, network infrastructure... and logical elements such as alarms fired by survillance system, fire detection system, obstacles detection system... as well; in a nutshell, a very huge amount of data changing most of them every second. Think about 10-14 trains running above 250 km/h!!!. The challenge was impressive, to maintain a mirror of the field.

       After some researches I took the decision of using db4o as the core of the integration infrastructure. At that moment some people thought it was a risk, but I demostrated to them, with help from db4o team and mine, there was no chance for such a fear. I began to use the version 2.0, and to be honest, we had problems and requirements didn't cover by the product like: indexes and "connection-less" remote query API (between the db4o client and the server we had a message middleware, Tibco Rendezvous), defragment problems.
    My team was able to implement the solution based on db4o and it became the heart of all the system. Most of the integrated subsystems use db4o as an global image of what is happening out there.

       A year and a half after I left the company, I keep in touch with my colleagues and I know the system is running with almost zero management. I feel proud of my creature and the decision I took because the company was able to fulfill the high expected requirements in such as a complex environment.

       Finally, my advice is: test, test and test again. Try to reach the limits of the product. The version 2.0 had its merits and defects for our project, but the first were pretty many as the second. The new version 5.0 should be a delicious!!! Keep it up Carl!!!

    Best regards,

    Roque Caballero

    P.D. Feel free to ask for details, if you want.
  24. Native queries eliminate the need of trying to get the industry behind a JSR or other organization's backed database API
    Sorry but that makes no sense at all to me.

    You still need some standard, you are just moving the problem from requiring a standard language (SQL) to a standard API (the OODB API in Java).

    How is your solution going to work for the next language after Java? How was it working when C++ was popular?

    SQL was working across these three paradigm shifts.

    History has shown us that paying the impedance mismatch price close to the language (SQL + O/R mapper to your programming language) is the only viable solution in a world where data outlasts programming languages by an order of magnitude in years.

    --
    Cedric
  25. How is your solution going to work for the next language after Java?

    It works for .NET1 and .NET2, as provided by db4o, so why not for the language after Java. You are right, it needs implementation in different languages, but you need that for SQL, too, if you want to use it (e.g., JDBC)
    History has shown us that paying the impedance mismatch price close to the language (SQL + O/R mapper to your programming language) is the only viable solution in a world where data outlasts programming languages by an order of magnitude in years.

    If history were always to be a predictor for the future, you are right. I dare to doubt this: Innovation is based on solving a real problem at its source, and that is the inherent incompatibility of OO languages and non-native, string-based APIs which degrades developer productivity. NQs solve exactly this problem.
  26. If history were always to be a predictor for the future, you are right.
    Fair enough.
      I dare to doubt this: Innovation is based on solving a real problem at its source, and that is the inherent incompatibility of OO languages and non-native, string-based APIs which degrades developer productivity.
    On the other hand, this statement flies in the face of reality.

    The "inherent incompatibility" between what's in your database and what's in your programming language is there, no matter what. You choose to solve that problem at the expense of querying, and that's your prerogative, but claiming that the current O/R mapping approach degrades productivity even though it's used by 99% of enterprise applications doesn't really present you (and by extension, your product) as very credible.

    Start by understanding and respecting the choice of the majority before you try to convert them.

    Revolutions always start with humility, not arrogance.

    --
    Cedric
  27. Use native Java as the OO database API[ Go to top ]

    Revolutions always start with humility, not arrogance.
    Well, at least all the good ones. :)
  28. Use native Java as the OO database API[ Go to top ]

    It's really nice. I used db4o a lot and found well but sometime feel that databasebase is somewhat big or there is no optimization if I delete objects from the database.

    Manoj
  29. How is your solution going to work for the next language after Java?
    It works for .NET1 and .NET2, as provided by db4o, so why not for the language after Java. You are right, it needs implementation in different languages, but you need that for SQL, too, if you want to use it (e.g., JDBC
    If db4o data is portable then you need programming language neutral DDL with mapping to PL datatypes and structs. You need PL neutral query languge for this reason too. So I do not think data stored in db4o is "persistent".
  30. If db4o data is portable then you need programming language neutral DDL with mapping to PL datatypes and structs. You need PL neutral query languge for this reason too. So I do not think data stored in db4o is "persistent".

    I don't agree.

    Think of a complex type of XML on WebService. The portability of objects depends on you.

    If you carefully design your objects stored in db4o or any other OODB, your objects becomes self-descriving(XSD), portable than ever.

    Takenori
  31. Use native Java as the OO database API[ Go to top ]

    You still need some standard, you are just moving the problem from requiring a standard language (SQL) to a standard API (the OODB API in Java).

    Thats true. But by moving the problem closer to my object model, I am able to eliminate one whole layer of standardization (the SQL layer). Without the NQ approach, I have two standards to play by... the SQL standard and the JDBC standard (read ADO.NET for .NET and similar modules/APIs for other languages).

    The NQ approach does need a new standard API (just like JDBC, as pointed out by Christof) but it works straight off of your object model, whatever that may be. No string based SQL queries...

    Believe it or not, its a great relief for object purists out there... and works very well, as demonstrated by a growing community out there. I am very willing to believe that these guys will improve upon (performance and ease of use) this stuff over a period of time and gain industry wide acceptance.
    How is your solution going to work for the next language after Java?
    Thats not too hard to imagine, really. You already have a different API/library/module for each language for talking in SQL... at the very least we can do the same here.
  32. Use native Java as the OO database API[ Go to top ]

    No string based SQL queries...
    This idea really needs to die.

    Here is the scoop: people *like* string-based SQL queries.

    String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*.

    Why?

    Because their defects are outweighed by their advantages.

    And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.
    Believe it or not, its a great relief for object purists out there...
    Well, not to me. And I'm an object purist.

    First of all, using ORM layers allow you to manipulate string-based queries and still use OO techniques to move your objects around.

    Second, the OODB approach is hardly more object-oriented. You are invoking methods on chains of objects... big deal. More typed? Yes. More object-oriented? Not even close.

    Look, I have nothing against OODB or Query API's. Really.

    But the attitude shown by advocates of these technologies is driving me (and a lot of people) nuts. Be nicer, pay attention to what users complain about instead of making up pain points and maybe they'll start listening to you.

    --
    Cedric
  33. Use native Java as the OO database API[ Go to top ]

    people *like* string-based SQL queries.String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*.Why?Because their defects are outweighed by their advantages.And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.

    I don't think there is that strong a connection between OODBs and use or non-use of string-based queries. Years back there seemed to be some (in my view) mistaken criticism of JDO based on the idea that it was tailored primarily for OODBs. JDO has always used string-based queries. OODB vendors have not been ignoring this point. For example, the Versant OODBMs can run an SQL dialect over an object database.

    The issue of native-language/string-based queries is an interesting one, but my view is that it is not that relevant to the matter of RDBMS/OODBMS.

    Personally, I prefer the use of string-based queries, as they can also be used in a more interactive and dynamic situation than Java currently allows.
  34. Personally, I prefer the use of string-based queries, as they can also be used in a more interactive and dynamic situation than Java currently allows.

    To interactively test and develop Native Queries with db4o you can use BeanShell... works like a charm and you can copy the finished code to your sources without changes.
  35. Use native Java as the OO database API[ Go to top ]

    This idea really needs to die.Here is the scoop: people *like* string-based SQL queries.String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*.Why?Because their defects are outweighed by their advantages.And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.

    Anyone who has debugged a stored-procedure in a package with 10 other stored-procedures and a dozen functions that talk to a bunch of tables and lots of views, knows how much s/he *likes* SQL... and the expressive power it gives. :-p

    IMHO, SQL sucks for OO programming... the impedance mismatch has to go. ORM hides (or shall I say tries to hide) the mismatch, it does not eliminate it.
    Well, not to me. And I'm an object purist.First of all, using ORM layers allow you to manipulate string-based queries and still use OO techniques to move your objects around.Second, the OODB approach is hardly more object-oriented. You are invoking methods on chains of objects... big deal. More typed? Yes. More object-oriented? Not even close.
    OK. So I write a String literal in my .JAVA file (or some XML/properties file). An intelligent parser (SQL or HQL or OQL or whatever) parses this String literal and invokes methods on my objects. Thats OO. And thats _not_ error prone. ?? !!

    Moreover, wouldn't dealing with just domain object models (read .JAVA files) be a whole lot more fun.

    BTW, with db4o (and others, who are listening), can I write a stored-procedure in Java? What about a trigger or a view?
  36. Use native Java as the OO database API[ Go to top ]

    This idea really needs to die.Here is the scoop: people *like* string-based SQL queries.String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*.Why?Because their defects are outweighed by their advantages.And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.
    Anyone who has debugged a stored-procedure in a package with 10 other stored-procedures and a dozen functions that talk to a bunch of tables and lots of views, knows how much s/he *likes* SQL... and the expressive power it gives. :-p IMHO, SQL sucks for OO programming... the impedance mismatch has to go. ORM hides (or shall I say tries to hide) the mismatch, it does not eliminate it.
    Well, not to me. And I'm an object purist.First of all, using ORM layers allow you to manipulate string-based queries and still use OO techniques to move your objects around.Second, the OODB approach is hardly more object-oriented. You are invoking methods on chains of objects... big deal. More typed? Yes. More object-oriented? Not even close.
    OK. So I write a String literal in my .JAVA file (or some XML/properties file). An intelligent parser (SQL or HQL or OQL or whatever) parses this String literal and invokes methods on my objects. Thats OO. And thats _not_ error prone. ?? !! Moreover, wouldn't dealing with just domain object models (read .JAVA files) be a whole lot more fun. BTW, with db4o (and others, who are listening), can I write a stored-procedure in Java? What about a trigger or a view?

    Oh... and just watch how long it takes for them to bring the LINQ stuff on board...

    http://msdn.microsoft.com/netframework/future/linq/

    Think productivity...
  37. Use native Java as the OO database API[ Go to top ]

    This idea really needs to die.Here is the scoop: people *like* string-based SQL queries.
    Anyone who has debugged a stored-procedure in a package with 10 other stored-procedures and a dozen functions that talk to a bunch of tables and lots of views, knows how much s/he *likes* SQL... and the expressive power it gives. :-p
    Such a person will probably dislike stored procedures or bad programmers as a result of the exercise, but this has little to do with SQL.

    Besides, you don't need to convince me that it's possible to abuse SQL and write unmaintainable code with it. The same can be said for any technology out there.
     IMHO, SQL sucks for OO programming... the impedance mismatch has to go.
    It can't go. Period.

    As I said above, there is an unreconcilable difference between the data in your database and the objects in your program, and anyone who pretends to sell a product that makes this impedance disappear is being dishonest.

    All you can do is bridge this gap, and so far, SQL and O/R mappers have been tremendously successful at it.
    Thats OO. And thats _not_ error prone. ?? !!
    I never said that SQL was not error prone, please reread my messages above.
     Moreover, wouldn't dealing with just domain object models (read .JAVA files) be a whole lot more fun.
    Absolutely, and Hibernate, Ruby on Rails and ASP.Net have proven that it was quite possible to do so with SQL.
     BTW, with db4o (and others, who are listening), can I write a stored-procedure in Java? What about a trigger or a view?
    I don't know, but these are good questions.

    --
    Cedric
  38. people *like* string-based SQL queries.String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them

    All I hear you saying is "legacy". I respect that you like it more because you know it. But it wouldn't harm to try to understand why other people *like* a truly object-oriented persistence solution and a native API -- and to learn that nearly 10,000 registered users of db4o are not all masochists.

    My guess is that we're talking about different use cases. I'm not saying you should drop Hibernate + Oracle, for instance, in the data warehouse of a fortune 500. But you should retire if you want to implement the same on a PDA, on 10,000 desktop clients, or in the distributed data acquisition architecture of a SCADA system. And why shouldn't a PDA have a database? Where is the legacy here?

    All we want to do is to give developers a truly OO choice when it comes to persistence, be it for performance reasons, resource constrains on devices, to meet zero-admin requirements, for automatic schema evolution for seamless updates, or to accomodate very complex, perhaps dynamically evolving object models, e.g. for a proteine simulation.

    I think the problem with object databases so far were that they were slow, difficult to learn, non-native in their non-standardized APIs, and targeted to replace Oracle rather than to do what they are best for: be embedded invisibly in a larger application stack. No wonder that these vendors are now entirely verticalized and sell more services than licenses (ObjectStore for instance has been renamed into RFID division of Progress.)

    I think we have listened to users requests and eliminate all of the above flaws with db4o's open source object database offering.

    Christof
  39. I think the problem with object databases so far were that they were slow, difficult to learn, non-native in their non-standardized APIs, and targeted to replace Oracle rather than to do what they are best for: be embedded invisibly in a larger application stack.

    I simply don't recognise this description. A decade ago I was using one of the most popular object databases of that time - Gemstone/Smalltalk. It was fast, transparent, native and because of this transparency was largely invisible in an application.

    db4o seems to have a real advantage in terms of size - I can see this being a major benefit in the growing mobile/PDA market. However, by setting up straw men arguments about other systems being less OOP, less transparent or less performant you are not helping to promote this. That is my view anyway.
  40. Use native Java as the OO database API[ Go to top ]

    Thanks for saying it for me ;-)

    Robert Greene
    Versant Corporation
    www.versant.com
  41. and to learn that nearly 10,000 registered users of db4o are not all masochists.
    I'm not really interested in comparing numbers or challenging that these users are really active, but I am just observing that for one OODB user, you probably have 100 using SQL. You can always find users of any technology, no matter how abandoned or obsolete it is, but using these numbers as evidence of popularity is amusingly naive.
    All we want to do is to give developers a truly OO choice when it comes to persistence
    You seem to assume they don't already have one, which makes me think you really need to take a look at some Hibernate or Ruby on Rails code.
    I think we have listened to users requests and eliminate all of the above flaws with db4o's open source object database offering.Christof
    That's actually another thing that keeps happening: I have pretty much only ever seen OODB defended in public forums by employees of OODB companies.

    Doesn't really help make the technology more credible.

    --
    Cedric
  42. and to learn that nearly 10,000 registered users of db4o are not all masochists.
    I'm not really interested in comparing numbers or challenging that these users are really active

    These are new, i.e. entirely active registrations over the last year for a new product with a brand-new query API which you assumingly have not evaluated but try to counter with old arguments...
    ...you really need to take a look at some Hibernate or Ruby on Rails code.

    ... and without listening to my argument, that you probably would not choose Hibernate to run in a zero-admin environment, e.g., on a PDA, would you?

    Don't forget: The fast fish eat the big fish.
  43. You seem to assume they don't already have one, which makes me think you really need to take a look at some Hibernate or Ruby on Rails code.

    Comparing Hibernate or Ruby on Rails (???) with an OODBMS is amusingly naive. Did you even *try* one before saying? There was no *real* alternative to RDBMS. There is an open source project called Ozone, wich seems dead, and it's API a bit intrusive.
    That's actually another thing that keeps happening: I have pretty much only ever seen OODB defended in public forums by employees of OODB companies.Doesn't really help make the technology more credible.

    You can count me as an OODB defender, and I don't work for an OODB company.

    I used Zope for a long time, wich embed an OODB (ZODB). And believe me, it was a joy. Not for the Python language, but for the persistence transparency that it provided. There was no O/R to worry about, no query to write, nothing. And the performance was spectacular and contant as the size of the BD increased.

    I didn't try db4o yet. I definitively will. If my experience with ZOBD is any indication of what the technology has to offer, I'm sold.

    Honestly, I only see two reasons for the slow adoption of OODBMS : Inertia, and RDBMS vendors FUD. Do Oracle or Microsoft, or...) really want OODB to be perceived as a credible technology?...
  44. You seem to assume they don't already have one, which makes me think you really need to take a look at some Hibernate or Ruby on Rails code.
    Comparing Hibernate or Ruby on Rails (???) with an OODBMS is amusingly naive. Did you even *try* one before saying? There was no *real* alternative to RDBMS. There is an open source project called Ozone, wich seems dead, and it's API a bit intrusive.
    That's actually another thing that keeps happening: I have pretty much only ever seen OODB defended in public forums by employees of OODB companies.Doesn't really help make the technology more credible.

    You can count me as an OODB defender, and I don't work for an OODB company.I used Zope for a long time, wich embed an OODB (ZODB). And believe me, it was a joy. Not for the Python language, but for the persistence transparency that it provided. There was no O/R to worry about, no query to write, nothing. And the performance was spectacular and contant as the size of the BD increased.I didn't try db4o yet. I definitively will. If my experience with ZOBD is any indication of what the technology has to offer, I'm sold. Honestly, I only see two reasons for the slow adoption of OODBMS : Inertia, and RDBMS vendors FUD. Do Oracle or Microsoft, or...) really want OODB to be perceived as a credible technology?...

    I don't believe that statement is true. Since Sql99, the specification has included "create type" feature, so most of the modern Sql99 compliant database already provide OO like features. Looking at the various OLAP offerings out there, you'll see even more OO like features. If anything, the major RDBMS vendors are gradually adding more and more OO features.

    One problem with OODBMS from the past was the cost of altering the model. From my limited experience with LDAP, if the model changes, you have to reload the entire database. I don't know how Versant or db4objects handle model evolution, but that is one area where RDBMS has made it easier.

    peter
  45. Use native Java as the OO database API[ Go to top ]

    Since Sql99, the specification has included "create type" feature, so most of the modern Sql99 compliant database already provide OO like features.

    Storing an object as a blob in a table column has nothing to do with Object Database :))

    There's HUGE difference between true OODB and these "OO like features" of modern RDBMS.
    The latter is completely useless.
  46. Use native Java as the OO database API[ Go to top ]

    Since Sql99, the specification has included "create type" feature, so most of the modern Sql99 compliant database already provide OO like features.

    Storing an object as a blob in a table column has nothing to do with Object Database :))There's HUGE difference between true OODB and these "OO like features" of modern RDBMS.The latter is completely useless.

    Granted it's not the same as OODBMS, but it does make it easier to store and retrieve blobs. Of course, one limitation is blobs aren't indexed, but OODBMS do index objects. Given the trend since Sql99, I'm guessing SQL will continue to add OO like features. My point is, there's proof the major RDBMS products are slowly adding structured data like XML and object graphs. Some might consider the pace glacial, but it's better than nothing.

    Here is a page from Sun's jdbc documentation http://java.sun.com/j2se/1.4.2/docs/guide/jdbc/getstart/appendixB.html
    It states:
    A.4 Support for Advanced Data Types

    The JDBC 2.0 core API adds support for using advanced data types, making it as easy to use them as it is to use simple data types. This support includes the ability to store, retrieve, and update even the new SQL data types that are essentially objects, blurring the distinction between object databases and relational databases. The next four sections ("What Are the SQL99 Data Types?" on page 145, "Summary of Support for the SQL99 Data Types" on page 146, "Mapping of the SQL99 Types" on page 148, and "SQL Locators" on page 149) describe how the JDBC 2.0 core API provides support for these advanced data types.

    In addition to being able to store objects defined in SQL as values in a database table, programmers writing Java applications can also store objects defined in the Java programming language as values in a database table. The section "Support for Storing Java Objects" on page 149 describes this capability.

    Note that a driver is not required to implement functionality that its DBMS does not support, so not all drivers necessarily implement the functionality described here. DatabaseMetaData methods such as getTypeInfo, getColumns, and getUDTs may be called to get information about which data types a driver supports.

    I disagree that "it's completely useless." Probably "mostly useless" or maybe "misunderstood". Not that I know anything, but it's food for thought.

    peter
  47. Use native Java as the OO database API[ Go to top ]

    I disagree that "it's completely useless." Probably "mostly useless" or maybe "misunderstood"
    ..peter

    Object capabilities of RDBMS could be considered not an object storage, but an object graveyard.

    Once you stored your object, how do you gonna find it ? Using another, non object column and sql language ?
    After you found your object, how do you gonna find other, related objects ? Using other non object columns and relational sql ?
    This defeats the purpose of storing objects into database altogether.
    Why would you even bother to store your objects in this case ? What advantage will it give you ?

    Compare it to true OODB, where you can search and retrieve objects in fashion similar to sql language.
    Retrieveing master-detail related objects is transparent. You do not even do any specific action for that. They are just there.

    Besides object capabilites were in Oracle for ages.
    Have you ever seen a single project where they been used ?
  48. One problem with OODBMS from the past was the cost of altering the model. From my limited experience with LDAP, if the model changes, you have to reload the entire database.

    That has been absolutely true - before db4o:

    db4o has automatic schema evolution. You just change your object model and that's it. The database engine adapts the persisted model automatically. No pre-compiler, no post-compiler, no enhancer. Ideal if you have to deploy updates in distributed, embedded DBMS environments.

    I think automatic schema evolution is the single most differentiating feature of db4o's native Java object database engine.
    "The database has been nothing short of amazing. Just the ability to modify the structure and it automatically updates the versions is worth its weight in gold."

    ... and I better quote a customer, so I am not attacked on basis of me being an employee of db4o but at least on grounds of arguments.

    Christof
    www.db4o.com - Objects are here to stay!
  49. One problem with OODBMS from the past was the cost of altering the model. From my limited experience with LDAP, if the model changes, you have to reload the entire database.
    That has been absolutely true - before db4o

    This is false. Many OODBMSes have long supported easy transformation of models in on-line databases. A quick Google search will reveal many reports and papers on this subject.
  50. Honestly, I only see two reasons for the slow adoption of OODBMS : Inertia, and RDBMS vendors FUD.
    You only see *two* reasons? And of all the possible reasons, you pick these two? Seriously?

    I guess that explains a lot.

    <shaking head>

    --
    Cedric
  51. Honestly, I only see two reasons for the slow adoption of OODBMS : Inertia, and RDBMS vendors FUD.
    You only see *two* reasons? And of all the possible reasons, you pick these two? Seriously?I guess that explains a lot.<shaking head>-- Cedric

    Interestingly, you don't provide any of these good and *still valids* reasons yourself... Inertia?...

    <shaking head>
  52. There is an open source project called Ozone, wich seems dead, and it's API a bit intrusive.

    Ozone is not dead in any way. We have a large user community and several active developers. Ozone is used and has been used in production systems for many years. The API is simple and powerful although maybe a bit intrusive compared to an IoC based solution. Work is underway to to use IoC/AOP to make persistant objects plain POJOs where persistance is transparent (unlike Hibernate, JDO etc. where persistance needs to be explicitly addressed).
    That's actually another thing that keeps happening: I have pretty much only ever seen OODB defended in public forums by employees of OODB companies.Doesn't really help make the technology more credible.

    Ozone is an open, collaborative project where all currently active developers contribute on their spare time. As this time is very valuable we spend it on things that are fun. Defending OODB technology in open forums is apparantly not someting that gets prioritized. Comercially oriented OODB companies (or people that works for them) seem to prioritize differently. I do not think it is fair to judge OODB credibility in general based on who is participation in debates in public forums.

    Best regards,
    Per Nyfelt
    Ozone developer
  53. OODB design is for short and long term[ Go to top ]

    Companies that use RDBMSs don't want to migrate to ODBMSs because it's hard and expensive to move data from old relational DB design to OODB design.

    Let's take the example of employees working in different departments of a company. You have to define three tables: An Employee table, a Department table and a Department_Employee table with forein keys. Whereas in OODB design, you have to define the two classes: Employe and Department, and make the links between objects via references.

    The two designs are quite different. It may be easy to take your data from the three preceding tables an put them into the two classes. But imagine that you have trees and networks with thousands of related tables in your old RDBMS and try to solve that!

    That's the problem with migrating to ODBMSs. But if you have a new company with no big database yet, I think the best thing to do is to work with a good ODBMS.

    With ODBMSs You have not to worry a lot about the language you are using today to access your DB. You may use Java, .Net...etc, but the OO design of your DB will always be there. If you want to work with a different programming language/platform, all your challenge will be to adapt your old primitive data types, without altering the main design structure.

    And even if a new design may replace the OO one, say after a decade or two, it will be easier to migrate from OO design than from RDB design. We have to think about the future, when the difference between RDBMSs and future DBMSs will be as significant as between Java Platform and assembler languages.
  54. OODBMSs[ Go to top ]

    That's actually another thing that keeps happening: I have pretty much only ever seen OODB defended in public forums by employees of OODB companies.Doesn't really help make the technology more credible.-- Cedric
    I am neither an OODB company employee nor even an OODB customer but I have worked with pure OO databases, object-relational and even OO/deductive databases. Some of that was 10 to 12 years ago when Versant was a startup and Ontos was the dominant OODB company (Object Design later took the first spot). In the old days, OODB companies make the tragic mistake of trying to overtake RDBMS. To cut a long story short, that didn't work out! However, the few remaining OODB companies have found some profitable niches where their technology is the only one that can meet the customer requirements. If you check out their customer references and case studies and get in touch with some actual users, I think you will find out that their customers are not idiots but actually hit the wall with RDBMSs and had to consider something else. You will see that none of these applications are in mainstream business data processing, where RDBMS are the best tool for the job for obvious reasons. However I would encourage you to do a little research about the niche markets where OODBs are the right tool for the job and I think you will then agree that this technology makes sense for some usage scenarios.
    As the OODB companies are relatively small and on a challenging market, they have to look for new niches of profitability and I think that explains why e.g. Versant acquired a JDO company and is now interested in the EJB3 market. It's a natural extension of their business.

    Cheers,

    Alain
  55. I agree that there may well be a demand for this. We must not however blow it out of proportion. I've read through the documentation, and it would be great for persisting objects to a file (A single file is what the storage seems to be, please correct me if Im wrong).

    This is fine for a PDA, where a single user has an app that writes to a file. It might be great in examples where a city or country map may be stored in a database. I would not however feel comfortable with a medium to large size database where you have multiple departments connecting to multiple applications using the same database.

    As an example, if I have a customer databse, it might be used by the order application, the accounts application, the customer service appliaction, the sales and marketing application. With the data possibly being replicated across separate databases (You dont want all the data in a single database, to keep the different applications loosely coupled).

    Stored procedures were queried here... I think the way you'd do it is set up a server with an embedded database and custom java code listening over RMI or a session bean.

    If I were to want to use it however, it would probably be for a single user, single app, where the object model needs to be persisted to a single file.

    Maybe there is something I dont see... convince me otherwise...
  56. String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*....And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.

    Just to clarify, Versant has always had a fully indexible server based query execution engine to compliment it's object data management capabilities. And.... we support string-based queries in an SQL type syntax.....Not necessarily because we need them, but certainly because we listen. People want to be able to use standard reporting tools. Now we also support JDOQL and EJBQL, concievably standards to be used in a next generation of reporting tools.

    Right tool for the right job.

    Cheers,

    Robert Greene
    Versant Corporation
    www.versant.com
  57. Use native Java as the OO database API[ Go to top ]

    Now we also support JDOQL and EJBQL, concievably standards to be used in a next generation of reporting tools.Right tool for the right job.Cheers,Robert Greene Versant Corporation www.versant.com

    This is all very well, but this 'right tool for the right job' attitude seems to be contradicted by differing statements on links from the Versant website.

    At this URL:
    http://www.eclipse.org/jsr220orm/
    "At this time, resources working on this project are completely occupied on the JSR220 implementation and the project team can see no reasonable timeframe with which specific work will be done for the JSR243 standard."

    You state you are only going to provide support for EJB 3.0 (and, therefore, EJBQL).

    However, at this URL:
    http://www.versant.com/opensource
    "The JSR220-ORM Project's goal is to provide a 100% compliant implementation of the JSR 220 persistence and JSR 243 specifications integrated with the Eclipse platform."

    You state you are going to provide support for both JDOQL and EJBQL.

    I have already started to move one project to an alternative vendor because of this uncertainty.

    This would seem to be a good opportunity to clear up any confusion about Versant's support for JDO 2.0 and EJB 3.0.
  58. Use native Java as the OO database API[ Go to top ]

    Yes, there has been some confusing messaging around this issue. It has been really confusing because when we fist established JSR220-ORM Eclipse project, it was supposed to be both Tooling and API, but after accepting the project proposal for both, Eclipse foundation changed it's mind and made us remove the API. So, a lot of the original messaging for the Eclipse project does not make any sense. I guess the first point of clarity is that the Eclipse project is just a tooling initiative and a seperate initiative within Versant for the open source community.

    Future for Versant - it is an issue of resources, expectations of future demand by the development community. It is further and issue of tooling support and API support.

    Resources are limited.

    Expectations are that JDO will have demand near to medium term and EJB will have demand medium to long term ( though we also expect that longer term the current EJB spec will look even more like JDO as we've already seen this progression ). Remember, as stated by Sun, in the end, these specs are "merging" nothing is being abandoned.

    Tooling: We already have support for JDO tooling in our workbench. Due to resource constraints and expectations on demand, we will continue with tooling for JDO in that form only. The Eclipse platform will be the tooling for our EJB support, we will not incorporate EJB tooling support in our workbench.

    API: We already have JDO support and preview support of EJB3. We are incorporating these API's into a complete product packaging for the Versant database. We will support both of these API's in the Versant object database moving forward. Expectations are as stated above.

    Hope this helps,

    Cheers,

    Robert Greene
    Versant Corporation
    www.versant.com
  59. I am taught SQL in my local universities. That was the language that I knew and used at work after graduation. I couldn't say that I loved it in particular. This was the only query language I knew to query a DB. I got to use it and get the job done or die.

    Then I found and converted to Hibernate.

    Then I found and converted to db4o.

    I am now working for db4o.

    You said "programmers *like* string-based SQL queries" and "using ORM layers allow you to manipulate string-based queries and still use OO techniques to move your objects around".

    I am not sure how many people, programmers were you referring to. Are you sure that they really love "string-based SQL queries", or are they forced to use SQL because they don't have a choice?

    "O/R mapping approach ... used by 99% of enterprise applications", you said (in other thread). ORM is not pure string-based queries based. The query interface can be something else, such as Hibernate Criteria API .

    http://www.javalobby.org/articles/hibernatequery102/?source=archives

    ORM gives the user a choice, and 99% enterprise applications use ORM instead of pure SQL. Are you sure that all ORM (99% enterprise applications share) users are in love with string-based queries? At least the above javalobby author is not in love with string-based queries.

    By the way………….

    I was a user of Yahoo.

    Then I found and converted to Google.

    Similarly, I was a user of Yahoo! Mail.

    Then I found and converted to GMail.

    Similarly, I was a user of JUnit.

    Then I found and converted to TestNG. Actually, I was recommending TestNG to my colleagues in a meeting yesterday.

    Why did I convert to TestNG? Simply because TestNG fixes all the problems of JUnit. Why do i know that?

    Because i tried TestNG myself. I spent time using it. If I have never spent time trying out TestNG, how can i know it is better than Junit?

    Just spend some times trying out new ideas.

    Revolutions always start with humility, not arrogance.
    No string based SQL queries...
    This idea really needs to die.Here is the scoop: people *like* string-based SQL queries.String-based queries might be error prone, untyped, hard to refactor and have bad breath, but programmers *like them*.Why?Because their defects are outweighed by their advantages.And as long as OODB people keep ignoring this, their market share will keep plateauing under 1%.
    Believe it or not, its a great relief for object purists out there...
    Well, not to me. And I'm an object purist.First of all, using ORM layers allow you to manipulate string-based queries and still use OO techniques to move your objects around.Second, the OODB approach is hardly more object-oriented. You are invoking methods on chains of objects... big deal. More typed? Yes. More object-oriented? Not even close.Look, I have nothing against OODB or Query API's. Really.But the attitude shown by advocates of these technologies is driving me (and a lot of people) nuts. Be nicer, pay attention to what users complain about instead of making up pain points and maybe they'll start listening to you.-- Cedric
  60. object purist?[ Go to top ]

    Believe it or not, its a great relief for object purists out there...
    Well, not to me. And I'm an object purist.

    By which or whose definition I wonder...
  61. Standard[ Go to top ]

    You still need some standard, you are just moving the problem from requiring a standard language (SQL) to a standard API (the OODB API in Java)
    The elegance of the approach of Native Queries:
    The programming language is the standard. Native Queries only provide a very light wrapper around expressions in the programming language. This works for any object-oriented language. Here is a sample in C#, where delegates reduce the typing work (which the IDE can do for you anyway) even further:

    IList products = db.Query (delegate(Product product){
        return product.Price < 100
            && product.Vendor.Name.StartsWith("d")
            && product.Vendor.Country == Countries.USA;
    });
    How is your solution going to work for the next language after Java?

    Very likely it is going to work very well. The programming language will define the syntax.
    How was it working when C++ was popular

    It *is* working very well. You can write native queries in managed C++ and work with db4o.
    SQL was working across these three paradigm shifts.

    db4o works across paradigm shifts. You can access the same db4o database both from Java and from .NET. We use the technology ourselves. Our object browser to browse .NET databases is written in Java.
    History has shown us that paying the impedance mismatch price close to the language (SQL + O/R mapper to your programming language) is the only viable solution in a world where data outlasts programming languages by an order of magnitude in years.
    You argue from an enterprise data-warehouse perspective and I don't disagree that SQL will continue to be the right choice for very heterogeneous huge systems where umpteen different applications in umpteen different languages need a common repository.

    db4o is built for devices, for embedded use, for packaged software and for realtime industry applications.
  62. Standard[ Go to top ]

    db4o is built for devices, for embedded use, for packaged software and for realtime industry applications.
    It looks like there are no joins, aggregation functions, projection, .... . It see no problems to use plain B+tree for this kind of applicatins, but I do not need a database to use B+tree.
    BTW there are "easy-of-use" SQL libraries for "small" applications too, SQLite is a good one.
  63. Versant participated in the pushs to establish JDO 1 (disbanded).
    When exactly was "JDO 1" disbanded ? You're presumably referring to JDOQL here since the context is query languages. In JDO 1 this was a simplified form of Java syntax. In JDO 2 this has not been disbanded, but has been extended significantly.
  64. Versant participated in the pushs to establish JDO 1 (disbanded).
    When exactly was "JDO 1" disbanded ? You're presumably referring to JDOQL here since the context is query languages. In JDO 1 this was a simplified form of Java syntax. In JDO 2 this has not been disbanded, but has been extended significantly.

    This may be a reference to Versant cutting back on plans for JDO support in future versions of OpenAccess - they are concentrating on EJB 3. However, this does certainly does not mean that JDO is disbanded, as other vendors are continuing with strong support for JDO - indeed they are emphasising potential combined use of JDO 2.0 and EJB 3.0.
  65. Use native Java as the OO database API[ Go to top ]

    This may be a reference to Versant cutting back on plans for JDO support in future versions of OpenAccess - they are concentrating on EJB 3. However, this does certainly does not mean that JDO is disbanded, as other vendors are continuing with strong support for JDO - indeed they are emphasising potential combined use of JDO 2.0 and EJB 3.0.

    Just to clarify, We will continue to support JDO for the Versant database, we just do not have enough resources to build JDO support to the Eclipse ORM tooling project JSR220-ORM We already have visual support for JDO development in our own workbench. The Eclipse project will focus on JSR220 tooling.

    JDO is CLEARLY not "disbanded". Lack of awareness for that point raises questions about the integrity of other statements made by the author choosing those words.


    Robert Greene
    Versant Corporation
    www.versant.com
  66. Use native Java as the OO database API[ Go to top ]

    Just to clarify, We will continue to support JDO for the Versant database, we just do not have enough resources to build JDO support to the Eclipse ORM tooling project

    Apologies. I posted a request for clarification without having read this. However, I still think you could make things clearer on your website.
  67. JDOQL *IS* Java syntax[ Go to top ]

    <NOT>
    Versant and other of the old-style ODBMS vendors have participated in the pushs to establish OQL (disbanded) and JDO 1 (disbanded).
    </NOT>

    I assume by "disbanded" you mean "available in dozens of products, now including BEA since they aquired Solarmateric, and enhanced in JDO 2.0".

    Yeah, the cool thing about JDO is the query language essentially IS java syntax. The BNF is basically a subset of java. True, it is still a text-based query that has to be compiled, but when I look at JDOQL, it sure looks clean to me. In fact, it looks like .....oh ma god ... JAVA.

    customers.contains(cust) && cust.lastname=="smith"

    I mean, it's pretty intuitive.

    -geoff
  68. Uses for Object Database[ Go to top ]

    Everything I have read seems to point towards an object database being slower than a relational database. However, one thing that occurs to me that would be really cool to do with an object database is configuration and object cacheing. Basically, an object database would be a really cool native way to store configuration objects and objects that would need to persist a crash of the application.

    Any thoughts?
  69. Uses for Object Database[ Go to top ]

    Everything I have read seems to point towards an object database being slower than a relational database.

    It depends on what you want to do with them. Object databases can be very fast indeed. CERN use them to store the phenomenal amounts of data from their experiments.
  70. Uses for Object Database[ Go to top ]

    Everything I have read seems to point towards an object database being slower than a relational database.
    It depends on what you want to do with them. Object databases can be very fast indeed. CERN use them to store the phenomenal amounts of data from their experiments.
    Are you sure they use Object databases ?
  71. It seems so...[ Go to top ]

    according to this
  72. The use is very specific[ Go to top ]

    according to this

    I don't work at CERN, but I thought their specific case is a highly tuned deployment. What they do have very specific needs that are quite different than normal business needs, so how they use it is not representative of OODBMS in general. It does provide great proof that OODBMS can perform at the highest levels. Of course, you have to know what you're doing and take the time to architect it correctly. NRC also uses object databases, but again their case is very specific. OODBMS are useful tools, but I could do without the hype.

    peter
  73. It seems so...[ Go to top ]

    "Practical Limitations of Objy" doe's not seem "the phenomenal amounts of data"
    http://wwwasd.web.cern.ch/wwwasd/cernlib/rd45/meetings/oct23_97/VLDB_requirements.html
  74. It seems so...[ Go to top ]

    It seems CERN migrated to RDBMS after OODBMS evaluation (google)
  75. Advanced Native Query Example[ Go to top ]

    It's interesting to see people write "SQL is easier" after a very first look at the concept of Native Queries. Are you sure that you already understand what you can do with them? How long did it take you to learn SQL ? Would you have dared to answer "SQL sucks" after reading briefly over a portal page?

    Give Native Queries a try, think about more complex examples, take advantage of object-orientation and being able to call Java methods from within a query.

    Here are two very simple features that you will find hard to implement in an SQL query:

    (1) Multiple exit points, using a guard-pattern in this case.
    (2) Local mutable variable in a query.
    List products = db.query(new Predicate (){
      public boolean match(Product product){
        if(product.getLicense() == Licenses.GPL){
          return true;
        }
        Float price = product.getPrice();
        Vendor vendor = product.getVendor();
        return price > 10
            && price < 100
            && vendor.getName().startsWith("d")
            && vendor.getCountry() == Countries.USA;
        }
    });
  76. Query question[ Go to top ]

    Just a quick question.
    The queries call methods such as product.getLicense() etc...
    These methods may be using any number of properties of the
    object (I am assuming the methods don't have restrictions on
    what they do).
    Would that not mean that you would need to load the whole
    of the object's state to perform a query?
  77. Optimization of Native Queries[ Go to top ]

    The queries call methods such as product.getLicense() etc...These methods may be using any number of properties of theobject (I am assuming the methods don't have restrictions onwhat they do).Would that not mean that you would need to load the wholeof the object's state to perform a query?

    Good question!

    Our Native Query optimizer analyzes bytecode, follows all method calls and attempts to construct a representation of the entire query with our Criteria Query API (Yes, we have one too.).

    Indeed methods in a Native Query expression have no restrictions on what the can do, but they should not modify persistent objects. Indeed there are limitations on what the optimizer can analyze.

    The development of the optimizer to produce maximum performance for all possible Java constructs will be an ongoing process. As an open source project I think we are in a good position for rapid improvements. We are in contact with a couple of ORM vendors that show interest to cooperate with us on the task.

    As a fallback, if the optimizer can't cope with all or parts of a Native Query expression, we do indeed instantiate the persistent objects to run the Native Query expression against the actual objects. This happens on the server, so objects do not have to be instantiated over the Client/Server connection. Only those objects that can not be ruled out by parts of the Native Query expression need to be instantiated.
  78. Re: Query Question[ Go to top ]

    The queries call methods such as product.getLicense() etc...These methods may be using any number of properties of theobject (I am assuming the methods don't have restrictions onwhat they do).Would that not mean that you would need to load the wholeof the object's state to perform a query?
    Good question!Our Native Query optimizer analyzes bytecode, follows all method calls and attempts to construct a representation of the entire query with our Criteria Query API (Yes, we have one too.).Indeed methods in a Native Query expression have no restrictions on what the can do, but they should not modify persistent objects. Indeed there are limitations on what the optimizer can analyze.The development of the optimizer to produce maximum performance for all possible Java constructs will be an ongoing process. As an open source project I think we are in a good position for rapid improvements. We are in contact with a couple of ORM vendors that show interest to cooperate with us on the task.As a fallback, if the optimizer can't cope with all or parts of a Native Query expression, we do indeed instantiate the persistent objects to run the Native Query expression against the actual objects. This happens on the server, so objects do not have to be instantiated over the Client/Server connection. Only those objects that can not be ruled out by parts of the Native Query expression need to be instantiated.

    Thanks for clarifying that. It sounds interesting.

    I have been using Hibernate lately. With it I have a programmatic Query API as well as an SQL like query language,
    with constructs for representing Objects.
    Sure we have the impedance mismatch, in that I am storing
    my data in a relational database but I am modeling it as
    objects. An OR-Mapping tool such as Hibernate tries alleviate this problem, by allowing me to express how this
    mismatch should be dealt with.

    In an OODB people say this impedance mismatch does not exist. But I believe a mismatch exists in OODBs even if it
    is not an 'impedance mismatch'.

    Taking your explanation of how Objects are analyzed
    so as to provide efficient queries. I assume the analyser
    will be a very sophisticated piece of code/algorithm.
    Now how do I as a programmer know what the implications of
    how I design and access my persistent objects will have on the specific heuristics imployed by query generators etc?

    Now, I don't have ANY experience with an OODB, so my
    points are probably invalid if so please set me straight.
    With an O/R tool I can express and handle the mismatch, in
    an OODB too many 'complex' things are happening behind
    the scenes.

    I suspect that for 90% of the time a good OODB will probably
    be quite efficient while at the same time removing the burden of having to explicitly do O/R mapping. But I also
    believe an O/R mapping or SQL approach will always be faster and more efficient. It then becomes a matter of weighting up
    the pros and cons.

    On a different note, is there an overlap between O/R mapping
    and OODBs? How would Hibernate differ from an OODBs if I
    had a tool that analyzed my code and created an appropriate
    mapping file.
  79. Advanced Native Query Example[ Go to top ]

    It's interesting to see people write "SQL is easier" after a very first look at the concept of Native Queries. Are you sure that you already understand what you can do with them? How long did it take you to learn SQL ? Would you have dared to answer "SQL sucks" after reading briefly over a portal page?Give Native Queries a try, think about more complex examples, take advantage of object-orientation and being able to call Java methods from within a query.Here are two very simple features that you will find hard to implement in an SQL query:(1) Multiple exit points, using a guard-pattern in this case.(2) Local mutable variable in a query.
    List products = db.query(new Predicate (){&nbsp;&nbsp;public boolean match(Product product){&nbsp;&nbsp;&nbsp;&nbsp;if(product.getLicense() == Licenses.GPL){&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return true;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;&nbsp;&nbsp;&nbsp;Float price = product.getPrice();&nbsp;&nbsp;&nbsp;&nbsp;Vendor vendor = product.getVendor();&nbsp;&nbsp;&nbsp;&nbsp;return price > 10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; price < 100&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; vendor.getName().startsWith("d")&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; vendor.getCountry() == Countries.USA; &nbsp;&nbsp;&nbsp;&nbsp;}});

    Since I'm the one that said "sql is easier", I'll state it again. It's a matter of preference and training. How long did it take for me to learn SQL syntax? probably a few weeks. How long did it take to learn the subtleties and low level stuff like execution plans? I'm still learning the differences between db2, oracle, sqlserver, mysql, and sybase. I won't speak for others, but the syntax does matter to me.

    I prefer Sql syntax and I also prefer LISP syntax over java. Not everyone thinks the same or prefers the same syntax. I've met plenty of business analysts who could read sql and understand what it meant without any professional programming experience. The same may not be true of Native Query. Regardless of what I think, there is a significant barrier to object oriented queries. If java and C syntax was easier, I would think the market would have proven that. Though I wouldn't consider that definitely proof. The success of SQL could be attributed to marketing and not necessarily easier syntax. Quite honestly, I can't make much sense of the example, but that's just me.

    peter
  80. Advanced Native Query Example[ Go to top ]

    It's not a problem to learn SQL. I personally know how to do some SQL tricks. However I also found that developing multidatabase system is difficult task. On other side you can generate pretty good queries also from pure object queries. At least better than average SQL programmer can do.
    With pure object queries you can recognise that somebody changed "your" class at compile time. All String encoded queries even OQL or HQL are simply wrong.
  81. Advanced Native Query Example[ Go to top ]

    We seem to mix two different discussions. First there is the discussion on OODBMS vs RDBMS. The second discussion is on query languages. Most people here seem to like OQL, SQL and HQL for its expressiveness. However a few things are lacking from these languages.

    For instance, when we are using ORM there's no static type-checking in the query. This means we cannot use our Java toolset to its full extent. Another big problem with those query languages is that there is no easy way to use abstraction, there are no variables or methods. Using a Java-like query language, methods can be used to hide complexity. In standard query languages all criterias must be "flattened" a hiding complexity is difficult.

    A major objection against using Java as a query-language is that there is a lot of tools to ease the use of SQL. All databases comes with query tools and optimizers.

    Another possibility, which I'm working on on my spare time, is to convert Java predicates (like the samples posted here) to some standard query language, e.g. SQL, HQL or OQL, at runtime. This enables you to use Java queries where it makes sense, and to revert to SQL or HQL when advanced queries (like aggregating queries) or bulk updates needs to be done.

    Magnus
  82. Advanced Native Query Example[ Go to top ]

    All databases comes with query tools and optimizers.
    That smells of vendor lock in. If I choose a freeware database (eg Postgres, Hypersonic), are the tools you mention missing? With an OR mapper (eg EJB3), the database vendor is pluggable. From what your saying, I infer that whenever I switch database vendor, I must relearn news tools, and some tools might not be available. This makes hand working SQL sound terrible.
    Another possibility, which I'm working on on my spare time, is to convert Java predicates (like the samples posted here) to some standard query language, e.g. SQL, HQL or OQL, at runtime.
    Or build time, since (ignoring reflection) it's a static exercise. With some tuning it might be possible for the SQL generator to craft more optimal queries than a novice or intermediate SQL programer. The SQL generator could also emit DDL and declare indices chosen by static analysis. The SQL generator could also automate caching, do field interception, use annotations, etc. I assume EJB3 already specifies these kinds of features. I suppose safe queries could be easily layered atop EJB3, and I guess it's a winning combination.
  83. Indices[ Go to top ]

    I like this but...

    Does db4o use indices? It seems the query would have to load the entire object collection into memory to perform the search if it doesn't.

    What if the objects are huge? What if there are many of them?
  84. Indices[ Go to top ]

    I like this but...Does db4o use indices? It seems the query would have to load the entire object collection into memory to perform the search if it doesn't.What if the objects are huge? What if there are many of them?

    Doh && Sorry, I should have paid more attention to the original post!
  85. Native queries suffer from horrible java syntax.
    Unfortunately java language does not have adequate semantics to work with sets, lists etc.
    It does not have closures.
    And anonymous classes are a sick joke over programmers.

    Things are much better on the dotnet side.
    Especially now with just released dotnet 2 and C# 2 with Generics, Nullable types, closures etc.
    And upcoming Dlinq makes native querying even easier in terms of suntax.

    Java is definitely not the best language for OODB.
    As someone said here - it is not LISP.
  86. There are so easly make on RDBMS alter, drop on the big data when change the Business Rules.

    What happends ODBMS on the pyshical database , can someone explain or can send a documents or a link.

    Thanks,
    Someone interesting with ODBMS
  87. What happens when change Object MetaData[ Go to top ]

    There are so easly make on RDBMS alter, drop on the big data when change the Business Rules.What happends ODBMS on the pyshical database , can someone explain or can send a documents or a link.Thanks,Someone interesting with ODBMS

    I think its time we started writing our posts in SQL !! :-))

    j/k
  88. You can find a segment on schema evolution in various object databases in the following overview document on how to select and object database.

    ftp.versant.com/pub/outgoing/howto/HowToEvaluate_Evaluate.pdf

    Cheers,

    Robert Greene
    Versant Corporation
    www.versant.com
  89. http://research.microsoft.com/~gray/papers/SDSSArchive_OO_to_SQL.doc
    If somebody wants to evaluate OODBMS then this document must be usefull too.
  90. Great paper ...... I had not seen it before.

    Versant doesn't really come to competition with Objectivity database very often, which is what this paper is about, the company Objectivity's OODB and thier particular OODB architecture, as I said before, some are very very fast and some are dogs. However, if we do come to competition in the future, this will serve to compliment the other paper well.....thanks!

    In Objectivity's case, no server side query execution engine, which was always one of their big weakness's, seems the be the primary reason for the move. In objectivity, you have to ship the objects to the client to process the query.....not a very scalable solution.

    Thanks for the reference.

    Cheers,

    Robert Greene
    Versant Corporation
    www.versant.com
  91. It is not a strong argument, it is possible to find this kind of paper about any tool (as more popular tool as more negative information/misinformation). This stuff is usefull for FUD on TSS only.
  92. Perspective on Native Queries[ Go to top ]

    Hi, I am the author of the research paper "Safe Query Objects: Statically Typed Objects as Remotely Executable Queries" published this summer, which is the basis for db4objects Native Query implementation. This work was based in part on my experience developing commercial enterprise software in the Partner Relationship Management space (I was software architect at Allegis Corp, now Click Commerce).

    POJQ is another implementation by Wes Biggs. It generates JDOQL to allow use of native queries with JDO. It is open source, and available and working now. See https://pojq.dev.java.net/

    I want to clarify some points about native queries:

    * Native Queries are not specifically tied to OODBs. They work just fine as a query language for object-relational mappers. My original prototype generated queries for JDO, HibernateQL, and straight SQL.

    * SQL is a great language. The problem is the *interface* between an OO language and SQL. Putting *any query language* in strings leads to a semantic disconnect between the host language and the query languages. It interferes with syntax checking, type checking, and makes parameters and conditional/dynamic queries very messy and error prone. People do it because its the only way to get good performance, and it certainly works. But it is not a good language design.

    * Queries have many different facets. While these are rolled together in SQL, they tend to be split out in object-based query languages like JDOQL and HibernateQL. The two main facets I'm talking about are
    - finding objects of interest
       (the WHERE clause in SQL)
    - specifying what objects and related objects to load
       (the SELECT clause in SQL)
    The JOIN clause is used to create relationships, which is analogous to navigation in an object view of data. My point is that JDOQL only handles the first part, finding objects. HibernateQL has a notion of prefetch profiles, and some JDO vendors have proprietary ways to specify prefetch. I'm working on ways to improve the prefetch part of a query... more on this later.

    * About impedance mismatch. I think it is a combination of two problems:
    - The difficulty of modular, encapsulated, OO style programs to effectively leverage high-performance database techniques like query optimization, indexes, etc. That is, the inability of OO programs to move from navigational/one-at-a-time processing to optimization based on overall query plans. But I do think that we will eventually solve impedance mismatch, by allowing OO languages to seamlessly move between the worlds.
    - The common techniques to overcome this problem, using string based query languages, introduce the semantic disconnects I mentioned above.

    Native queries are a way to get the best of both worlds. The point is that you write them as Java, but they execute as an optimized database query. They are very similar to Microsoft's LINQ project, but don't require language changes.

    Some other references people might find interesting:

    * Programming Languages & Databases: What’s the Problem?
    William R. Cook and Ali H. Ibrahim:
    http://www.cs.utexas.edu/users/wcook/Drafts/2005/PLDBProblem.pdf

    * Safe Query Objects: Statically Typed Objects as Remotely Executable Queries
    William R. Cook and Siddhartha Rai, ICSE 2005
    http://www.cs.utexas.edu/users/wcook/papers/SafeQuery/SafeQueryFinal.pdf

    * POJQ project: https://pojq.dev.java.net/

    William Cook, PhD
    Department of Computer Sciences, The University of Texas at Austin
    http://www.cs.utexas.edu/~wcook
  93. thanks for the links[ Go to top ]

    I think that is a fair summary, but whether a language is theoritically/mathematically "better" is one thing. Whether a language is good from a human perspective is also an important issue. I think there's a lot of value in using both techniques, and understanding both models of development.

    peter
  94. Perspective on Native Queries[ Go to top ]

    Thanks for the summary and references. Even though I am a big fan of SQL, I am looking for a Java based Query language so I express SQL via Java APIs, especially for Search API exposed to client.

    Thanks A lot.


    Chester
  95. A Better Query Syntax[ Go to top ]

    Employees emps = new Employees();
    emps.Where.LastName.Value = "%A%";
    emps.Where.LastName.Operator = WhereParameter.Operand.Like;
    emps.Query.Load();

    And a more complicated one ...
    Employees emps = new Employees();

    // LastNames that have "A" anywher in them
    emps.Where.LastName.Value = "%A%";
    emps.Where.LastName.Operator = WhereParameter.Operand.Like;

    // Only return the EmployeeID and LastName
    emps.Query.AddResultColumn(Employees.ColumnNames.EmployeeID);
    emps.Query.AddResultColumn(Employees.ColumnNames.LastName);

    // Order by LastName
    // (you can add as many order by columns as you like by repeatedly calling this)
    emps.Query.AddOrderBy(Employees.ColumnNames.LastName, WhereParameter.Dir.ASC);

    // Bring back only distinct rows
    emps.Query.Distinct = true;

    // Bring back the top 10 rows
    emps.Query.Top = 10;

    emps.Query.Load();

    See http://www.mygenerationsoftware.com/portal/dOOdads/CUsage/tabid/53/Default.aspx