IBM announces Derby: Open sourcing Cloudscape database

Discussions

News: IBM announces Derby: Open sourcing Cloudscape database

  1. Raising its stakes in open-source software, IBM plans to create an open-source project around Cloudscape, a specialized Java database. The open source project, code-named Derby is said to be released into the Apache Software Foundation. IBM acquired Cloudscape when it bought Informix in 2001.
    Cloudscape is a niche product in IBM's overall data information line and has tiny market share compared with its multibillion-dollar DB2 franchise. IBM has used Cloudscape as an embedded data store as part of its Workplace desktop application line.

    As a Java-only database, it does not compete directly against mainstream relational database servers, such as DB2, Oracle or Microsoft's SQL Server, according to industry executives.

    Still, the move to make its database products open source deepens IBM's commitment to the open-source development model. With its multibillion-dollar investment in Linux, Big Blue is credited with having made open-source technology more palatable to corporate customers.
    IBM to make Java database open source

    Some people have noted that they wish that CloudSync was still around and open sourced, as it has something different compared to the other db's such as HSQL and McKoi

    Threaded Messages (52)

  2. Nice!
        This will be a great addition to open-source software and I'm sure will be a very useful addition to a lot of projects.

    I've not used Cloudscape for a while, it used to be embedded in a few products, anyone know if JDO/Hibernate works with Cloudscape? I can't find anything on the Apache site either am I just too early?

    -John-
  3. How is Cloudscape different from HSQL?
    Yes, I have noticed Cloudscape’s support for user data types, triggers and such, but if I need such features from database I personally would trust better to a well time-tested non-java RDBMS like Postgres. Developing extensive pure Java RDBMS seems like wrong direction for me, like James (java mail server).
  4. Developing extensive pure Java RDBMS seems like wrong direction for me, like James (java mail server).
    I am sure it is the right direction, I am not sure, but I think Cloudscape must be
    something like the real database with concurrency control implementation.
    BTW it supports optimistic concurrency control, we will not need any caches for web applications with this kind of database.
  5. Developing extensive pure Java RDBMS seems like wrong direction for me, like James (java mail server).
    Konstantin,
    I was just wondering if you have had bad experience with James mail server? Does it not scale well? Thanks.
  6. No, I do not have negative experience with James, nor I would like to acquire any experience with it.
    I have ideological issues with projects like James: there are solutions like Sendmail and Qmail, which are guaranteed to work and scale well, so I do not see any reasons for using/developing James. My stand is this: rewriting everything with using Java (or Perl, or Python, etc.) is wrong, technologies should play well together and do not reinvent wheel without a good reason. Creating a full-featured RDBMS for Java, mail server and other such things promote a notion of Java as OS, which is not good IMO.
  7. My stand is this: rewriting everything with using Java (or Perl, or Python, etc.) is wrong, technologies should play well together and do not reinvent wheel without a good reason.
    Perhaps there is a good reason?

    When I need to manage many deployments of an application, there is a significant advantage to being able to install, deploy, configure, and monitor my database through the same interface that I use for all of my other application components. That's what I get out of a pure Java db that I can run inside my app server. If my application does not require the power of a native database, this tradeoff works very well for me.
    such things promote a notion of Java as OS, which is not good IMO.
    I have trouble characterizing such things as good or evil. Meeting application requirements are more important to me than adhering to philosophies.
  8. Meeting application requirements are more important to me than adhering to philosophies.
    One does not exclude another, there are many ways of “meeting requirements” and one radical way is “not meeting them”. That is probably topic for another thread, but let me elaborate a bit on this: I firmly believe that a project long-term success is impossible without adhering to _some_ philosophy, however blind following to the requirements might have short term benefits and definitely helps paying the bills.
    In my experience projects went exceptionally well when requirements were not considered being carved in stone and project ultimate goal was to satisfy client needs rather than “requirements”.
    If my application does not require the power of a native database, this tradeoff works very well for me.
    It is actually exactly what I tried to say: it is OK to have simply and lightweight embeddable RDBMS ( HSQL is a good example. It is quite SQL compliant and works pretty well as an embedded DB). My point is that if we start using triggers, extensive transaction control etc., it probably means that we need full power of RDBMS and it is better to use a ‘native’ one. And therefore I consider Cloudscape features as a “scope creep”.
  9. Why do you think ‘native’ is better ?
    And I see no meaning to compare HSQL with Cloudscape, Cloudscape is as good as any 'native' RDBMS and It can better in some cases (probably in the most of cases).
  10. Why do you think ‘native’ is better ?
    They are time proven and not limited to Java, so various reporting and other tools could be used (MS Access for example).
     
    Cloudscape is as good as any 'native' RDBMS and It can better in some cases (probably in the most of cases).
    Is there an example of Cloudscape holding a terabyte database?
  11. Is there an example of Cloudscape holding a terabyte database?
    Quote from Cloudscape forum:
    Hi Greg,

    Regarding Cloudscape, the current version, 5.1.60, has no limitations on the
    size of the database. However, there is a size limit for a table or index
    that is only imposed by the operating system.

    For the number of records in a single database this is not restricted - the
    size of a row can span pages and the number of rows per table is not limited.

    Going forward, future versions of Cloudscape are not planned to restrict the
    size of the database, however certain database objects that may now be
    unlimited in size may change in the future.

    Regards,

    Susan
    IBM Cloudscape
    http://63.209.20.13/cgi-bin/www2/cloudscapeforum.d2w/view?SID=20040803084618306001&NRL=2
  12. full power RDBMS?[ Go to top ]

    My point is that if we start using triggers, extensive transaction control etc., it probably means that we need full power of RDBMS and it is better to use a ‘native’ one. And therefore I consider Cloudscape features as a “scope creep”.
    I dont agree with you. My company ships our product with bundled HSQL database for evaluation purposes. If you just want to play with our registry, you don't need to install Oracle, DB2 etc. Just select Hypersonic as database engine.

    And for me as developer is really important that its expression capabilities are similar to real RDBMS. We try to maintain same SQL commands for all RDBMS. So my message to HSQL and Cloudscape developers is: go on! Do not concentrate on speed but on power.
  13. James is a good idea too, but JAVA database is more interesting. As I understand
    it will be very trivial to write and debug stuff like stored procedures, integrate security, cache, concurrency control. A lot of applications can use web server and RDBMS in the same process and the same thread pool, it must be performant and to scale without app level data caches.
    This kind of databases are usefull for desktop applications, but most of web developers will drop db2 if Cloudscape will be free and better, web services become more popular for remore clients any way.
  14. Maybe beside the point, but James is a quite diffrent application than SendMail and QMail. It is not only a mail server, but it is to Mail Application what a Java Web Container is to a Web Application. It is a Mail Application Server, aimed at providing an environment for developing and executing mail based applications.

    As for RDBMS in Java there are a many advantages, the biggest I can see is embedding inside an Java application, where a big and full-blown RDBMS is not needed.
  15. Daffodil Software has launched a free version of their Java database Daffodil DB.

    Go on, download One$DB
  16. when i try to register to download (which is required), i get:

    java.sql.SQLException: Problem in Execution of Insert Statement for Table 'users.users.GroupMembers'. Violation of UNIQUE KEY constraint 'UK2' ... etc

    frustrating.
  17. http://www.daffodildb.com/onedollardb-10goodreasons.html#7
    One$DB adheres to SQL 99, JDBC 3.0 and J2EE standards
    SQL 99 – understand;
    JDBC 3.0 – isn’t it about JDBC driver? Well, I might guess that you support all the features in the JDBC3;
    J2EE – oops, that does not make any sense for me, what are you talking about here?
  18. Cloudscape is, as you put it, a “well-time-tested” production database. It has been around a long time and has been used successfully in a large number of production environment that require a zero administration database engine. The fact that it is written is Java is a good thing, because it makes for easy embedding in java applications.

    Cloudscape implements much more of SQL standars than HSQL, and sports very sophisticated replication and synchronization technologies that are important for occasionally connected applications. The combination of high performance, small footprint, and high end database features, delivered in a small (2MB) footprint, make for a number of interesting application development possibilities.

    I have used HSQL for development, and it is great for some applications, but cloudscape goes well beyond what is possible with HSQL.
  19. I can give a couple of reason[ Go to top ]

    How is Cloudscape different from HSQL? Yes, I have noticed Cloudscape’s support for user data types, triggers and such, but if I need such features from database I personally would trust better to a well time-tested non-java RDBMS like Postgres. Developing extensive pure Java RDBMS seems like wrong direction for me, like James (java mail server).
    I agree blind porting of everything to Java isn't necessarily good, but having worked on portal projects, I can think of plenty of reason why something like James is desirable and potentially better. Let's say I want to be able to provide email and calendaring for PC, Palm and WAP phones. If I try to integrate with Sendmail or QMail, chances are it's going to be tough. The first thing is I may have to save the mail in some other format to accomodate the additional information I need. If I want to automatically send out an email with calendar changes and then update the user's calendar from the response email, I may want to route those emails to a specific process. The latest sendmail supports WAP, IMAP and POP, but it may not necessarily integrate well into an advanced portal application. Ideally, I want to be able to search my emails and categorize them by the size, originating device, from, to, subject and body content. None of those features would be easy to integrate using sendmail or qmail. On the otherhand, if I use James and store the data in the database, i can use either java or stored procedures to perform additional processing.

    For these types of applications, doing a native call out to sendmail or qmail may not be simple. It would especially be true if some actions are triggered by inbound mail. At that point, you would have to extend sendmail, beyond its design and impact scalability.

    sendmail is solid and reliable because it does a very specific thing and does it well. integrating it into a complex app sometimes do require writing a java version. that's first hand experience trying to integrate an email application based on sendmail, which scaled terribly.
  20. hi there

    and yes, we used for prototyping purposes with Hibernate 2.0.3 and it worked well.
    we needed to do some tweaking re. the data type for some of the columns, but other than that, it worked well.
  21. JDO and Cloudscape[ Go to top ]

    Hi

    At least LiDo is a JDO implementation with a specific support for Cloudscape (versions 4 and 5). Probably other JDO implementations support it too.

    Best Regards, Eric.
  22. Re: JDO and Cloudscape[ Go to top ]

    Yes, but which version of JDO - Sun or Hibernate.
    I prefer Hibernate, because Sun is not perespective.
  23. Nice!    This will be a great addition to open-source software and I'm sure will be a very useful addition to a lot of projects.I've not used Cloudscape for a while, it used to be embedded in a few products, anyone know if JDO/Hibernate works with Cloudscape? I can't find anything on the Apache site either am I just too early?-John-
    For SolarMetric's implementation of JDO, Cloudscape is one of the supported databases
  24. help me if u can[ Go to top ]

    Hi guys,
    I am in the process of finalising a Database for my Java Application.I had narrowed down to 2 choices which were Cloudscape and a Java based RDBMS by the name of DaffodilDB .While DaffodilDB is a very feature rich and competitively priced product in comparison to the others but Cloudscape scores high on Brand image.
    I am unable to take a decision,more so,after IBM`s decision of making Cloudscape open source.I could use some suggestions in this regard.
  25. help me if u can[ Go to top ]

    I'm disappointed this topic received so little attention.

    The database engine is the single element in your product
    that must not ever fail. You can't afford to write the wrong
    database blocks when modifying data. With everything else
    you can take chances as you can restart.

    The problem with database products is that you can't really evaluate
    the reliability. You would have to perform millions of transactions
    using different database objects in all combinations and test all
    extrem situations as RDBMS abort. This is why I would go with the
    database that has the most users, there are more chances the problems
    were already discovered.
  26. help me if u can[ Go to top ]

    Daffodil DB is not only a feature rich RDBMS but it also have a large user base.

    Daffodil DB is establishing itself as a brand in the community, ofcource at this time you can't compare it with brands like IBM or oracle.

    www.daffodildb.com
  27. my 2 cents[ Go to top ]

    Hi every1,

    I am surprised that the IBM / Cloudscape movement (read OS) has recd. scant attention from the 350,000+ members of TSS. Surely this cannot reflect the interest of the community with regards to embedded databases.

    Personally, I am of the opinion that embedded databases will emerge to hold their own against the 'big boys'. IMHO, the Cloudscape news is the best I have heard in a long long time. Whatever IBM's 'real' motive is, the bottom line is that this will lead to a (much reqd. and long overdue) market shakeup.

    Cheers,
    Uday Parmar

    PS - Lets keep this thread alive!
  28. "From a technical point of view, Cloudscape is a good database, but it's all Java, and developers usually want to work with a full-blown server, which is why we haven't seen much of these products," said Zack Urlocker, vice president of marketing at open-source-database company MySQL.
    I disagree with Zack above. I think Cloudscape has it all:

    "Standard database functionality including multiuser support, indexes, triggers, transactions and failure recovery. ... Versatile support — J2EE, JDBC (1.2 and 2.0) and SQL-92, partial support for SQL-99 and SQL-J."

    I wonder how MySQL is technicly better than Cloudscape?
  29. I wonder how MySQL is technicly better than Cloudscape?
    MySQL is not better, probably PostgreSQL is better, but I am not sure too. There is no reasons to ignore JAVA databases, they are not limited to JAVA and I see no problems to implement some call level interface like ODBC, it can work as server without problems too. IMO HTTP interface is the most ineresting for this kind of database, I think it is a good idea to use it as servlet container and to call servlet as stored procedure.
  30. And firebird ?[ Go to top ]

    I read lots of comments saying that postgresql and mysql are better... ok but have u took a look at firebird ?
    great database, all features (triggers, stored proc), runs on windows and linux...

    since we tried it, we stoppped using mysql and postgreSQL
  31. CloudScape & JDO[ Go to top ]

    Great,

    Before yesr we use CloudScape for some of our projects. It is good to implement JDO specification in the new version of CloudScape. Now we use Hibernate which is the best model for JDO.
  32. This is the best news of the year.

    There are countless web projects that don't need a lot of performance
    or a lot of data. From my personal experience Java performance is
    comparable with native performance.
  33. It's an excellent news from all points of view. I would love to see how cloudscape storage manager is implemented as well as concurrency control stuff.
  34. As I know from some readings about Cloudscape they have only table level locking no row locking is supported. Do they have row level locking now?
  35. As I know from some readings about Cloudscape they have only table level locking no row locking is supported. Do they have row level locking now?
    A quick google search says yes.
  36. Yes, google says it uses something like 2PL (reader blocks writer), but probably it is some old version, but it is possible to find something about cloudscape and SGT (optimistic concurrency control).
    Doe's somebody know how to use IBM documentation ?
  37. Doe's somebody know how to use IBM documentation ?
    Open and read? :) What are you having problems with?

    http://www-306.ibm.com/software/data/cloudscape/pubs/collateral.html
  38. Thanks, it says Cloudscape uses row level read locks for concurrency control implementation (probably it is too conservatyve) .
  39. You're Welcome.

    Its locking mechanism is probably good enough for what it should/can be used for. Most applications (IMHO) seldom need all that DB2, Oracle, etc provide.
  40. Best in Class features[ Go to top ]

    Features- and Support-wise; you can't go wrong with Daffodil DB. Have a look at http://www.daffodildb.com or click http://daffodildb.com:8080/eval/filldetails.jsp for a free 90-day evaluation version.

    Best,
    Uday
  41. Looks like it is official[ Go to top ]

    http://jakarta.apache.org/site/news/elsewhere-2004-2ndHalf.html#20040803.1
  42. Some tests[ Go to top ]

    I ran the following test agains HSQL Server,
    Cloudscape Embedded(ij) and Oracle9i. I didn't
    do any tunning on any of the installations:

    SQL> create table t1 (c1 number(8), c2 number(8));
    SQL> insert into t1 values( 1, 2);
    SQL> insert into t1 select * from t1;

    The last statement was ran a couple of times to create 128K rows in T1.

    Oracle is the first and has a large lead over the second.

    One potential problem with Cloudscape is that it creates a file for each
    new table. I wonder what happens if my database has over 3000 tables.
    I went through the developer documentation and Cloudscape seams a good choice
    for embeding this DB into a server application. I'm looking for a good implementation of isolation levels, row level locking and the ability to start multiple transactions in parallel using multiple connections. It took me 2 hr to install, read the docs and run the test, not a bad learning curve. I found the documentation to be very good.

    HSQL is fast but is caching a lot of data. As per documentation it is not reusing freed space unless you run a compacting command. I wonder how does compacting work when the database is 100Gb. On the positive side it's the easiest to install and use, everything worked from the beginning and I was ready in 30 mins.

    Catalin Merfu
    http://www.accendia.com
  43. Some tests[ Go to top ]

    You can try your test with Daffodil DB, I am sure you will find excellent result. To download Daffodil DB click here

    www.daffodildb.com
  44. MySQL and Daffodil DB[ Go to top ]

    I just noticed that both MySQL AB and Daffodil DB have purchased the "cloudscape" adword on google (i.e. a search for "cloudscape" will bring up sponsored advertisements for both of these products). Well done IBM - I've just spent the weekend playing with cloudscape and it's very fully-featured. I like the idea of an embedded database with a supported upgrade path to DB2. My only concern is that it's not clear whether this upgrade path will continue with the Derby project.

    It does seem clear that you've got your minor-league competitors running scared enough to pay for anti-cloudscape advertisements! Great product.

    Check it out on Google.
  45. Cloudscape - getting it.[ Go to top ]

    ...Well done IBM - I've just spent the weekend playing with cloudscape and it's very fully-featured. ...
    Where did you get Cloudscape? I'd like to test with it too. I know it comes with WSAD but I don't have a copy of that.

    I've been using hsqldb for small and demo projects. As a side note - tried to upgrade to 1.7.2 and it wouldn't work with Eclipse/Hibernate/Tomcat. Never came back from the query/logging in. Had to roll back to 1.7.1. Works fine with manager tool that comes with it.
  46. Cloudscape - getting it.[ Go to top ]

    http://www-106.ibm.com/developerworks/db2/library/techarticle/dm-0408cline/
  47. Cloudscape & Java Databases[ Go to top ]

    PointBase is a 100% Java DB that has been around since 1998. It supports more features than Cloudscape, MySQL, HSQLDB, etc. and is very robust.

    If your app is Java, an embedded Java DB can be faster than a server DB like Oracle or MySQL, since there is no interprocess communication required.

    I think Cloudscape going open source is going to raise the profile of Java databases. I doubt it will make Cloudscape the leading player in this space that it once was. The leading player in Java DB today is PointBase.
  48. Philip, thx for (NOT) including Daffodil DB and sharing with us that Pointbase is the No.1 Java DB. (btw, could we have somthing to back this claim?)

    Mike, as far as I remember, there is a player (who claims to be No. 1 in the market, who (not so long ago) used Daffodil DB as a key word, as have other small(er) players. (Not that this totally justifies the usage; but we consider it to be fair play. All we are doing is making people aware that there are other Java DB's in the market.)

    Though I (like most of us) am happy about the IBM/Cloudscape/Apache movement, I want to raise the profile of Daffodil DB. And I am prepared to put my money where my mouth is because the only way the 'minor league' players will be able to hold their own is via a rock-solid product.

    Lets have some independant testing conducted on Cloudscape, Pointbase, MySQL, HSQL and Daffodil DB. I leave the parameters for testing to be decide amongst all of us. Any takers?

    Uday Parmar
    www.daffodildb.com

    PS - You can download a free, 90-day evaluation version of Daffodil DB by clicking: http://daffodildb.com:8080/eval/filldetails.jsp
  49. Try Daffodil DB[ Go to top ]

    Try Daffodil DB, it is a feature rich complete RDBMS it support BLOB, CLOB datatypes other major features are Full-Text search, XA transaction, Unicodesupport, encription, multiple data file support, all 4 transaction isolation levels and lots more. Most optimized version ( Daffodil DB 3.3) is about to release.

    Daffodil DB is tested with Hibernate,tomcat and many application/web servers, JDO products.

    See List of supported database

    You can download Daffodil DB from
     Daffodil DB
  50. What about McKoi SQL?[ Go to top ]

    Since nobody mentioned it, another small league player in the embeddable java database market is McKoi DB.

    McKoi is standard compliant and has many great features, small footprint and is real easy to get started with. It also works perfectly with Hibernate.

    It is released under a GPL license and an affordable commercial license can be purchased if distributed with a commercial product (~30$ per copy).

    Having compared it with HSQL, I can testify that it is much more solid. It probably of course won't measure up so well against Cloudscape or Pointbase.

    And regarding Daffodil DB, no matter how good it is, I don't see how we could use it for our needs, as it would almost double the price of our product. But still, compared to Cloudscape or Pointbase it is affordable.

    Another very good product that has not been mentioned is FirstSQL. Just like the others it is an embeddable java database that supports a lot of features. Costs ~700$ per developer and ~30$ per license. It unfortunately does not work well with Hibernate as of now.
  51. What about McKoi SQL?[ Go to top ]

    Dear Cedric,

    First things first .... Daffodil DB works perfectly not only with Hibernate, but also with popular JDO brands like Kodo-JDO, JDO Genie etc. Not to mention its perfect compatability with app servers like JBoss, Tomcat, WebSphere and Pramati.

    We are (to my knowledge) the ONLY embedded Java database to offer features like:
    1. Full Text Search
    2. Support of all 4 Transaction Isolation Level, and
    3. PSM (Persistent Stored Methods .... like PL/SQL in Oracle).

    Btw, these came to me in a jiffy; I bet I could think of some more ;0)

    I habve no doubt as to the core strengths of MySQL; however 'commercial' options have their own advantages, and their own business models. If we compare ourselves with Cloudscape and Pointbase ---- We have better features than both, we are faster than both when one talks about inserting 'realistic' numbers of records ... IMO, 50k upwards.

    Prima-face, it might seem that we cannot compete with a 30$ price per copy; however lets also look at the blanket liscence scenario (read bulk discount ... i.e. 200 - 1000 licences), our pricing per copy come down to something which is extremely competitive with the $30 figure. (Not to mention our royalty-based pricing, wherein you pay us a percentage of revenues accrued via the sale of your application which had Daffodil DB embedded.)

    Cheers,
    Ashish Srivastava
    www.daffodildb.com
  52. FYI[ Go to top ]

    http://www.asiacomputerweekly.com/acw_ViewArt.cfm?Magid=1&Artid=24738&Catid=8&subcat=75
  53. FYI[ Go to top ]

    Pls dont let this thread die!!!! Any news wrt Cloudscape/OS?

    :0(