Best Price and Performance: Oracle Now #1 on the ECperf Charts

Discussions

News: Best Price and Performance: Oracle Now #1 on the ECperf Charts

  1. Oracle has achieved the highest Performance figure of 51007 BBops/min@Std (Price/Performance: $37/BBops), and now tops the ECperf rankings in both Price and Performance. Oracle today also posted another, mid-range result with a Performance figure of 26039 BBops/min@Std and a Price/Performance figure of $16/BBops.

    Last week, Oracle tied BEA with the best Price/Performance figure of $7/BBops in which they achieved a Performance figure of 19945.00 BBops/min@Std.

    Check out Oracle's Latest ECperf Results


    In a follow up Q&A, Vineet Buch, Director of Performance Product Management, Oracle Server Technologies, provides some further insight into Oracle's 3 latest results.


    Interview with Vineet Buch
    ===========================

    Q: Congratulations on three very impressive results! You have certainly entered the ECperf arena with a bang. Could you summarize your results for our members?

    Thank you! The following are our recent announcements:

          -Today we announced our very first performance result of 51,007 BBops/min on a Sun Fire 3800 server. We now have the best performance record with a comfortable lead of 13,216 BBops/min.

          -Today we also announced a mid-range result on a Sun Fire 480R server with an impressive performance of 26,039 BBops/min at a very affordable price of $16/BBop.

          -Last week we announced our very first price/performance result of $7/BBop on Linux. We achieved this best price/performance result at a substantially higher BBops/min number than the previous record. In fact, over 2.6 times higher.



    Q: What do these results mean for customers?
     
    With these results, we have demonstrated both our ability to scale up at the high-end and to be cost-effective in terms of price-performance. Platform software like a J2EE server has to perform for a variety of customers – those who want very efficient resource utilization on low and mid-range hardware, and those who want to push the envelope of performance and scalability. We have shown that we can address both requirements. The icing on the cake is our mid-range Sun result, intermediate between our other two results; good midrange performance at a good midrange price.


    Q: Regarding the best price/performance ECperf result, could you explain the reasoning behind your configuration?

    The lowest price-performance result was achieved on a reasonable-sized configuration (three 2-CPU mid-tier nodes, one 4-CPU database server), and not on a stripped-down, bare-bones system. Our goal was to show what Oracle9iAS could do on a system similar to what a real customer might use. So we chose not to go for artificially small configurations, like 1-CPU database servers. It turned out that Oracle9iAS was able to achieve BBops that gave us the best price-performance anyway.



    Q: The ECperf submission window has now closed, but not all submitted results have been reviewed yet. Has Oracle submitted additional results to improve upon the three results currently posted on http://ecperf.theserverside.com?

    The answer is an enthusiastic yes, and the results should show up here in two weeks. We were able to raise performance significantly and, hence, lower the $/BBop, in all three results.




    Threaded Messages (64)

  2. Very interesting! Especially the timing - having waited for all this time, Oracle has been carefully watching and learning from all other vendors, digging into their configurations, and working on the driver that is not available to anybody else.

    Finally, when the ECperf race is almost over, they flud us with all these resuls, not leaving any room for anybody to respond. Great strategy! Just smells fishy...

    On a more technical side, though:

    Oracle has turned ECperf into a database benchmark. All their results demonstrate is that they can tune the DB better for a thirty minute run of this benchmark. Notice the huge amount of memory and fine-tuning (NOLOGGING on some tables, additional indexes, mounting the file system with noatime, etc.) on the DB server;

    Their results use versions of the DB and 9iAS not yet publicly available (committed GA is August 10)

    Their price is a completely bogus benchmark special created for ECperf. The DB and 9iAS price is only for a 1 year lease of the software. In addition, their support cost is only $2K/$3K for the DB/9iAS respectively for up to a 24-CPU license vs about 22% of the license cost previously. Someone should find out what their support pricing structure is for longer terms. They're practically giving it away!

    Looks like a database vendor is a foreigner to the ECperf spirit that has been more a less maintained by the other vendors. Well, now that the ECperf game is over, we can look back and think...

    Through the years of my consulting, I strongly believe: you buy DB from the leading DB vendor, h/w from the best h/w vendor, and application server from the best app server vendor. With other combinations you just lose something, and eventually this is the place where it hurts.

    George.

  3. I completely agree. Of course, isn't the problem at hand to find out who the best vendors are?

    It is really a loaded question because the best vendor changes on the needs of the client.

    Regardless, like others before me, I have become desensitized to ECPref. The idea is good but the execution has been poor. On the other hand, given the nature of our industry I didn't expect anything else.
  4. STOP THE PRESSES!!! You have got to be kidding me...WAKE UP EVERYBODY, THIS IS A COMPLETE SCAM!

    Look at the software license costs at www.oracle.com

    http://oraclestore.oracle.com/OA_HTML/ibeCZzpHome.jsp?site=OracleStoreUS&respid=22372

    They have come up with a pricing scheme that is 100% incomparable to the other ECperf p/p #s out there. These numbers are based on a 2 year term license @ $7,000/CPU. If you want a perpetual license, which is what the other vendors are using, it will cost you $20k/CPU!!!!!!!! That's almost a 300% increase in "real" cost if you plan to keep the product for more than a few years. The same applies to the database.

    This scam reminds me of the Oracle $1M guarantee that their app server was 3x faster than IBM and BEA, only to then see them trash it and license Orion. Oracle will spin the heck out of this one...guaranteed. From my point of view anyone that trusts Oracle is absolutely insane. This is app server #3 for them. We gave up on them after they dropped us after OAS and the prior version of Oracle9iAS. As you can tell, I'm slightly frustrated with Oracle's willingness to mislead. They also claim #1 market share for their app server on their web site. Are they insane??? I guess they believe if they say something long enough us developer drones will start to believe it.

    For this guy to call this an apples to apples comparison to IBM and BEA when they are using a two year license compared to the others using a perpetual license makes me sick! Some of us need to hold Oracle accountable for their consistantly rotten tatics.

    Also, how could ECperf allow this?

    Harvey
  5. ECperf will allow anything. It is no apples-to-apples comparison. The only test that does this is CSIRO (Paul Brebner and others). Whenever I'm in a situation to choose an appserver, I'll buy their report, because they do real comparisons for AppServers. That means NO DB-, JDBC-, JVM-test at the same time just to confuse the audience.

    My 2 cents
  6. Harvey is absolutely right. Oracle has had so many app servers that they can't be trusted. I remember OAS3 and what a dog that was (built using C++ and CORBA). They tried to fix it with OAS4 but it used the cartridge architecture and wasn't much better. Does anyone remember Intercartridge Exchange? Makes me shudder just thinking about it.

    They had one application server prior to 9iAS that came out at about the same time as Oracle8i - can't remember its name. Does any one remember? They also produced WebDB which was OK but came with its own runtime engine so that it didn't have to reply on OAS. Getting it to work with OAS was a real pain.

    They then purchased the Orion application server to build 9iAS. I wonder how much they've mucked up Orion in release 2 of 9iAS.
  7. And if the rumors that circulate here pan out then Oracle could have a 4th App Server if they end up purchasing HPAS.

    Oh well, it is time to prepare for the unrelenting barage of the Oracle Hype that is sure to surround these benchmarks. I had thought their claims were bad enough before when they had no "evidence" to back them up.
  8. Same Name[ Go to top ]

    Hi Mike.

    My name is also Mike Bamford. I work as a Java Developer for Deutsche Bank in London. My email address is michael dot bamford at db dot com. drop me a mail, if you want - there can't be that many of us!

    Regards
  9. The CSIRO is a firm believer in 'level playing field tests'. this is the only real way to directly compare the application servers rather than a complete system (hardware, OS, DB, etc).

    To this end we have produced our J2EE application server report see:
    http://www.cmis.csiro.au/adsat/j2eev2.htm

    To extend the level playing field to include ECPerf I am now approaching vendors to initiate a 'bake-off'. I am pleased to announced that I am receiving a favourable response to this idea from vendors. If all goes ahead this will be of great benefit to the community.

    Watch THIS SPACE!
  10. Of course, a good answer to your point is that you *could* just go ahead and buy Orion directly, instead of Oracle 9iAS. The article mentions that 9iAS is licensed from Orion, and the two products are nearly identical... except Orion's cost is far, far less than Oracle's.
  11. I´ve asked it many times, on many foruns... Will orion keep going? Will it evolve? Anyone knows something???
  12. Orion is still going. They're working on releasing full EJB 2.0 support ("soonish," as they tell me), hopefully with Servlet 2.3 and JSP 1.2 soon to follow. They *are* still working on it, via a massive refactoring.
  13. you really believe this, do you? The only sign of activity in almost 2 years has been the move from release 1.52 to 1.54 (the docs are still 1.52, however). Orion is farther behind with respect to the standard than any other offering out there.

    I have no doubt that Oracle has made sure the 2 IronFlare guys are working on the Oracle product, and not hacking away in their swedish flat.

    I find it amazing thet they still want to charge $1500 for this.
  14. <quote>
    I´ve asked it many times, on many foruns... Will orion keep going? Will it evolve? Anyone knows something???
    </quote>

    Believe it or not, Orion is being actively developed and they will have a version (likely 1.6.0) that should be coming out reasonably soon (i have both said and heard this a lot lately...). Unfortunately, a lot of their time is taken up working with Oracle. I'm not sure as to the scope of the release but it will go a long way toward being j2ee 1.3 compliant.

    Ray

  15. Actually, Oracle has made a lot of changes to the base Orion product while they have had it. My understanding is that they have IronFlare guys working with them on it, though thats just been mentioned on mailing lists.

    If you look around at the OC4J distribution, you will see new changes that don't exist in Orion.. JMX capabilities, new CMP capabilities, and quite a bit of SOAP..er.. WebServices utilities as well.

    Of course, I have always been a fan of IronFlare's product, but given the lack of responsiveness on their mailing lists (its offline right now!), and the slow rate of changes/fixes to their server, its hard to recomend anymore. I personally would probably look over towards JBoss and friends these days.
  16. Oriaon is just a J2EE container. Oracle 9iAS is a NOT just a container but a complete App Server platform , J2EE container in 9iAS is called OC4J which is based on Orion.
    9iAS has more technologies like Portal, personalization, Dynamic Web cache, Wireless, Business Intelligence (Clickstream Intelligence), Oracle Enterprise Manager, Workflow engine , Web Services, Unified Messeging etc...
    All these technologies are included in the 9iAS price. BEA and IBM charges seperatly for them and still Oracle's price/performance is best.
    ECperf is a Enterprise JavaBeans(EJB) benchmark meant to measure the scalability and performance of J2EE servers and containers. ECperf stresses the ability of EJB containers to handle the complexities of memory management, connection pooling, passivation/activation, caching, etc
  17. <quote>
    9iAS has more technologies like Portal, personalization, Dynamic Web cache, Wireless, Business Intelligence (Clickstream Intelligence), Oracle Enterprise Manager, Workflow engine , Web Services, Unified Messeging etc...
    </quote>

    Which of these products are actually implemented on J2EE? Even Oracle partners had serious issues trying to run some of the vertical applications. They have been built on the old application server platforms, not compatible with J2EE. They only recently got the J2EE app server right, the apps will take much longer.
  18. Do you think that J2EE is a best tool to write parts of the app server as Dynamic Cache or personalization ? It is pretty logical that this parts of the server are developed in C and run as native code on the appropriate platform....
  19. Dear all,

    I wanted to clear up a couple of technical points:

    1) LOGGING IS NOT OFF!
    On the Oracle results logging is most definitely not off. The effect of the NOLOGING on the table create is to turn logging off for the table creation and for bulk loading only. So again during the benchmark run logging is most definitely happening. In fact please note that we use a fast Sun T3 controller for the log files as a faster device for logs most definitely gave a performance boost by reducing the wait times in the DB for log file sync. For more information on the NOLOGGING option please consult chapter 15 of the Oracle 9i admin reference guide.

    2) If LOGGING WERE OFF
    The result would have failed the durability run rules of the ECperf spec and the review committee would not have approved the result for publication!

    This is I believe an example of where the benchmark run rules and the associated peer review work together to add meaning to the published result.

    3)ECperf rules do allow a window for beta products
    The ECperf rules (like many benchmarks) do allow for results to be published against beta products and yes there is a 3 month window where end users can't test the results that are out there. What happens at the end of the 3 months however is that if the products are not available then the results will be marked NON COMPLIANT. This is something that no vendor wants as reduces confidence in the figures that they have produced or will produce in the future. Again this ECperf rule is very similar to many other industry benchmarks.

    4) the use of the noatime and directio features
    These are pretty standard features of the Solaris OS and are often used in production by real customers. If you run ECperf and look at the benchmark runs at all you will see that the database read activity is extremely low (at least for mature J2EE servers. This is by design as the focus is supposed to be on scaling the middle tier not the database. The use of the noatime and directio features makes a difference for the performance of the red logs but really the database could be on any old device or partition and does not really benefit from the use of either of these features.

    If you still think this is somehow invalid please let me know as it seemed pretty reasonable to me.


    Hope this helps to at least explain some aspects of the exercise.
    regards

    Tom Daly (Sun)
     

  20. Congrats Oracle!

    :D
  21. Here we go again ...

    Looks like every Oracle employee (including our dear friend Buch) is learning from the master himself on the art of spin!)

    It is just not fair on Vineet Buch to claim that this is an apples-apples comparison with the other vendors. For starters, the Oracle report says many of these products (incl the db and jdbc drivers) will be available sometime in August. How can this be an apples-apples comparison if these products aren't even available to the public?

    This is a completely bogus result ... the pricing has been invented just for ECperf. Anybody in their right mind would think a maintenance cost of $2k for the first year, followed by an astronomical jump to $20k from the second year onwards is a complete farce! The total maintenance costs as published in the FDA is $5k (when the license costs $250k!).
    This sure doesn't look like anything we've seen in reality.

    Again, this points to the (many) shortcomings of ECperf as I've been pointing out all along. There is nothing to stop a vendor from coming up with creative pricing schemes that bear no resemblance whatsoever to reality. I don't know why anybody would opt for a 1 year licence to start with and later upgrade to a perpetual license by forking out a pile of cash that is 10x the original 1 year license cost). Either ECperf should completely drop the price/performance metric or come up with a more meaningful set of guidelines.

    Frankly, the recent set of ECperf results from Oracle have demolished whatever little credibility there was to the ECperf benchmark. Many of us have become desentisized to these impractical configurations/pricing and have learned to ignore all the hoopla surrounding these results. So much for a benchmark that was supposed to be *the* application server benchmark!

    I couldn't be more thrilled to hear the end of ECperf as we know it! Good-bye and good-riddance.

  22. Why not to use "per named user" licensing schema for Oracle DB to decrease overall SUT cost ?
  23. Good explanation Ramesh !!

    So this report is reviewed ecPerf Board that has representatives of IBM, BEA, Borland, etc. before this is posted .. So we should not be to cynical about the validity of the report and should be worried that the ecPerf rules has been bent

    All ORACLE bashers should appreciate that Oracle9iAS EJB performance is atleast faster than the fastest results posted by any other vendors i.e. BEA and IBM if not three times faster (the TALL CLAIM made by them) !!


    Interesting though BEA used the same price model for Oracle DB for their benchmark. See page 33 on their disclosure at http://ecperf.theserverside.com/ecperf/resultslinks/HP-ECperf-FDR.pdf

    <quote>
    Oracle software and support: List price quoted from Oracle. Oracle Global Pricelist,
    pages 2 and 3. See Oracle sales representative to process
    request for a 1-year term..
    </quote>

    So all vendors are in the same boat and we can't blame just Oracle !!

  24. Maybe I'm to much of an idealist, but I don't appreciate Oracle's attitude;

    http://biz.yahoo.com/rc/020626/tech_oracle_1.html

    "Why Oracle's future is so good is because of a concentration of spending on the few surviving suppliers: Microsoft (NasdaqNM:MSFT - News), Oracle, SAP (XETRA:SAPG.DE - News) and IBM (NYSE:IBM -News)," Larry Ellison told reporters on the sidelines of an Oracle conference here.

    "It will be killing fields. We will grow and prosper. Customers will have fewer choices," he said.

    Lovely!
  25. "It will be killing fields. We will grow and prosper. Customers will have fewer choices"

    That just about sums up Oracle's attitude towards it's customers. Force them to use a crappy product ... I couldn't be more happy to see DB2 come on so strongly against Oracle. Oracle must be feeling the heat from all corners for Larry to make such a statement. I also hear that they have shuttered many of their CRM projects and moved them offshore to India for maintenance.
  26. "That just about sums up Oracle's attitude towards it's customers. Force them to use a crappy product ... I couldn't be more happy to see DB2 come on so strongly against Oracle. Oracle must be feeling the heat from all corners for Larry to make such a statement. I also hear that they have shuttered many of their CRM projects and moved them offshore to India for maintenance."

    Oracle's database is far from crappy, and I think you would have a very difficult time showing that DB2 is somehow technically superior to it. Ditto for 9iAS vs. WebSphere 4.

    Their relationship with their customers has always been a very wierd thing (people apparently hate them but still buy from them).

  27. "Oracle's database is far from crappy, and I think you would have a very difficult time showing that DB2 is somehow technically superior to it. Ditto for 9iAS vs. WebSphere 4."


    *strongly* disagree ... I've used DB2 and Oracle and see no reason to switch to 9i anytime soon.

    This is the first time I've heard anybody say 9iAS is vastly superior to WebSphere (you must work for Oracle!).

  28. "*strongly* disagree ... I've used DB2 and Oracle and see no reason to switch to 9i anytime soon. "

    I never suggested you should switch. DB2 is a fine database. I believe Oracle is technically superior, and has been for some time. 9i just solidifies this more. If you'd kindly outline the areas DB2 is superior, I will outline the areas 9i is superior.

    The one area I would suggest there's an interesting debate is in DB2 shared-nothing clusters vs. Oracle Real Application Clusters. There still isn't much historical evidence to show that in practice RAC is better than the shared-nothing approach.

    "This is the first time I've heard anybody say 9iAS is vastly superior to WebSphere (you must work for Oracle!)."

    This is the umpteenth time I've seen someone who can't debate a technical subject and make an ad hominem attack to look like they can.

    The issue with WebSphere vs. 9iAS isn't that 9iAS is "vastly superior", it's more that WebSphere is so poor, in my opinion. I've been using WebSphere since its early 1.x releases. WebSphere has been plagued with problems with maintainability, stability and compatibility up until its 4.0 release. 3.x was never close to J2EE compatibility. EJB compatibility was always also quite a joke. WebSphere was originally frankenstein of purchased products & acquisitons, with most of the new engineering occuring in the 4.0 release (and from what I understand from friends using it, it too has problems compared to BEA WebLogic or Orion server, which is how I based my 9iAS comparison).

  29. <quote>
    The one area I would suggest there's an interesting debate is in DB2 shared-nothing clusters vs. Oracle Real Application Clusters. There still isn't much historical evidence to show that in practice RAC is better than the shared-nothing approach.
    </quote>
    Stu,
    your statement about shared cluster does't reflect the reality.
    IBM position is that clustering is a major requirement for high end applications. And not only database:
    if you need high availabilty and load balancing, you may need it not only on Database system but as well on application, mail system, ERP and whatever...It's the reason why IBM (as Hardware manufacturer and OS vendor) provides clustering at OS level in their zOS, OS/400 and AIX (with HACMP). They are working with OSS community to provide such function on Linux as well. If you need clustering, you buy it for your entire system and you get it for every application you need: RDBMS but also ERP or Application Server if ERP or Application Server is critical to you. And you don't have to administrate it (many time in different ways) for every application you have.
    Oracle position as RDBMS vendor is different. For them, clustering is a differentiator (and in many case it works well from sales perspective).
    From my personnal experience, I've used both products, and from performance and scalabilty they are the best (far away from SQL Server wich is more friendly to deploy). I've developped J2EE appl with Oracle and DB2 on Unix, Linux and NT systems. On Unix, I think Oracle & DB2 are very closed (exept DB2 on HP-UX and on AIX-HACMP, Oracle RAC is tricky to tune and manage...because of this 2 layers clustering).
    On NT,based on Solaris or AIX experiences, I like Oracle Clustering, but the JDBC performance is disapointing (I used Oracle 8 and thin driver).
    Now, I'm not sure that RDBMS clustering debate is accurate on a J2EE performance benchmark forum...
    Anyway congratulation to Oracle for their results on this ECPerf. I performed 3 benchmarks with J2EE application severs, and I know the work (and headache)it is... I also believe that others (Bea, IBM , Sun...) will respond quickly.

    Fred
     
  30. "Stu,
    your statement about shared cluster does't reflect the reality. "

    I'm not sure how, as it wasn't a statement. I was posing a question for debate: Is 9i RAC with its shared-disk + shared-memory-cache approach a more scalable and performant for a majority of applications vs. the DB2 shared-nothing approach?

    Certainly one that desires a high level of fault tolerance (4-5 9's) should have clustering on all aspects of their system -- application server and database.

    Clustering at an operating system level is usually integrated with database-level clustering (Oracle 9i, for example, integrates with Sun's HA or Microsoft's wolfpack). Application servers tend to have their own failover/load balancing algorithms and bypass the OS-level clustering (with out much penalty).
  31. Stu,

    < Is 9i RAC with its shared-disk + shared-memory-cache approach a more scalable and performant for a majority of applications vs. the DB2 shared-nothing approach? />

    I think that Oracle RAC is better. Oracle RAC doesn't require data partitioning, IBM EEE does.
    So If I have to add a node in 9i RAC you can just add the machine . In DB2 EEE you should repartition the data. 9i RAC runs SAP( it's certified ) with no problem( "out of the box" ). I don't think that that's the case with DB2 shared-nothing. In fact in SAP benchmark Oracle9i RAC is far better than IBM.( For SAP bench. IBM uses DB2 on OS/390, which as far as I know uses shared-disk approach( I think it's called Parallel Sysplex ( or something like that)). I don't see any results using shared-nothing approach. I don't think that shared-nothing is good for OLTP.( The TPC-C test for shared nothing are good just because the bench. design allows the over 90% SQL to be local for( doesn't use alternate indexes, just pks) the nodes, which will not be the case in the real world application )
    For OLAP type of applications both approaches make sense( Oracle RAC and IBM shared-nothing ) . So in general I think:
    OLTP - definitely Oracle 9i RAC
    OLAP - both ( Oracle RAC or shared nothing ) will make sense
  32. And this is the first time I see someone saying websphere is better than ANYthing...

    Ps. I don´t work for Oracle
  33. Andy,

    Looks like you are having some problems with Oracle( as a company or with Larry as a person). Correct me if I am wrong but I think that you weren't happy with IBM also in some of older discussions.

    I am Oracle customer and I don't think that they make "crappy" like you said products.( Especially the database is the best).
  34. Far from it ... I'm a very happy DB2 user and will use it anyday over Oracle.
  35. Andy,

    Sorry , my mistake.
    Well I am happy Oracle user and I don't even consider DB2 , but I am not calling the product crap. At least I respect the people that are developing it and the people that are using it.


  36. Veda and Ramesha:

    Why do you folks ignore the fact that ECperf is a committee and only the voice of the majority is what counts?

    Also, the fact that the other vendors use the same (unrealistic) pricing scheme only proves my earlier point that the benchmark itself is fatally flawed! If I were arguing this in front of a jury, I would simply "ladies and gentlemen, I rest my case" :-)

    Why doesn't ECperf have a pricing scheme that reflects reality and not some fancy seat of the pants mechanism invented to artificially lower the cost for the publication?

    I also note that Oracle's pricing scheme extends to their application server license as well (the perpetual license and maintenance costs are gargantuan when compared the special 1-year license to meet ECperf requirements).

    'nuff said!

  37. <quote>

    Why do you folks ignore the fact that ECperf is a committee and only the voice of the majority is what counts?

    </quote>

    Andy,
    Do you think most members of the eCPerf are Oracle supporters? In that case they would not have allowed BEA to post their results ?

    We should remember "when you point a finger towards someone..realize that three fingers are pointing at you " ?

    You should see the positive things all companies like Oracle, IBM and overall the benchmark NOT just the negative things ..

    "To be or Not be is the question ?"

    Oracle RDBMS has been faster than other databases since ages (according to the benchmarks) Oracle9iAS is now faster than other faster than App Servers (according to benchmark) ..

    May be BEA or IBM will publish another pending result this week or next week that makes them faster than Oracle9iAS ..

    Until then Oracle9iAS ..we have to live with the facts..

    Benchmark counts performance and not the price .. Why so many companies are buying Websphere, Weblogic or Oracle9iAS even free EJB servers available ..
    Why companies are still buying Solaris or HP-UX even Linux is available for ages ??






  38. Veda:

    "Oracle RDBMS has been faster than other databases since ages (according to the benchmarks)"

    The TPC results indicate quite the contrary!

    http://www.tpc.org/tpcc/results/tpcc_perf_results.asp?resulttype=all
    (oracle is #7)

    http://www.tpc.org/tpch/results/tpch_perf_results.asp?resulttype=all
    (oracle leads in two categories)

    http://www.tpc.org/tpcw/results/tpcw_perf_results.asp
    (oracle nowhere in the top-10)
  39. "Oracle RDBMS has been faster than other databases since ages (according to the benchmarks)"

    The TPC results indicate quite the contrary! "

    If you look at the "non-clustered" category, Oracle certainly has a dominant position.

    "Clustered" databases under TPC-C can be optimized such that over 99% of data access is local to a cluster node, giving advantages to shared-nothing architectures such as DB2 and SQL Server. This is the fault of the TPC-C spec. To paraphrase Gregory Pfister from "In Search of Clusters", TPC-C takes the position that one could carry data across cluster nodes on turtle backs, and it would still scale.

  40. George,

    1. OLTP systems:
    Top Ten Non-Clustered TPC-C by Performance( single machine )
    http://www.tpc.org/tpcc/results/tpcc_perf_results.asp?resulttype=noncluster

    Oracle has it almost all of them.

    Top Ten Clustered TPC-C by Performance
    http://www.tpc.org/tpcc/results/tpcc_perf_results.asp?resulttype=cluster&version=5

    Oracle doesn't have the lead there. Microsoft is using federated database approach , that's why they have such good results. To date I know only two companies implemented this approach ( Each node "owns" part of the data,you cannot have referential integrity , you have to partition the data across the nodes and if you add a node you have to partition the whole thing again, no package applications ( SAP, Peoplesoft and etc ). IBM is using shared nothing ( DB2 on UNIX/Windows )approach ( you have to partition the data across and if you add a node you have partition the whole thing again). Oracle is using shared disk approach( IBM is using similar approach for DB2 on mainframes ) which always will be slower than shared nothing but you can run any real application on the cluster without any changes. In fact in the SAP Parallel benchmark ( for clusters )( this is a real application not just 9 tables ) Oracle is number one by far.
    The SAP benchmark is:
    http://www.ideasinternational.com/benchmark/sap/sap3sdpr4.html
    2. OLAP systems:
    http://www.tpc.org/tpch/results/tpch_perf_results.asp
    IBM has 100GB and 300GB,
    Oracle has 1000GB and 3000GB.

    So I don't think that Oracle is doing bad in the database benchmarks.
  41. Stu,

    You wouldn't happen to know how to disable the stupid oracle 9i servlet engine would you? I have found zero docs on this, and we are trying to test out Oracle9i.

    -Newt
  42. Start-up the Oracle Installer application, and tell it to remove all installed applications. You'll be better off.

  43. Back to the benchmark results:

    In my opinion benchmarks must be analyzed carefully. What I like on Oracle benchmarks (TPC-C, ecperf, SAP, etc) is that it uses real world scenarios. Check out the hardware Oracle used for the ecperf on Linux: it's almost the same I'm considering to use in the project I'm working on. Now take a look at the hardware MS uses on its TPC-C benchmarks...

    [for development we're using OC4J and WebCache (9iAS 9.0.2) on a 1-CPU PIII Linux server and 9iDB R2 on a 2-CPU PIII Linux server]

    Oracle's results means to me that you can use 9iAS for J2EE apps with good price/performance using a real-world scenario.

    BTW, Oracle JDBC driver is not v3 but implements part of the v3 spec. Be sure they're working on it...

    Oracle database is a excellent product. I've worked on a project with a +15TB Oracle database, tables with +10billion rows. It's not easy to manage a DB like this, you need a good DBA and to use many Oracle features like range and hash partitioning, materialized views, directpath loads, query rewrite, exchange partition, star transformation, bitmap indexes...

  44. BTW, when is Oracle going to achieve J2EE1.3 certification. Although they have been claiming their appserver as J2EE1.3 but they haven't yet implemented the full J2EE1.3 e.g EJB2.0 is not yet fully implemented. I have asked this question several time on their J2EE development forum, but it always goes unanswered.

  45. As expected, more FUD from the high priests themselves ... bring it on folks!

    What is this I hear about the end of ECperf? Where do we go now for an "apples-apples" (excluding the half-dozen or so variables in each result and not including products that are months away from GA) comparison of the different app servers?

    Just when I was expecting a one gazillion "beebops" result at a cost of $-15 per "beebop" ie: the more "innovative" vendors would pay you to run a benchmark if you bought the just the one day license; while the perpetual license costs are more in synch with reality (but then,why would anybody ever want a perpetual license, RIGHT!). You gotta love this ECperf thingy -- only a committee could design such a gem with all the zillion loopholes!
  46. All Oracle has managed to prove with these numbers is that you can bend the rules of the benchmark and come up with cost calculations that simply defy logic in the real world (this also points to a deficiency in the ECperf spec as well).


  47. What's the quote?

    "Lies, Dam Lies and Statistics"
  48. Great News. Looks like Oracle is surging ahead to become a leader in J2EE arena. Too much good news for Oracle9iAS :

    1) Oracle Acquired TopLink - market's darling O-R mapping tool

    2) Now #1 in ecPerf - Both Price and Performance

    In my opinion Oracle is really striving to become market leader in Apps Server arena:

    We always see the bad part in Oracle. Let's try to look at Oracle positively.

    - Licensed JBuilder from Borland few years back and now they improved and made JDeveloper one of the best IDE

    - Licensed Orion and make OC4J enterprise ready and hope to see further improvement

    - Acquired Toplink and hopefully bundle with Oracle9iAS with lower price and make Oracle9iAS attractive to consider

    - Now they are #3 or #4 in market share (a long way from nowhere)

    - Best Price/Performance and Performance to make IT management to atleast consider Oracle9iAS during evaluation process

    Oracle needs to be a little agressive in implementing J2EE features early and achieve so called certificiation earlier than it's competitors to give them run for their money and see them in the top..

    Should we be the lovers of other App Servers be scared of Oracle now that we have to now switch to Oracle9iAS from our beloved app servers like BEA or Websphere if Oracle eats into their market share ?

    As a J2EE developer we should not be ? We are developers and should always keep our mind open to switch WHOSOEVER gives us the BEST !!

    Now everybody in the thread welcome to blast on me if you disagree!!

  49. Ecperf is an EJB performance benchmark. The primary purpose is to measure app server performance and scalability. The price-performance is a secondary measure. The Ecperf review group comprises of representatives from ALL leading J2EE App Server vendors. And the reviews involve extensive scrutiny and deliberations. (No vendor will allow a competitors result that is in violation of the Ecperf spec/rules!). Given this, there should be no concerns about teh credibility of teh benchmark nor the results review process.

    About some of the comments/suggestions that have come out in this thread, Consider:

    1. Price-performance rules not clear/rigid enough?
    This is an attempt to reflect the total-cost-of-ownership. The intent is very purposeful. However, with various vendors having different pricing schemes it is very difficult to have a pricing clause that will fit all the models very well. So the approach chosen is to allow a 1-year cost of ownership. Now if other vendors (Pramati included) dont have a 1-year term price shouldnt count against Oracle using it.

    Secondly, if any user wants to make a comparison of submissions using an alternate pricing (like perpetual license) they should feel free to do this. All the information for this is available in the price disclosure. One can apply any pricing and recompute. And then recompare the results.
    2. Require a standard hardware platform where all vendros make a submission?
    This is easier said than done. Which vendor's hardware do we choose. Will all J2EE vendors agree to a given hardware vendor?


    A possible lack of clarity on pricing used doesnt in any way take away the credibility of the benchmark. Nor does it take away the credibility of the submissions or the reviews thus far.

    Cheers,
    Ramesh
    - Pramati
  50. Orion going down[ Go to top ]

    A little off topic but anyway...

    Reading about the development of Orion I have a strange feeling I have seen something like this before:

    Back in the not so old days application servers where not what it was all about, relational databases where. Really open RDBMS products where hard to find but there where two good candidates, Oracle and Sybase. Have you heard of the latter lately? Some years ago Microsoft bought the software for Sybase SQL Server version 4.x and turned it (demolished it) into Microsoft SQL Server. The Sybase product lived on as a good platform independent product and Microsoft had it´s product evolve on Windows platforms. These days market share of the Microsoft product is much bigger than that of Sybase and going up while Sybase is slowly going down, better product or not, platform independent or not. This has to do with size of the maker and what the public expects of it.

    I think this is history repeating. Having Orion software in their product Oracle will stick to it and make it better. With their market share they will get there. Orion itself will hardly stand a chance. I wonder what will happen to it.
  51. Congratulations Oracle !
  52. I'm so sorry to repeat this about ECPerf benchmark but I give no ... on this "pretty magazine for girls"
    Why? Because it has no respect for what means a real correct scientifically experiment as it should be as a
    result of an engineering process. Measurements tests are done by engineers from the times of Egyptians
    and I'm sure that the criteria where the same with 3000 years ago.
    So for having correct results we should ask for an identical benchmark environment:
    - Identical tests suites
    - Identical benchmark system architecture :
       - Identical 100% hardware platform( including all servers, routers, additional LAN equipments , clients etc. )
       - Identical operating systems( by provider, version, build, drivers etc. )
       - Identical persistence layer
    .. and as is normal repeatability of results 2 or 3 times.
    As long as the ECPerf is a free public available benchmark this will happen because each benchmark costs money
    and this money are paid by companies not by users ... so companies are free to do( including a high level of tricks )
    what they want to do for winning/selling ;-) ... if you want real benchmarks you should pay for them specialised
    companies or real professionals ...
    I think here is not the place and time to go with scenarios about market battle which turned on as a pre-historical fight
    but I'm sorry for the way it seems that IT industry started to go ... McDonald's and Coca Cola succeeded even here
    to sell their style in doing business ...
    regards,
  53. ... if you want real benchmarks you should pay for them specialised

    companies or real professionals ...

    Right. Look at this CSIRO (The Commonwealth Scientific and Industrial Research Organization )report at:

    http://www.cmis.csiro.au/ADSaT/j2eev2.htm

    Regards,
    Mileta
  54. Ah, if it were only that simple. If you were to take that approach then you would have to either a) freeze significant testing components for a long time or b) require re-testing as the components change.

    Large government agencies tend to do this. Draw up a list of requirements and conditions and force extremely fair competition. And, when all is said and done, they buy the best solution with 2 to 4 year old technology.

    Would you want to show numbers based on an old slow configuration when there was a newer and cheaper configuration on the market? How do you compare today's prices with those of a year or two ago?

    The ECperf follows in a great (sort of) tradition of attempting to get vendors to show what they can do. EVERY vendor will try to take advantage of everything they can to show the best numbers. Expect it. I don't have the time or energy to find the tricks each and every vendor has used.

    What I think you can do is look at the top three or four players and see who remains up there for a while (thus showing dedication to the market). I would look at their configurations for ideas (because obviously they have their best people working on the tuning).

    Then, I would look at the real problems I was trying to solve. This doesn't always mean the latest EJB feature. It means vendor relationships, support, reliability, performance, current staff skill sets, long term scalability issues, etc. And will all of this in mind try to pick the best one and go with it.

    Having been involved in many purchases over the years I have become convinced that if you narrowed a choice down to the top three and on day one intentionally picked the worst of the three and then went to work you would be far ahead of the team that spent a year or two trying to pick the best solution.

    I guess all of this was just a long way of saying ECperf's are not perfect, but they are a useful tool. You have just got to be aware of their limitations and apply some good common sense.

    -Bob
  55. Cristian and others ,

    As one of the members of the group that created the ECperf benchmark I can emphatically state that we have always be supportive of companies, individuals and really anybody being able to do their own J2EE application server testing in-house and have gone to some lengths to protect this ability into the future. So you can do your own app server to app server shoot out and use the results for your own purposes e.g. for purchasing, comparison, education, modelling..whatever. I believe from your earlier note and others that this is what you really want but are not seeing from the public results, so I urge you to take what you can from the public results like:-

    - What app servers run the benchmark
    - How far has the J2EE platform been scaled
    - What is the cost looking like as I scale upwards
    - Some idea of the 1 year cost (does this scale to a 3 year cost))
    - What tuning should I be looking at (check the FDA's)
    - What DB's are known to work well
    ...and more

    You get all of this information for free!

    If you want more than what is here for free or you feel that the information is somehow in error then I urge you and everyone to try it for yourself.

    Final comment: The purpose of the FDR and FDA is to allow anybody to assemble the configuration listed and get the same results. There is a wealth of information here being supplied so as to enable the entire community to get the best possible results from the J2EE platform



    regards
    Tom Daly (Sun)
  56. A lot of people seem to be ragging on Oracle over this benchmark. I have some issues with these complaints.

    Firstly, let's get the one problem with these benchmarks out of the way: the maintainence fees. All of Oracle's benchmarks list a $2000 maintenance fee for 9i DB, whereas others tend to have a maintenance cost of slightly higher than sticker price. This needs to be clarified by Oracle, what happened here. Price/performance figures should be updated if there is an unfair advantage here. These figures will not change much for the new #1 server, but it will definitely affect the midrange benchmarks.

    As for the database price itself: Oracle priced its database using the 1-year option @ $10,000/CPU. This is exactly the same price that other vendors used. There's nothing wrong with this number, if ECPerf or the new SPEC benchmark want to limit Oracle's advantage in this regard, they should follow the TPC and measure a 3-year TCO instead of 1-year.

    Now, onto technical matters:

    - Database optimization is necessary. I would argue that database performance has well over a 50% impact on application server performance. This is why TPC-C, (That Other Benchmark), measures the combination of TP-Monitor AND database in their benchmarks. ECPerf participants have to recognize reality in this regard.

    Did Oracle highly optimize their database? I looked at all the schemas, I looked at their configurations. The answer is "not really".

    - They turned off redo logging for most tables. This means all media recovery has to occur at the RAID level -- database media recovery won't work except to the last full database backup.

    - They generally only have unique indexes (in some cases, compound key indexes). There's a couple new ones, but nothing out of the ordinary for most databases.

    - They used CMP, so new wierd proprietary SQL statements anywhere.

    There's nothing voodoo or black magic about tuning a database. It does require experienced (and highly paid) DBAs. Thankfully, ECPerf doesn't include their salaries in its TCO measurements, so there's really no excuse not to tune your DB. With the assumption you have a tuned DB, you can then try to determine your app server's performance.

  57. I don't know this from personal experience but rather from other developers unlucky enough to work in an Oracle dominated shop. Just about everyone I know who has used the Orifice app server is looking to migrate to something else.

    A piece of advice for Larry Ellison: stick to sailing, the way of the samurai and databases. Leave J2EE app servers to those whose main business is app servers, not databases.

    BTW, when will Oracle ship a fully SQL-92 compliant database?
  58. "BTW, when will Oracle ship a fully SQL-92 compliant database?"

    When will *anyone* release a fully SQL-92 compliant database?

    (Oracle 9i supports SQL-92 join syntax now)
  59. When will *anyone* release a fully SQL-92 compliant database?

    >
    >(Oracle 9i supports SQL-92 join syntax now)

    ...plus a plethora of non-standard oracle-specific hacked-up SQL syntax that makes writing cross-platform (i.e. database vendor independent) applications quite an adventure.

    BTW, when will Oracle ship a fully JDBC v3 compliant driver?
  60. <quote>
    ...plus a plethora of non-standard oracle-specific hacked-up SQL syntax that makes writing cross-platform (i.e. database vendor independent) applications quite an adventure.

    BTW, when will Oracle ship a fully JDBC v3 compliant driver?
    </quote>

    I currently work with Sybase, Oracle and SAP DB. They all have vendor-specific quirks or extensions. In fact I have never worked with a database that doesn't have these. Oracle may have more of these than other vendors, but it's certainly not alone.

    As to JDBC v3 compliant drivers and Oracle:

    Hardly any company has true jdbc 3.0 compliant drivers - in fact if you look at the info out on Sun's site - only 5 companies are shipping and not one of them is a database vendor - or rather one is if you count DaffodilDB.

    Cheers
    Ray

     
  61. Since when is SAP a database? I thought that was something CIOs buy because they don't know any better...

    If Oracle is so hell bent on taking over the java world, why haven't they released their own JDBC v3 driver? I understand why Sybase hasn't; they're too busy shooting their feet off to do anything meaningful anymore. Microsoft has just grudgingly released their own type IV driver, and have no interest in the J2EE world beyond the need to kill it. DB2? If you're not part of an IBM shop, who cares? No, Oracle should concentrate on it's core and related technolgies, like a JDBC v3 driver.

    And when can we expect to see a J2EE v1.3 certified app server?
  62. <quote>
    Since when is SAP a database? I thought that was something CIOs buy because they don't know any better...
    </quote>
    SAP isn't a database, however SAP AG have a slick well supported open source database called SAP DB - it's an offshoot of Adabas. Check it out at www.sapdb.org.



  63. Well, you can expect one ... from Oracle, mid-July 2002.
    Partial implementation already available with OC4J 9.0.2 ( Servlet 2.3 , MDBs , JCA 2.0 ,... ) ... just have a look !
  64. We're using Jdev/BC4J/OC4J and it solves many unique problems,
    time to market and ease of use.

    To me, the actual numbers aren't as important as that there _are_ numbers. And that honest numbers are in the range of
    good performance.

    I know from my use of Orion then OC4J that this is one of the
    easiest containers to work with, deploy to, fast to startup,
    fast to hot deploy to .... It's ok to me that these numbers
    may not be as good as Oracle is claiming.

    My seat of the pants experience says that they probably aren't far off. And if this compitition drives down licensing costs to meet some magic numbers then great!

    Curt
  65. Guys, it's not tough, all they had to do was buy Orion.