Discussions

News: New J2EE vs .NET Performance Comparison Performed

  1. The Middleware Company has performed a new comparison of the performance and scalability of J2EE and .NET based on the familiar Pet Store application. This time, the Middleware Company has re-coded the J2EE Petstore and optimized the implementation for performance. In the comparison, a new implementation of the .NET Pet Shop has also been tested. This implementation uses dynamic SQL instead of stored procedures, and like the J2EE equivalent is an object-oriented, logical 3-tier implementation following Microsoft's recommended design pattern for building scalable Web applications.

    In addition, the new performance and scalability comparison includes new comparative performance data on .NET and J2EE for XML Web Services as well as distributed transactions handled via .NET/COM+ and J2EE/JTA. The performance study was conducted by the Middleware Company on the same hardware and test lab for all tested solutions, and the results are certified by the Middleware Company.
     
    View the report of the J2EE Petstore 2.0 vs. .NET Petshop 2.0 performance and scalability comparison.

    **************************************
    * TheServerSide Editor's Note - Oct 29 - morning
    **************************************
    TheServerSide.com is an independent unit at the Middleware Company and did not participate in the performance comparison (note also that the report was posted on TMC corporate site, not on TSS). We are simply linking to this report, much as onjava.com or javaworld would.

    TSS's role here is a neutral reporting agency - we believe that the facts should be known by the community -- even if the facts are sometimes not positive about J2EE.

    TSS will continue to announce news relevant to J2EE in a factual, neutral fashion - as you've all come to expect of us.

    Floyd

    ***************************
    *Editors note - Oct 30
    ***************************
    The Middleware Company has posted an FAQ on their site that directly answers many of the questions in this thread:

    http://www.middleware-company.com/j2eedotnetbench/faq.shtml.

    ***************************
    *Editors note - Oct 31
    ***************************
    The Middleware Company has posted a document on their site about their current thoughts about doing a rematch, based on feedback provided in this thread:

    http://www.middleware-company.com/j2eedotnetbench/rematch.shtml.

    Threaded Messages (759)

  2. The report says that the app servers were tested using Windows and Linux. I was wondering why they used Linux and not Solaris as the Unix operating system? Most of the "big" hardware out there is based on Solaris and most of the app server vendors have products that are performance-tuned for Solaris. If .NET gets to run on Windows then to be fair the J2EE solution should have had the chance to run on Solaris.
  3. There are really two main reasons for not running the J2EE app servers on Solaris. The first is because to "do it right" you really should run Solaris on a SPARC box. However, that means that you run into trouble because you are using two dissimilar hardware platforms. Now, making sure that the hardware configurations match exactly, becomes a much harder thing to do. The second reason is more mundane? setting up a lab with Solaris-SPARC machines in addition to lab of Intel boxes would have taken more time and money than we had. However, you are right, it would have been very interesting to see how Solaris worked.
  4. ouch
  5. "There are really two main reasons for not running the J2EE app servers on Solaris. The first is because to "do it right" you really should run Solaris on a SPARC box. However, that means that you run into trouble because you are using two dissimilar hardware platforms. "

    But isn't that the whole point - it's IMPOSSIBLE to run .net on anything but Intel? so, the best-performing .net will NEVER outrun Java on an os/390/db2 combo.
  6. And then, based on which solution was faster, everybody would be complaining about how it wasn't fair because of the difference in hardware platforms. Although it would be interesting to see performance differences on Intel boxes vs. Sun vs IBM mainframes, I don't exactly think that's what they were going for here.
  7. Maybe they should run everything on Sparc ... The M$ could show just how cross platform they are ... Oh ... wait ... Wintel wins again ...

    All in all another test built tuned and engineered for a Microsoft win.
  8. It is time for java/j2ee to just go away. c#/.net is really superior. I mean this is THE 1.0 VERSION of the framework. How long did it take sun to provide a some what tolerable jdk to work with? Well, about 5 years. MS has done it in about 1 and 1/2. WOW!.


  9. <quote>MS has done it in about 1 and 1/2</quote>

    Actually it could be longer than that, summer of 1999 when MSR worked with other Universities on multi-languages on CLR, so internally they must have a version that was running to show, so the date could be early 1998 or late 1997, the code name original was Project 7, see http://research.microsoft.com/project7.net/project_7.htm .
    But anyway MS has done a good job on .NET regardless of how much they have learnt from the previous work on the jvm when they were still one of the java vendor.

    Regards,
    Hun Boon
  10. I don't think they used JavaNCSS to get the LOC. My version of NCSS reports 8095 lines for the Java.

    Still a lot more than .NET, but I'm curious how the other was counted?

    Here's my command line:
    call javancss -all -recursive -out pets_all.txt "c:\my documents\tmc\bench\petStore"

    Here's the output I got (summaries only):

     Packages Classes Functions NCSS Javadocs | per
    -------------------------------------------------------------
        46.00 244.00 1179.00 8095.00 429.00 | Project
                   5.30 25.63 175.98 9.33 | Package
                             4.83 33.18 1.76 | Class
                                       6.87 0.36 | Function

    Average Object NCSS: 28.89
    Average Object Functions: 4.83
    Average Object Inner Classes: 0.00
    Average Object Javadoc Comments: 1.76
    Program NCSS: 8,095.00

    Average Function NCSS: 5.28
    Average Function CCN: 2.30
    Average Function JVDC: 0.19
    Program NCSS: 8,095.00
  11. I dunno, maybe you should READ THE FREAKING REPORT!!??

    Page 38


  12. All this makes it clear that M$ is monopolistic evil empire.
  13. Start making sense, please![ Go to top ]

    Please explain how "All this makes it clear that M$ is monopolistic evil empire." You are not making sense. Support your arguement with insightful well reasoned logic please.


    To quote Monty Python: "I came here for a *good* arguement..."

  14. LOC and where do they come from[ Go to top ]

    oops, that was unclear...

    Try changing the Title when the topic chnages, 95% of the post have the topic "New J2EE vs .NET Performance Comparison Performed"


    In response to...
    "I don't think they used JavaNCSS to get the LOC. My version of NCSS reports 8095 lines for the Java"


    I dunno, maybe you should READ THE FREAKING REPORT!!?? See page 38
  15. LOC and where do they come from[ Go to top ]

    I just read the FREAKING report's page 38 again. I can see the numbers for the LOC alright but it doesn't say what they used to count that. Maybe you can send us the except stating HOW they obtain the LOC.

    Thanks,
  16. You have to account for the configuration (over 2000 lines alone), but it still puts it only at about 10k.
    Note that high LOC is not necessarily bad. The question is really: how much _hand_coded_ LOC are really there? If out of 8k worth LOC 4k is generated as stubs/skeletons and what have you, that's not bad. The same goes for classes etc.. If I use (say) IDEA 3.0(Ariadna), and do my ejb genereration by a couple of shortcuts with not too much writing (and can do the maintenance in the same way), or XDoclet with @tags, who cares that behind the scenes it creates three or five other classes, bunch of xml stuff etc? I need to deal with 2 to 4 of them in my code, but I don't have to maintain them.. Fact that Sun did not promote automation of this process from the very start is just one of the evil Sun examples.
    Regards,
    Vlad
  17. This comparison is illogical and extremely misleading.

    I would be surprised if .NET were not faster than j2ee. After all, C# is a native Microsoft language running on its native platform. It has no portability and offers no options or alternatives to it users. It's a little like comparing efficiencies under a brutal dictatoriship and a bustling democracy.

    No doubt, things can run much faster in a dictatorship. But would you want to spend your life there??

    J2EE offers so much more than .NET in terms of openness, portability, vendor participation, private sector ingenuity, options, options, alternatives, etc etc etc. that it is inappropriate to compare it to the .NET world without broader inclusion of the impact and relative importance of many many other points.

    Wayne.
  18. Just to bite the troll, M$ didn't took 1 1/2 years to give us .NET, it took more or less since VB 1.0 to get a grip of what a programming language should be (C++ don't count, it was already there)... Well, perhaps I'm too radical... Let say since ASP came to market.
  19. sigh!
  20. Ack, the J2EE performance numbers don't look good compared to the .net version. At least TSS was honest in their reporting. I hope the performance improves. I guess the only thing that could improve things in the short run is to have the tested app servers utilize the 1.4 JDK which makes improvements to the IO subsystems (I think). Would be nice to see a benchmark using that JDK. But, still, it's a little disheartening to see the .net setup working for cheaper and faster than the J2EE implementation (but at least not as badly as the origional petstore).

    -Chris
  21. If you understand the underlying facts here, I think you will change your mind if you believe that .NET outperforms the industry leading J2EE Application Servers. Here it is in a nutshell:
    1. Pet Store is NOT an example of a real-world
       workload, "best practices" application
    2. Isn't it funny how Microsoft hosted this benchmark
       in their labs, funded the testing, and paid for the
       travel expenses of the Middleware Company employees???
    3. Why was Microsoft the only participant in this so-
       called "independent" benchmark. The other vendors
       were not even invited - very strange!
    4. What does Microsoft know about tuning J2EE Application
       Servers or J2EE applications for that matter?
    5. Why did Microsoft choose the configuration that they did
       and why did they choose to run the benchmarks NOT
       utilizing obvious performance enhancements which would
       have increased the J2EE performance?
    6. I will bet that J2EE Application Server vendors have
       their own comparisons which tell a much different story.

    This benchmark is a clear ploy my Microsoft to make the .NET infrastructure look good compared to J2EE. Why not, they are losing business each day to J2EE Application Servers and Intergration Application Servers. J2EE performance is where it is at and IS the future - don't be fooled by the desperation of Microsoft.
  22. Tom:

    it is only clear that you haven't read the faqs or other posts

    1. Pet Store is NOT an example of a real-world
       workload, "best practices" application
    According to sun, it is:
    From http://java.sun.com/blueprints/code/index.html
    "This sample application shows how to use the capabilities of the J2EETM platform to develop flexible, scalable, cross-platform enterprise applications."

    2. Isn't it funny how Microsoft hosted this benchmark
       in their labs, funded the testing, and paid for the
       travel expenses of the Middleware Company employees???
    If BEA, IBM, or Sun had made the same offer (and TMC asked them), I'm sure TMC would have accepted

    3. Why was Microsoft the only participant in this so-
       called "independent" benchmark. The other vendors
       were not even invited - very strange!
    The other venders were invited, they refused to participate

    4. What does Microsoft know about tuning J2EE Application
       Servers or J2EE applications for that matter?
    MS didn't touch the J2EE apps, TMC hired people (whom MS didn't pay for nor select) to tune

    5. Why did Microsoft choose the configuration that they did
       and why did they choose to run the benchmarks NOT
       utilizing obvious performance enhancements which would
       have increased the J2EE performance?
    See 4.

    6. I will bet that J2EE Application Server vendors have
       their own comparisons which tell a much different story.
    Actually they do not, except for oracle who claim that their version is 28x faster but refuse to detail the test and will not release the code so that the test can be validated.


    MS is capturing additional market share, their stock price is up 10% this week, and they have $40 BILLION in the bank...you got a funny definition of "desparate"...
  23. Michael -

    <quote>
    "This sample application shows how to use the capabilities of the J2EETM platform to develop flexible, scalable, cross-platform enterprise applications."
    </quote>

    Anyone who works with J2EE knows and has always known that the petstore is not a real-world application. The key in the above phrase is "how to use the capabilities of the J2EETM platform" - it shows many of the APIs inherent in the J2EE platform. You can use them to create "flexible, scalable, cross-platform enterprise applications." The petstore is not such an application. Even if Sun meant for it to be. If it amuses you to continue to think it is, please continue to do so, but your arguments only hold sway with yourself and Microsoft.

    Ray
  24. Michael, this is a very interesting take on this situation. I won't elaborate on why some of your statements are incorrect. My advice to you is "stay tuned", you may change your mind in the weeks or months to come.
  25. Whoever mentions the future *nix implementations of .NET should take a DCOM book and read through. Actually, I happen to have one of those: Professional DCOM Programming, 1997 Wrox Press (ISBN 186100060X). It talks about the work-in-progress Unix implementations of DCOM and how DCOM will become the ubiquitous distibuted component model in the near future, etc. etc.

    Err... Hmmm... So, how many of you have worked with DCOM on a platform other than MSFT's?

    It's basically the same lullaby and obviously it still works for some people ;-)
  26. .Not is not portable - end of discussion, end of story. It immediately hits a known ceiling before a line of code is written. This discussion can end now.
  27. That's the fourth time you've said this on this thread. We understand you're upset, just have some chamomile tea or something.
  28. Ken, it's not that I'm upset, it's that I see it as stuipd that this is even still being debated.
  29. 2. Isn't it funny how Microsoft hosted this benchmark

    >> in their labs, funded the testing, and paid for the
    >> travel expenses of the Middleware Company employees???
    >If BEA, IBM, or Sun had made the same offer (and TMC asked >them), I'm sure TMC would have accepted

    Think in this way: if it is Oracle which sponsor the benchmark, and the results show Oracle App Server outperform .Net, what do you Microsoft will think about the benchmark?

    I suppose, first of all, you'll look at the details for the fact of how the benchmark was done. To see who are involved, what are the rules, how does the Oracle sponsored company do to the .Net PetShop etc. And then, you'll find the result doubtful, because the benchmark is sponsored by the opponsiting party.

    If the benchmark is done by any independent party, and is announced in advance, then it is a totally different story.

    >>3. Why was Microsoft the only participant in this so-
    >> called "independent" benchmark. The other vendors
    >> were not even invited - very strange!
    >The other venders were invited, they refused to participate

    It is perfectly normal for the other Java venders to decline to join, because the test is done in Microsoft's place under MS's rules.

    Besides, we don't know the details how TMC invite the vendors. Even I am a techie, I know the basic of communication and there are plenty ways to invite a party but making them not to accept the invitation.

    >>4. What does Microsoft know about tuning J2EE Application
    >> Servers or J2EE applications for that matter?
    >MS didn't touch the J2EE apps, TMC hired people (whom MS >didn't pay for nor select) to tune

    And the result is, TMC claimed they are lacking of time and money and cannot rebuild the application from scratch nor optimize the Petstore properly.


    >>5. Why did Microsoft choose the configuration that they did
    >> and why did they choose to run the benchmarks NOT
    >> utilizing obvious performance enhancements which would
    >> have increased the J2EE performance?
    >See 4.

    If you have read the messages in the above, you may come across some doubts on the rule of benchmark. Take an example,
    "The new implementation has also been extended with support for distributed XA-compliant transactions across two physical databases."
    Obviously, it is MS which decides the rule of game and the rules are set to amplify the strength of certain MS proprietary technology (i.e. distributed transactions via COM+). It is also MS who provide the lab and machines. If MS is serious in getting a meaningful benchmark, they should provide sufficient fund to TMC to do whatever they want, e.g. buy a Solaris, build the Petstore from scratch.


    >>6. I will bet that J2EE Application Server vendors have
    >> their own comparisons which tell a much different story.
    >Actually they do not, except for oracle who claim that their >version is 28x faster but refuse to detail the test and will >not release the code so that the test can be validated.
    >


    >>This benchmark is a clear ploy my Microsoft to make the
    >>.NET infrastructure look good compared to J2EE. Why not,
    >>they are losing business each day to J2EE Application
    >>Servers and Intergration Application Servers. J2EE
    >>performance is where it is at and IS the future - don't be
    >>fooled by the desperation of Microsoft.
    >
    >MS is capturing additional market share, their stock price >is up 10% this week, and they have $40 BILLION in the >bank...you got a funny definition of "desparate"...
    >
    I think the stock price and deposite of Microsoft got nothing to do with the performance of .Net and J2EE. To my understanding, you mean:

    because "MS got a lot cash and its stock perform well"
    so "it doesn't need to market itself" and/or "it is not lossing business to J2EE"

    The cause and result are logically irrelevant.

    If anybody want to know more about MS Business, please check
    http://news.com.com/2009-1001-964248.html
    it talks about 11 key business areas of MS, none of them is middleware/EAI.
  30. MS is serious in getting a meaningful benchmark


    That's why they pay for it! Right?

    Companies don't care about fair benchmarks. That's why we have a concept called marketing. Marketing isn't only promoting "good products" and has tens of different forms. How hard is to get *that*? That's why you should go with a technology beyond the companies, where you have choices. There's no meaning to be loyal to a company unless you work for it or you have a pile of shares.

    You guys are really from a different planet. We should submit your messages as a proof of extraterrestrial life form.
  31. this really hurts ...
  32. It is not surprising that .NET will outperform J2EE, since in many regards what is being compared is the performance of C# vs. Java. I would assume C# would be inherently faster than Java on Windows/Intel.

    This is the price Java pays for portability. Portability and interoperability between J2EE implementations also leads to more app level coding, as there are many factories and configuration details that are part of the open architecture, so it is not a surprise that the Java petstore is a more complex application.

    The benefit of J2EE is a technology and an industry in which many vendors can participate, and in which developers have a choice. The alternative is a technology dictated by one company, where developers are at the mercy of Microsoft. Sure, .NET may be cheap today, but Microsoft doubled its profits last year by switching up its licensing models on unsuspecting customers who were locked in and had no choice.

  33. All said and done, I'd say Floyd and gang should be commended whole-heartedly for injecting a largely unbiased outlook into this whole petstore madness. Identifying *your* weaknesses goes a long way in dealing with competition. If taken in the right sense, this report can act as a catalyst for all those lethargic vendors out there to improve their offerings. Know thy enemy...

    --Dilip
  34. First, let me start by saying I am a J2EE person and have been for about 4 yrs.

    I agree the TSS should be congratulated !

    This should be a reality check. The arrogance on this site about the .net stuff has been unbelievable ! I was getting very concerned because it reminded me of the OS/2 vs NT debate and the arrogance of the OS/2 zealots ( I was an OS/2 programmer for 4 yrs) Where are those OS/2 zealots now ???

    But having said all of that, I would like too see a Technology to Technology comparision, NOT a best practices to best practices. It is unclear to me that EJBs are a part of MY best practices. I find that EJBs are usefull for only a few of the projects that I've worked on. (EJB 2.0 with local interfaces may may change that but are local interfaces reall better than plain old java objects ? )

    Let me state. I TOTALLY understand the "Best Practices" to "Best Pratices" comparisons and appreciate the VALUE of such a comparisons but...

    I want to see a technology to technology comparison !

    0) What is SUN's best practice, and MY best pratices are two TOTTALY DIFFERENT things.

    1) Am I crazy or do ejb === MTS objects.

    2) Why can't we have a true technology to technology comparison ? java to c#, asp to jsp, dao/ado to dao/jdbc.
    Implement the apps using the equivalent technologies. In short APPLES to APPLES comparison ( actually it would be Macintosh to Golden delicious comparison :) ).

    3) Side note, why wasn't NT used for the J2EE platform ? Is linux really faster ?

    If we can have a technology comparison, we can let the customer decide what their best practices are !

    Would EJB 2.0 change these numbers ( use of local interfaces ).



    thanx,
    and please, no flames !
  35. I agree about technology to technology comparison. I'd be interested in any language level (java vs C#) performance comparisons. Comparing intensive algorithms (sorts, etc) to get an idea of useful work vs language overhead (object allocation etc).

  36. J2EE/.Net performance ratio in the benchmark seams to be close to Java/native code performance ratio.

    I think that key performance difference comes from platform, not from application code. Most of the Microsoft platform, or at least most of the performance critical parts, are native coded (c/c++). J2EE platform is mostly Java coded.

    We need mach faster JVM or more native code in platform (like in Standard Widget Toolkit).

  37. How about inverting the question ?

    Suppose you have US$XXX,00 and want to run a web based pet store for X years.

    What can you do with .NET ? (performance, availability, ...)
    What can you do with J2EE vendor A?
    What can you do with J2EE vendor B?

    This scenario should include 3 month user interface revision, for example. Operating System and hardware maintanance costs, etc.
  38. if MSFT doubled their costs? Hmmmmm, still seems like a deal to me.
  39. Some of what you mention is really a non-issue. Yes, there is a comparison of C# and Java, but, realistically, as each compiles down to byte code that is later compiled to machine specific code, I would say the test is testing the core VM (or CLR, in the case of .NET) more than the language.

    Java does, indeed, have more portability, which makes it a great choice for operations which need to deploy to a variety of platforms. With current applications that have to be multi-platform, you almost have to choose Java if you do not want to rewrite for each platform. Even then, there are often little tweaks due to differences in the JVM. With the open source community working on .NET for Linux, it is a possibility of tests on a variety of platforms in the future. .NET is not there at this time, so it is a moot point.

    As far as Java being a technology and an industry with many vendors, I will agree partially with the statement. Ultimately, it all goes back to Sun, however, which means .NET versus Java (at least on this level) is a matter of one giant or another. It is certainly possible Microsoft will lock people in, but, with open source initiatives it will get harder and harder to lock people in.
  40. There are a couple obvious reasons for the performance difference, but C# is not really faster than Java. The CLR is faster than the JVM for I/O in particular because it uses the native OS calls. Think of perl and I/O. This could be one area for improvement in java, and I'm thinking of an IO library something like what IBM has done with the Eclipse UI elements, but as you noted, portability is lost. Another thing to note is that the Microsoft Petshop 2.0 is not really implemented in C#, but C# is used in conjunction with compiled C++ COM+ objects, which handle much of the difficult functionality in your EJBs. Again portability suffers, but there isn't really a clean and easy way to do this with Java when it is necessary. JNI is not my idea of fun, and can't help in the LOC contest either.
  41. RE "This is the price Java pays for portability."

    True, but in many cases even the portability is not there. We use BEA and Websphere servers, but have found even third party "Standard J2EE" apps will not run on one or the other. Developers tend to use specifics, whether from IBM or Microsoft. Trust me, it is just as easy to get 'locked in' with IBM as it is with Microsoft. Either way the 'costs' are high... Now, your next reason is a very good point - even if just in theory, competition is good and does spark new products and keep prices in check!
  42. <quote>
    This is the price Java pays for portability. Portability and interoperability between J2EE implementations also leads to more app level coding, as there are many factories and configuration details that are part of the open architecture, so it is not a surprise that the Java petstore is a more complex application.
    </quote>

    Unfortunately with the advent of projects like Ximian's Mono, we may end up losing Portability as an argument as well! It's only a matter of time before the CLR (Common Langauge Runtime) is ported to FreeBSD (I believe this is already in the works), Linux (nearly complete), Solaris (not sure about this one), etc. etc.

    Java will not be able to tout portability and choice as it's advantages for ever. It seems like Microsoft might actually be doing something right with .NET; much of it is standards based, the many of the parts that are not the specifications are available for third-party implementation.

    ACK!
  43. Unfortunately with the advent of projects like Ximian's

    >> Mono, we may end up losing Portability as an argument as
    >> well! It's only a matter of time before the CLR (Common
    >> Langauge Runtime) is ported to FreeBSD (I believe this is
    >> already in the works), Linux (nearly complete), Solaris
    >> (not sure about this one), etc. etc.

    For complex applications, I am not sure it is as simple as porting the CLR.
    Even if that is successful, then all they have ported is an equiv of J2SE.

    For enterprise applications, I am not sure that portability will be achieved in a platform that was not designed that way from the beginning - and that true portabilty will come about without stewardship of the standards.

    Where Java and J2EE has succeeded where other standards initiatives have failed (prime example CORBA) is that Sun have not only embedded community engagement in the standard process (JCP), but they have also implemented a solid certification process (and more recently a performance comparison). A solid certification process means that there is little question over portability uncertainty. Remeber here that in the case of the CLR and the .NET framework, Microsoft are far from interested in portabilty, so who will supervise such a similar certification?

    Rememver also, that while the CLR stands a much better chance than COM, there was a lot of activity initially, porting COM/COM+ to *nix...

    -Nick
  44. Thanks for posting these results. It is important for the J2EE community to realize we are facing serious competition here.

    On another subject, it is a reasonable assumption that the two J2EE servers used for this study are Weblogic and Orion. So it is userful to know which one is A and which one is B, since A drastically outperformed B.

    The clue in the report is that A would run on JDK 1.4 and B would not. Any Weblogic / Orion users can tell us which is which?
  45. Ouch....
    My thought was A was WebLogic (hint: JRockit VM was used for the WebServices test), and B was WebSphere (hint: IBM's JRE for both tests). I may be wrong though...
  46. B was definitely Websphere. The comparison mentioned using the integrated Apache server for images.
  47. I'd also be interested to see this running using CMP2.x. This should result in a massive reduction in LOC, I'd have thought, as well as improved performance.

    Also, I'd have liked to have seen JBoss put into the mix, as having a zero licencing cost would have put it on a level playing field with respect to .NET offerings.

    Lastly, I don't think using Java is purely about performance. We had a fairly large site running on ATG Dynamo on a cluster of NT boxes, and the processor utilisation barely touched 5-10%. Personally, I think it should be possible to write Java applications in very few lines of code these days given Struts and the like, so the savings come in development, maintenance and flexibility:we're looking at running WAS on a mainframe -- you can't do /that/ with .NET.
  48. I would assume they are Websphere and Weblogic just based on licensing costs ($40k) but I don't know the licensing costs from Oracle on Orion.

    Here's the facts that I find most interesting about this benchmark:

    - 2096 lines for .NET verses 14,004 lines for J2EE.

    - 2 man-weeks verses 10 man-weeks to tune for performance.

    - Painfully better price/performance. In particular, even if you take away the licensing costs by using something Jboss and assume you get the higher performance among the J2EE servers tested (59 TPS), you still have a price/performance of approximately $625 - double the amount of the new .NET server.

    Clearly there needs to be some innovation here - at this point the arguements for J2EE are reduced to two: (1) choice and (2) portability - both solid reasons but performance needs to be answered.

    Lastly, I'd ask why they didn't use the same database for both applications. Seems to me that the database is a big unknown in this equation.

    Damian
  49. Maybe the different database plus EJB(always reflect millions of records to objects in memory) caused the poor performance.

    But, i think there are 2 things people can do to favor java:

    1)develop a better designed JVM or special hardware-targetted JVM to improve the performance.

    that is how MS always do(i can be sure windows CLR will be much different from freeBSD CLR). if we can not ran away from OS, then we can choose linux ,etc.

    ask intel or AMD for performance tips to have a full usage of x86 tech(intel already offer optimized c/c++ compiler). MS can do, why not others?

    2)improve database layer design and perofrmance.
    that is always the pain.
  50. AS LONG AS EJBS ARE USED j2EE WILL NEVER OUTPERFORM .Net in its speed .For the moment there is no mature alterbative to Ejb ...can SUN gurus find another solution.. I am sure the can if they start listening and reading about performance test ...hope it will be a good eye opener
    Faisal
    UK,Birmingham
  51. and if u RE DO this test using Clinton Begin's JPetstore2.0
    www.ibatis.com I ,am sure one hundred per cent, the result will be completely different.
  52. I agree. JDO is the only choice as alternative to manual JDBC coding now and this is a GOOD choice. It is pure objects persistence layer without any EJB overhead. Most of "really thinking" professionals recommend it now. BTW., most of JDO vendors say they have scalable distributed implementations. And all of them have free editions of there product.

    Igor Shabalov.
    Exadel JDO
  53. Simple economics are not supporting investment in J2EE performance enhancements by the major vendors (perhaps until now). Lets face it, it was and is in SUN's interest to push big iron, BEA/Oracle/IBM's are motivated by selling CPU instances. What possible motivation have these vendors had in investing in this area? Of course, up until now, the marginal spec.org leapfroging of the j2ee vendors have been mere marketing levers and discounted by purchasers.

    The order ot magnitude improvement by M$ cannot be ignored by any purchaser.

    Now, with this wake up call, its time for the entire industry to focus on throughput and scalability.

    (Note - how many times this site goes into garbage collection and freezes for a while)
  54. I would have to say that Server B was WebSphere. One of the reason is page 52 "Web Server used ships as part of the application server tested...". WebSphere ships with IBM HTTPD server (Apache). Plus why would one use the IBM JDK with Orion (or any other app server)?

  55. I would like to particpate in the next round represeting Java J2EE w/o EJB.
    I have P&T background, and have writen amog other things a Zero Copy DAO using a RowSet.

    Please contact me, vic at baseBeans dot com, should you want to showcase how J2EE is better/faster/cheaper than .NET (at least for now).

    .V
  56. The fact is that J2EE got beat so bad that it won’t make much difference if the Java-PetShop is re-designed to a different spec.

    Stop the excuses and face the truth.
    I've used MS tools for ten years now. One thing I've seen over and over again is that MS is already fiding ways to make their products better. That's the one thing the J2EE vendors need to start doing besides all of the marketing that makes them sound like their products are the best.

    I could always tell it was a joke, but it's not easy making these brain -washed J2EE developers see the truth.

    If .NET can beat J2EE this badly, J2EE will be non-existent in 3 years when the .NET era reaches its peak.

    All of the J2EE vendors are after Microsoft. If they ever get their way, they'll be going after each other. This is all about money. They're playing on everyone's fear that Microsoft will take over.
    Give me a break.
  57. Prophecy[ Go to top ]

    I could always tell it was a joke, but it's not easy

    > making these brain - washed J2EE developers see the truth.

    I can say the same thing for you. How objective this would be. Your truth is your concern. As somebody else pointed out you cannot fight against the faith.

    > J2EE will be non-existent in 3 years
    > when the .NET era reaches its peak.

    Amen!

    You forgot the part when the moon shine turns into blood red.
  58. New J2EE vs .NET Performance Comparison Performed[ Go to top ]

    slim slim, "The fact is that J2EE got beat so bad that it won&#8217;t make much difference if the Java-PetShop is re-designed to a different spec.

    Stop the excuses and face the truth. "

    You don't even get it, you are so narrow-minded and blind to m$.

    I can't believe TSS put out this "study." I just can't believe it. It's amazing.

    This study compares Java technology at its WORST, which is running on wintel platform - who in their right mind would do that. They are used to "old era" comparisons where you make everything the same, in this case, the platform and database. but the whole point of JAVA is that you don't run it on windows guys! Hello? it also compares windows at its best, which is running at a low level intimate with the OS and platform and hardware - of COURSE Java is going to loose!

    Here is what I propose as a FAIR test:

    1) Run m$ on the fastest and most powerful and huge wintel platform you can find, and pick a database, even sql server.

    2) Then, set up the same petstore on the fastest mainframe you can find running a JVM and WebLogic, and run it on a cluster of them.

    The, we'll have a fair test.
  59. New J2EE vs .NET Performance Comparison Performed[ Go to top ]

    BTW, does sanyone know what JIT was used?
  60. would the main frame cost same as the wintel machine ???
  61. Lost a LINK[ Go to top ]

    There was a great link somewhere here, I can't find it.

    It was to a site that had bencmarked store, action etc. in Java.
    If you know of a this benchmark link, please send it.

    Thanks,
    Vic@baseBeans.com
  62. Lost a LINK[ Go to top ]

    Do you mean JPetStore?
  63. Lost a LINK[ Go to top ]

    http://www.dreambean.com/petstore.html#Microsoft

    This is a VERY INTERESTING analysis of the comparison.

    Putting everything else aside, it points out somethings that are not a matter of opinion and absolutly irrefutable.

    For example,

    1. the J2EE app has code that is never referenced--yet included in the LOC

    2. "Many EJB-specific methods (e.g. setEntityContext, ejbActivate, ejbPassivate) that weren't used could have been placed in a superclass. This would cut away a large part of the EntityBeans."

    3. "Some utility methods, such as getConnection(), are repeated in each and every bean. Putting these in a common utility class would have removed a substantial amount of LOC."

    4. all methods in CatalogEJB run as TX_REQUIRED even though some are clearly Read-only in nature.

    5. JNDI homes are being looked up even when they're not used. See ProductROEntityBeanBMP for example, which looks up the Item home but never uses it.


    Let's stop here. These findings are not subject to interpretation.

    Having code that is never called, yet included in a metric for "lines of code" is incompetence. Having all methods in an EJB marked as TX_REQUIRED even when they are Read-only in nature, is a rookie mistake. However, when this rookie mistake is made by TMC developers (ie. J2EE experts)...


    It's an uphill battle to question the findings of this benchmark unless you consistently draw attention to these issues discovered by Rickard.

    I would like to see TMC address each issue. I would like to see what their answer is to the question of "why do you have code in the app that is never referenced?"

    Do they deny it is never referenced?

    -Michael.
  64. Better yet, spend the same amount of money for hardware to run each on.
  65. Assuming that people look at the designs and agree they truly are comparable (are Entity Beans and ADO.NET really are comparable?), what can we learn from this? Are the J2EE best practises used in the examples truly best practises? Or with another approach like JDO and Struts be better? One thing that I've read over and over again is that Entity Beans are for course grained interfaces to the datastore. Is this an example where Entity Beans are overkill? If they are overkill for this example, what should replace them?

    For reference, here's a link to the previous discussion:
    http://www2.theserverside.com/discussion/thread.jsp?thread_id=9797

    What would be great is if someone (perhaps TheServerSide) could put up an "Petstore shootout" site that allows all the major Java solutions to compete. With such a site, we could all get a better idea what truly works best. In order for the results to be valid, some guidelines like hardware, database type, and design constraints (e.g. MVC, database independence, etc) would need to be fixed.
  66. My reaction was: ok, the results suck for J2EE, but what is the lesson learnt after such an effort? We all know there is a huge amount of performing J2EE enterprise applications out there. Maintenance, stability, reliability are as importnat (or expensive) as performance and one-off development.

    I think it is significant to see that the 10 man-weeks required for configuration were also spent for evaluating the different options available out there, meaning that there is choice. It is definetly easier in the "one microsoft way", where you have to do things in one way and wait for MS to fix things if they don't work or come up with new ideas.

    For the front-end, the lines of code are only comparable if some framework would have been used. "Pure" J2EE is only the infrastructure, but many useful frameworks were created. Nobody does these things from scratch today, we all use pre-existing frameworks when writing enterprise apps.

    Otherwise the difference is indeed significant and it makes me think that of course the C# runtime is deeply connected with the OS core, and purely on Windows-performance criteria Java will never be faster, just because MS is the manufacturer of the OS.

    Companies who want pure performance, can afford state-of-the art data centres and J2EE is leading there. Companies that want cheap solutions, can go with JBoss or Perl. The battle will probably be somewhre in the middle.
  67. then, you mean the authors of JVM on linux haven't read the LInux kernel source code? :P
  68. Hi

    The results are disappointing, but lots of thanks to the serverside for being honest.
  69. Benchmark performance numbers can (should) always be viewed through the prism of "what's important for your specific application".

    If you have ever run performance testing on app/web/database servers using LoadRunner then you can appreciate that this was a well thought out and executed effort.
       
    If you have worked with multiple J2EE components and platforms your own experience says that different appserver/jre/service/version combinations have widely varying performance characteristics.

    There are lots of tweaks and combinations that could have been tried but in all projects (except those sucking up taxpayer monies) a point is reached where you have to make a choice and move on.

    The data is the data.
    The test methodology is one approach.
    There is no stated conclusion so draw your own.

    I think this benchmark is both fair and good and the results are enlightening. J2EE for complex intergalactic-scale applications. Consider .Net for anything else.

    If you disagree the results then you are free to spend your organization's time and money on your own tests.
  70. Please stop this f*****g nonsense. This is not about the time used to optimize the petstore, or how difficult it is to configure J2EE app server.
    The source code contains actual 'de-optimizations' - Huge queries are constructed by appending small pieces of strings, effectively restricting oracle from optimizing these repetitive queries. I thought that even 'freshmen' know how to use prepared queries - but obviously TMC guys haven't yet reached that 'high' level in programming basics.

    Every resultset and statement is painfully closed in separate method? Don't u guys know that disconnecting from the datasource (which is what u always do) automatically handles such tasks.

    DAO objects are littered with ctr+c -> ctr+v style code. Not even the slightest effort has been made to move common code to base classes.

    Connections are acquired in forever loops with 20000 tries before given up - what kind of moronic crap is that?

    hundreds of posts here wonder what the server B was, while you could clearly read it from the source files. (I'll save your time, there are weblogic.jar files everywhere..)

    Please, if u have any knowledge of programming at all, you'll immediately see that the test code is a piece of s**t. Of course u can buy this M$ fud but don't tell your bosses that your decision was based on technical expertise or knowledge.

    Lawsuit? server manufacturers A & B - you have good chances to win!
  71. This is the most sensible thing a J2EE developer can do rather than comming here and whining.

    http://sourceforge.net/projects/petsoar
  72. Thank you for the link!

    I see Rickard Oberg is on the team. (Excuse me Rickard for the spelling of your name but this site do not display diatrical characters correctly.)

    Rickard is regarded by some (including me) as one of the best programmers in the world. He also is a specialist in J2EE. It will be funny to see what excuses and explaining away the TSS community will use this time as the petsoar team will surely fail too.

    No matter how good you are you can not do better than your tools allow you.


    Regards
    Rolf Tollerud
  73. <aside>
    I dont want to diminish Rickards fame, but calling someone "one of the best programmers in the world" is among the rather stupid things in the world, IMO. If I were him, I'd rather do without these kinds of praises.
    </aside>
  74. Well.. I think the source is this:

    http://xdoclet.sourceforge.net/

    Look at the end of the page. (right frame.)
  75. I would say less than 3,000 lines of code is much easier to stablize than more 14,000 lines. Let's dont kid ourselves with some theorectical benefits java has. Even if the demonized MSFT doubles the prices on software, it's still a lesser demon than what our beloved leading J2EE app server vendors. For $15k/cpu + 20% annual license cost, as developers we are better off to work for companies that pay big bucks for expensive software, which makes our jobs kind of valuable. The scary question is how long does it take for a cost-concious manager to figure out the financial stupidity of J2EE that is technologically advanced, portable (for what purpose?) ...
  76. The Serverside reported on a paper published by Rice http://www.cs.rice.edu/CS/Systems/DynaServer/perf_scalability_ejb.pdf a few months back. This paper clearly showed that Servlets/JDBC is on the order of 10X faster than BMP and 2x faster than CMP with local interfaces.

    I think the complexity of CMP 2.0 with relationships is a worse problem than performance, since if you read the Rice paper (no pun intended) you see CMP with Local interfaces is not bad. The real problem is that in most companies, developers just get lost in the muddle of deployment desciptors and proprietary database mappings for CMP. Without a tool like EJBGen or XDoclet, you are lost. Projects get bottlenecked in the complexity.

    In most development projects, the database entities we work with are not "components" (ala entity beans) and we shouldn't have to invest the time and energy to package them as components for use by our own applications (which is what EB makes us do).

    I see JDO as the logical way out of the Entity Bean complexity/performance problem. JDO seems to be what developers need for the 90% of cases when you just need to get to the data.

    So while there has been a lot of negativity regarding the results from TMC, it seems to have clearly pointed the finger at entity beans, not at J2EE as a whole. I would like to see the Rice guys redo the study using JDO ... I think the results would be dramatic.
  77. This benchmark is going to have a cosmic impact.

    I think the most sensible thing to do for Sun and al. is to kill (in one way or another) the EJB technology.
    EBJs are cripling J2EE in term of complexity and performance, thus completly failing to reach their initial goals.

    In addition, Key APIs coming out of the JCP recently are not designed with ease of use in mind.
    For instance compare the JAX-RPC API with other toolkits. In some case (e.g. DII), doing a simple web service invocation require five to ten times more lines of code with JAX-RPC.
    This trend should be reversed in order to reduce the programming complexity gap.

    Phil

    PS: Congratulations to the guys that have done this benchmark. This is quite impressive.
  78. Agree 100%. Dump the EJBs and restore elegance and simplicity of J2EE.
  79. I'd be more interested to see comparison with iBbatis JPetstore

    -- Igor
  80. It appears that different database servers were used for the two tests, and that XA was added to the mix. What were the results with no XA and both running off the same Oracle database server? (apples to apples)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  81. I'm with Cameron: this test means nothing until both of them are run with the same database. Otherwise, this could all be 100% database performance and have nothing to do with J2EE or .NET.<p>
    The database will have to be Oracle, since the JDBC drivers for SQL Server are slow.<p>
    I also think the use of distributed transactions ought to be dropped, since this is not a typical design for stores that need high performance. EJB is a very questionable choice in general, and numerous benchmarks have shown that using straight JDBC in Java classes has dramatically better performance than EJB.<p>
    It would also be nice to see some open source choices in here...
  82. Perrin, this test does mean something. Sure the backends are different, but almost anyone will agree that Oracle properly and expertly tuned, _should_ outperform SQL Server, so the differences in backends do not equate to the difference in results.

    Also, if the cause was 100% just the database performance and have nothing to do with the web/appservers, then how do you explain the distinct and very large disparity between J2EE Application Server A and J2EE Application Server B, both tested using I believe Oracle 9i. Obviously, the implementations of the web/app-servers mattered a big deal in the results.

    I do agree that maybe EJBs contribute highly to the cost, but since EJBs are pushed by Sun and J2EE for high level of scalability and performance in distributed systems, and it was compared against .NET implementation using COM+, I believe the comparison is still fair.


  83. I was exaggerating about the differences being 100% database-related in order to make a point, but I don't see how anyone can say it doesn't matter that the databases were different. I would not agree that a well-tuned Oracle sever will be faster for all queries than SQL Server. Different databases are better at different things, and run better on different platforms.

    Just because Sun says we should use EJB doesn't mean we have to do it. It wouldn't be the first stupid thing Sun ever said. IBM has articles on their site comparing performance with and without EJBs, and instructing people to avoid them if at all possible.
  84. Three thoughts:

    1. A quick check against list pricing shows that WebLogic is US$10,000/CPU, which would be $40,000 for the 4-CPU system, so it is conceivable that server "A" is WebLogic. Note that WebLogic's TM logs XA transactions to disk, so there is a flushed write for each transaction state change, which kills performance. I think it's a valid test to show (it's good information), but they should have shown both XA and non. Same for .NET. It would be nice to see what the cost difs were, and what the vendors' explanations are for those.

    2. The real problem with server "B" is that it doesn't work. The document states: "J2EE Application Server B was unable to sustain peak throughput for more than 2 hours on the distributed transaction benchmark. It was unable to sustain any throughput beyond 4 hours, destabilizing over this period of time to the point of failure." Frankly, when it comes to working or not working, that's a lot more important than performance, and server "B" (for whatever reason) gets a failing grade.

    3. Did Microsoft agree to allow the results to be published before or after the tests were run? I'm curious for several reasons. First, it's "illegal" to publish performance results of .NET with Microsoft's permission, which was obviously granted. Second, what were the involvements of the vendors (including Microsoft) and why weren't those disclosed? For example, was this project funded? By one or more of the vendors involved? Disclosure would be good.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  85. Correction: "First, it's "illegal" to publish performance results of .NET without Microsoft's permission, which was obviously granted."
  86. In the beginning of (
    the M$ .Net Pet Shop 2.0 web page), there is a link to Middleware Benchmark report page. So it is obviously that it get M$ permission or even help to publish the .Net Petshop performances.
  87. "Microsoft Petshop 2.0 is not really implemented in C#, but C# is used in conjunction with compiled C++ COM+ objects, "

    This is not true. I looked at the code and it is all C#, even the code behinds the aspx pages.

    You must've assumed that COM+ requires C++ programming - which is not the case as the COM+ services are part of .NET as the System.EnterpriseServices library.
  88. u make mistake. COM+ Services are written using C++ (In M$ term: Unmanaged code), COM+ components that use COM+ services are written using C#(managed code).
  89. Correct, I meant components - which is what is being used in this application.
  90. Well, he might have meant that the hosting environment for COM+ is written in C++ ...

    But even so, COM+ was not designed for .NET like J2EE ( esp EJB ) was designed just for Java. Instead COM+ was written as a hosting environment for C++/VB COM objects. They then include hooks in the .NET framework to host .NET components inside of COM+, but this also comes with overhead ( Though I've heard that .NET components do not have to go through the COM interop layer to be hosted inside of COM+ ).

    So, while many are saying that EJBs ( Entity beans ) hurt peroformance, and that the results might be different if ejbs were removed all together ( or possibly use session beans )... if that is done then it would also make sense to remove COM+ and its high transactional cost ( to make a COM+ call means that I'll need to leave my process address space and incur that marshaling overhead ).
  91. Don't be childish...be gentle.
    what about $un?
  92. I can't believe this discusson is still going on.

    M$ only runs on wintel, and isn't portable. End of discussion, end of story, no need to debate ANY more guys. It's over! You start out with a KNOWN ceiling, and you know from day one you will NEVER be able to scale your app past a certain point. WHO would do such a thing? We've all seen the projects that "will always be small, and never grow." Ya right! As if. Those don't EXIST. So you're looking at an eventual REWRITE anyway, so why bother? Just start out with the right technology to begin with.

    Let's see the test re-run with the J2EE app running in a clustered DB2/OS390 mainframe environment or may on a Cray supercomputer, that's a more accurate demonstration of what J2EE is capable of, NO ONE would ever DREAM of running a good J2EE app in a winbloze environment. Get real, and wake up and smell the Java people.
  93. Not to be "pro" MS or anything like that, but that statement's ultimately false. Windows 2000 AS (Advanced Server) has clustering capabilities that works well with the web/IIS enviroment. Dell and Microsoft both are running .NET web servers, and they have a very high hit ratio.

    Clustering offers another advantage in that your uptime get considerably better (since if one box crashes, the others will take over those requests).


    There was also a post on ADO.NET being equivalent to JDBC. It is, and it isn't.. depending on how you implement it. It's true with ADO.NET that you build up the dataset/items using SQL, but there is alot of functionality that is different from the old ADO model that lends itself to a more OO approach to using the data.

    I think the TMC did a pretty good job though. There are only two things that (I think) would make it a fairer comparision:

    - They should have priced out a clustered enviroment for NT so that the uptime for both enviroments would be equivalent.
    - If they use EJB's on the the J2EE platform, then the should use COM+/MTS on the .NET platform. Since the current implementation of objects within the .NET platform doesn't support transactions, the interop layer would have to be utilized to use MTS.


    Coming from both a Microsoft .NET and a J2EE platform, I'm a little disappointed with those people who look for reasons why the comparision wasn't fair. TMC did what they though was a fair comparision, and in my mind it was fair.




    >You start out with a KNOWN ceiling, and you know from day >one you will NEVER be able to scale your app past a >certain point.
  94. You mean THIS SUN: http://www.businessweek.com/magazine/content/02_47/b3809001.htm?scriptFramed

    ?
  95. And your point is?
  96. on page 14, the report mentioned that the .net server was set up by an MS employee.
  97. I agree with Cameron's questions about disclosure (funding and company involvement - why not the App Server A was tuned by Vendor A?).

    Second, I don't understand why people think this was a "good job" by the Middleware company?

    I think this is a marketing investment, hoping that all these people just trained in J2EE will now rush to buy .Net training. Smart move again, Microsoft!

    Let's not forget that Bill Gates was unhappy about the .Net market penetration and developer adoption, so something revolutionary had to be done ;-) Remember also all the MS folks closely monitoring this site and inciting people to change sides.
  98. I'd like to point out that my boss (I'm hard-core java-j2ee BTW) happens to be a Regional Director and he recieved an email about this BEFORE it appeared on TSS, so I'm a bit suspicious.

    Anyway, when handed lemons, make lemon aid.

    Someone with the where-with-all download the 2 source sets and run there own comparison, a la the scientific method of publish and reproduce (I can't, I can't afford the M$ licencing ;) ). Then re-optimize with, oh say, jdk 1.4 and dropping the entity beans for Jarkarta OJB or similar. Or try on JBoss or OC4J with OCI drivers instead of thin drivers. there are all sorts of better ways to do the Petstore than the RI or even the TSS version. You see, with J2EE/Java there are all sorts of different vendors to use and ways to do things, depending on your priorities ;)....with MS there is, well, only their way...

    I am still confident that different versions will perform better and differently.

    I would like to thank TSS for shaking things up and getting people to call to action...


    BTW, the petstore is a joke...lets talk about real applications in the real world...oh wait, there are none for .Net yet ;)


  99. Interesting. Download the code. It doesn't take a lot of modifications to get J2EE server "A" (WebLogic 7.0) to run faster than .NET 1.0sp2.

    Hmm. Who made the decision to use XA? It's time for some full disclosure, guys!

    1. What were the roles that each vendor played in the tests?
    2. Who paid for the tests?
    3. Where were the tests conducted?
    4. Was each vendor given a chance to comment or modify the tests?

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  100. Cameron,

    "Interesting. Download the code. It doesn't take a lot of modifications to get J2EE server "A" (WebLogic 7.0) to run faster than .NET 1.0sp2."

    Have you got facts to back this ? And even for the web services part ?

    Yann
  101. Hmm. Who made the decision to use XA? It's time for some full > disclosure, guys!


    > 1. What were the roles that each vendor played in the tests?
    > 2. Who paid for the tests?
    > 3. Where were the tests conducted?
    > 4. Was each vendor given a chance to comment or modify the
    > tests?

    I agree!
    I think that, if the test wants to be credible, these informations MUST be given!!

    Enrico
  102. Note that WebLogic's TM logs XA transactions to disk,

    >so there is a flushed write for each transaction state
    >change, which kills performance.

    Do you mean that WLS doesn't do group commit ?

    --
    Dimitri
  103. Note that WebLogic's TM logs XA transactions to disk,

    >so there is a flushed write for each transaction state
    >change, which kills performance.

    Dimitri: "Do you mean that WLS doesn't do group commit ?"

    To implement a recoverable TM, there has to be a log of what XIDs would need to be recovered. WL logs that to disk. To avoid losing the information, it flushes after each write. Obviously that is a worst-case performance scenario, (which is why Microsoft added that requirement to the test ex post facto.)

    Regarding "group commit", the way XA works is that the manager iterates through the XIDs and sends each with a "prepare" call. Any that return "OK" are kept in the list. Any that return "read only" are assumed to be committed and/or rolled back (because there were no changes) and are removed from the list. If/when one returns "fail", then all previous "OK" XIDs are sent with a "rollback" call. If none returns "fail", then all XIDs are sent with a "commit" call. Each of the XA state changes has to be logged so that when the WebLogic server comes back up, it knows if there were any pending transactions when it went down.

    From the document: "Distributed Transactions. In the original Sun Java Pet Store, all database transactions are executed across a single database. In the Pet Store 2.0 benchmark, for every order placed, a distributed transaction across two physical databases is performed such that the order information is placed into one database, while the inventory count is updated on a separate database. If one part of the transaction fails, the entire transaction is rolled back. The application and benchmark have been created to compare the performance of J2EE EJBs using JTA transaction services to .NET transactions handled via a .NET Serviced Component utilizing COM+ for transactions."

    Bunk. This had nothing to do with JTA. JTA is always used by EJBs, even in the absence of a database! What they meant was that "the application and benchmark have been created to compare the performance of a vendor-neutral open standard (XA) with a proprietary distributed transaction scheme optimized for one vendor's product." It would have been a closer comparison to use Oracle RAC with some tables on one server and some on another; that would have obviated the need for logged XA in WebLogic. Of course, then you really would not have wanted to see the price tag.

    It brings up an important point, though, which is that Java doesn't have a high-enough speed recoverable transaction manager for XA transactions. WebLogic is a lot better than it used to be in this regard (quality and speed), but it doesn't compare favorably with an all-Microsoft stack built on MTS and a proprietary distributed transaction implementation. Not having ever seen MTS actually being used, it's hard for me to comment on its strengths and weaknesses, other than its proprietariness, but most of our work is in the financial and insurance industries where Windows just isn't used in the data centers.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  104. I meant the optimization when TM never goes to disk on behalf of individual transactions, but waits for a while for messages to accumulate before flushing disk block. That way individual transactions may wait a little longer for events to be saved, but overrall throughput will be higher (due to reduced number of disk writes).

    --
    Dimitri
  105. "I meant the optimization when TM never goes to disk on behalf of individual transactions, but waits for a while for messages to accumulate before flushing disk block. That way individual transactions may wait a little longer for events to be saved, but overrall throughput will be higher (due to reduced number of disk writes). "

    I would be very surprised if Weblogic did NOT do a group commit.
  106. Dima: "I meant the optimization when TM never goes to disk on behalf of individual transactions, but waits for a while for messages to accumulate before flushing disk block. That way individual transactions may wait a little longer for events to be saved, but overrall throughput will be higher (due to reduced number of disk writes)."

    John: "I would be very surprised if Weblogic did NOT do a group commit."

    To be a truly recoverable TM server, you cannot pend log writes. My limited understanding about WebLogic's implementation in this area comes from one of the BEA engineers that worked on it.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  107. To be a truly recoverable TM server, you cannot

    > pend log writes.

    Sure, you can. With a group commit. But you need a high parallelism of transactions to get effect.

    Principles of Transaction Processing (Amazon)
  108. The benchmark is very complete, but I need some explanations on Appendix 5 and the following phrase: "Number of application server threads 7 for 2 processors, 5 for 4 processors and 4 for 8 processors". Why threads decrese when CPUs increase? I would expect that threads increase with CPUs!

  109. Romano,

    As the CPUs went up we increased the number of running server processes. For each processor level the number of running server instances was equal to the number of processors. So the total number of threads did increase:

    2 CPUs = 2 processes * 7 threads = 14 threads total
    4 CPUs = 4 processes * 5 threads = 20 threads total
    etc.

    We spent a week or so just tuning thread and process counts and found those to be the best numbers for this app.

    Cheers, Will
  110. Okay, so where are the positives?

    I love Java, however it seems like a no brainer to me. (Better performance + Fewer lines of code) != J2EE

    Looks as though the advantages of Java can only be seen heterogeneous environments. I wonder how long we'll be able to cling to that claim as well?

    I have to admit I don't like seeing the results in black and white.

  111. I was very surprised that with the lab setup nobody bothered to benchmark Clinton Begin's re-write. I think that would have been a nice starting point. Then they could have gone on to their re-write. Clinton Begin's rewrite can be found at :
    http://www.ibatis.com

    Cheers.
    Charles
  112. Fast J2EE[ Go to top ]

    Use
    - JSP/Servlets
    - low-level JDBC drivers
    - Your own class libraries
    - that's it!

    Don't use
    - EJB
    - JDO
    - Struts or other layer-adding MVC framework
    - JavaBeans, which depend on expensive reflection
  113. Fast J2EE[ Go to top ]

    I was with you at the beginning when you mentioned EJB, but when you included Struts and JavaBeans you lost me. Developers are going to end up building Struts-like frameworks in the end anyway. There's nothing non-performant about Struts. It's free so it adds nothing to the price point, like other well-designed frameworks, it will save development time and it's based on JSP/Servlets.

    There's a single instance of each Action class and multiple requests are threaded through it. The Action class should contain no logic except to invoke a service method. The service layer can be EJB true, but it can also be a standard Java class that implements some business interface and does JDBC.

    I think it would be benefical for everyone to see where the majority of time is spent in the execution of these requests. I can almost guarantee you that it's not in the presentation tier. At least my experience has shown it to be further back and I've never had a performance problem with Struts.

    Chuck
  114. Fast J2EE[ Go to top ]

    I think people are being to harsh on EJBs. I think EJBs have alot to offer. I think with Local Interfaces, using Stateless Session Beans for the container services is very beneficial. Also, MDBs solve alot of issues and have little overhead. I think that CMP is the only odd ball out. It's really the web tier that makes CMP fail. CMP's can be fast when employing caching but you cannot trust the data on a web page not to be dirty because users leave their desks and HTTP is stateless and we can't lock data for a long period of time. Also, you can never guarantee that some other legacy application can touch the data. CMP is difficult to swallow with the web.

    I think most project try to replicate the data model into CMPs which is really a repeated effort. CMP's were suppose to be similar to a VIEW concept of databases. However, having the View in the application tier causes big performance issues.

    I think the EJB spec can stay but the CMP layer can be removed.


  115. >> I think people are being to harsh on EJBs

    Typically, the judgement comes from those just writing web apps and that dont need to do too much sophisticated stuff in the middle tier. For them, just Struts is probably enough.

    For the rest of us, that actually use RMI for remoting (GUI clients?) then EJB is almost a default choice. Most of our apps support a web and a GUI interface.

    As to whether CMP is a benefit or not? Again, for many apps it is. If JDO delivers on simplicity and vendor support, then perhaps it will rival CMP. But for the moment, its the only viable standards-based solution.

    The problem side-effect of benchmarks like this is that they promote thinking that ignores the PRIMARY question: "Is it fast enough?"

    Just like most web applications dont need the services of EJB container, most apps dont need to support x0,000 concurrent users.

    Still, its interesting to know what the performance delta is. Its just a pity that some jump to all sorts of weird and wonderful conclusions...

    -Nick

  116. >> I think people are being to harsh on EJBs
    >Typically, the judgement comes from those just writing web apps and that dont need to do too much sophisticated stuff in the middle tier.

    Uh, no. Typically it comes from people like me, who complain that Entity Beans aren't capable of supporting a decent domain model, since they cope badly with inheritance. It is precisely when you try to do something sophisticated in the middle tier that entity beans fail you.

    Of course, the other app server resources that an EJB container provides (connection pooling for data-sources, session pooling for JMS sessions, global transactions, association of transaction boundaries with RMI calls, etc) are very, very useful.

    And your comment about x0,000 users is misleading: a data entry app we wrote that validates invoices has under 150 users but does a hell of a lot of processing and has to perform very well. Intranet apps with fewer users are often *more* processer intensive than web apps with more users because the intranet apps tend to have much more complex business logic.


    Sean

  117. Typically the problems in these EJB debates is that when some people say "EJB" they refer to CMP Entity Beans, while others are actually referring to all 5 bean types.

    I was referring to all 5 bean types.

    I am well aware of the limitations of CMP. For mine, inheritance is not as big a deal as the lack of a dynamic query capability. It certainly isnt perfect, but it suits most apps we have looked at.

    As for 150 user intranet apps, they are exactly the kind that we tend to build. Typically, however, the higher the concurrent user number, the more the pressure on the GC.. (depends how the app is written of course)

    -Nick

  118. It seems to me from the comments on this thread that what we are seeing is that a particular J2EE app built to Sun's "Best Practices" has had some problems compared to a .Net app.

    Various people have said "so PetStore shouldn't use Entity Beans".

    Okay, but if so a key part of J2EE has failed here. That key part is, of course:
     * Sun's J2EE Best Practices.

    Don't discount the importance of this. Most developers don't have the time or skill to abandon the manuals and best practice guides and do something else.

    (And since I'm not a .NET expert - could someone who is tell us whether the .NET PetStore is full of performance-tuning unusual practices, or is an example of the documented .NET best practices?)


    Sean
  119. Oh, not the bloody EJB debate again![ Go to top ]

    I wonder why Middleware is doing that but I for one lost all confidence of theserverside and middleware for lying to the J2EE community by posting a flawed report. I went over the report and the source code and I was astonished by the lies.
    This is a terrible report. I for one would never trust anything published by The MiddleWare. I urge everyone to look at the source code and see that the report doesn't actually match the code and if that was not intentional then the developers have no experience with J2EE.
  120. A couple of people posted the ref to Rickard's article while I was writing my comment above.

    He answers my question: it appears the .NET app had some performance optimizations which we really shouldn't allow in a decent app.

    As for Rickard's other points: most look to me like valid criticisms of the comparison.

    The only comment by Rickard I'd take issue with is that he accuses TMC "lying". I realise he's feeling aggrieved, and there may have been deliberate deception. But since, as Rickard points out, any disparity between TMC's claims and it's optimized code are very bad for TMC's rep, and since we're in an industry where reputation is very, very important, I'd suspect mistake rather than malice.

    Sean
  121. Oh, not the bloody EJB debate again![ Go to top ]

    In response to http://dreambean.com/petstore.html

    Yes, the data access is in the business object BUT it could be easily separated out into a factory. This won't affect performance and it will barely add a few hundred lines of codes.

    Some of it is also a difference in philosophy. The J2EE app uses 44 different ___Exception.java files whereas the .NET app here just uses the following...

    throw new Exception("Unknown user, unable to read address");


    And the java code is rife with code like the following...

    try {
                AccountDAO dao = getDAO();
                dao.create(this.accountDetails);
                return (userId);
            } catch(AccountDAODBUpdateException se) {
                context.setRollbackOnly();
                throw new CreateException (se.getMessage());
            } catch (AccountDAODupKeyException acd) {
                throw new DuplicateKeyException(acd.getMessage());
            } catch (AccountDAOAppException aca) {
                throw new AccountAppException(aca.getMessage());
            } catch (AccountDAOSysException acs) {
                throw new EJBException(acs.getMessage());
            }

    Again, this has been the standard examples and practices Sun has advocated for years. The fact that Account needs 18 classes is a little overkill. Even worse, Order needs 22 classes.


    One interesting sidenote:
    Why are they using a table-based PK increment approach...
       "SELECT seqnum FROM sequence for update"
       "update sequence set seqnum = seqnum + 1"
    Have they not ever heard of Oracle built-in sequences...? What is this, dBase? Besides two extra DB hits, it is also bad practice, as it causes DB contention.


    Yes, .NET is using the built-in Cache object for some read-only objects but I fail to see why this is not allowed. Doesn't EJB also provide similar benefits?



    A good chunk of the code size (30%) was in the front-end. Now, based on the source, the .NET app uses server-side controls and code behinds - which eliminates the need to retrieve and set values from the Request object, etc - eliminating tons of code. Of course, if you want to utilize this, the pages have to post back to themselves - which is usually avoided in JSP MVC patterns. It works here because the app is pretty trivial and simple.

    Note: the above is not used everywhere - adding items to a cart does NOT post back to itself, for example. But editing an account does.


    Another example of .NET differences...

    Most of the .net model objects are standard classes w/ get & set properties, etc. But a couple (LineItem, Address) are structs. This is the entire LineItem.cs file....

    namespace PetShop.Components {
    public struct LineItem {
    public string id;
    public int line;
    public int quantity;
    public decimal price;
    }}

    compared to a 90 line .java file.


    Besides shorter code, it also performs better. Since structs are stored on the stack and not on the heap, they are much more efficient (no need for GC, etc). On the other hand they have limitations. From MSDN...
    "Structs can declare constructors, but they must take parameters. Also, you will not be dealing with references to an instance of a struct as you would with classes. You will be working directly with the struct instance. Because of this, when passing a struct to a method, it's passed by value instead of as a reference. There is no inheritance for structs as there is for classes. A struct cannot inherit from another struct or class, and it cannot be the base of a class. Structs, however, inherit from the base class Object. A struct can implement interfaces, and it does that exactly as classes do."

    Is this widely practiced? Probably not b/c structs would be mostly used for points on a drawing. Then again, does a lineitem on a order really need to be a regular object - depends on whether you planned on using inheritence, etc.

    From MSDN:
    "This optimization can be misused. Using structs instead of classes can also make an application run slower, or take up more memory, as passing a struct instance as a value parameter causes a copy of the struct to be created. There is no substitute for careful data structure and algorithm design."

    "Whenever you have a need for a type that will be used often and is mostly just a piece of data, structs might be a good option."

    "By using structs, you can create objects that behave like the built-in types and enjoy their benefits as well."


    Where is the big difference in code size? Exception handling, EJB implementation code vs. .NET attributes....

    Example:

    using System.EnterpriseServices;
    using System.Runtime.InteropServices;

    namespace PetShop.Components {
    [Transaction]
    [ClassInterface(ClassInterfaceType.AutoDispatch)]
    public class OrderCOM : ServicedComponent {


    .NET also only uses EnterpriseServices around the order object (it is the only one that requires XA transactions) whereas J2EE uses EJB's for every entity.


    Of course, being able to just put "[WebMethod]" above a method name to make it a web service helps cut down the code a bit.


    I myself would probably have used some interfaces in the .net code and set it up some more separation of the data access layer.
  122. Oh, not the bloody EJB debate again![ Go to top ]

    In fact, in a few cases, they share some common bad examples...has no one heard of for loops?

    <select name="expiration_month">
              <option value="01"
                <% if (cardExpiryMonth.equals("01")) { %> selected <% } %>
                >01</option>
              <option value="02"
                <% if (cardExpiryMonth.equals("02")) { %> selected <% } %>
                >02</option>
              <option value="03"
                <% if (cardExpiryMonth.equals("03")) { %> selected <% } %>
                >03</option>
              <option value="04"
                <% if (cardExpiryMonth.equals("04")) { %> selected <% } %>
                >04</option>
              <option value="05"
                <% if (cardExpiryMonth.equals("05")) { %> selected <% } %>
                >05</option>
              <option value="06"
                <% if (cardExpiryMonth.equals("06")) { %> selected <% } %>
                >06</option>
              <option value="07"
                <% if (cardExpiryMonth.equals("07")) { %> selected <% } %>
                >07</option>
              <option value="08"
                <% if (cardExpiryMonth.equals("08")) { %> selected <% } %>
                >08</option>
              <option value="09"
                <% if (cardExpiryMonth.equals("09")) { %> selected <% } %>
                >09</option>
              <option value="10"
                <% if (cardExpiryMonth.equals("10")) { %> selected <% } %>
                >10</option>
              <option value="11"
                <% if (cardExpiryMonth.equals("11")) { %> selected <% } %>
                >11</option>
              <option value="12"
                <% if (cardExpiryMonth.equals("12")) { %> selected <% } %>
                >12</option>
            </select>

    <% boolean found = false; %>
            <select name="expiration_year">
              <option value="2001"
                <% if (cardExpiryYear.equals("2001")) {
                    found = true; %> selected <% } %>
                >2001</option>
              <option value="2001"
                <% if (cardExpiryYear.equals("2001")) {
                    found = true; %> selected <% } %>
                >2001</option>
              <option value="2002"
                <% if (cardExpiryYear.equals("2002")) {
                    found = true; %> selected <% } %>
                >2002</option>
              <option value="2003"
                <% if (cardExpiryYear.equals("2003")) {
                    found = true; %> selected <% } %>
                >2003</option>
              <option value="2004"
                <% if (cardExpiryYear.equals("2004")) {
                    found = true; %> selected <% } %>
                >2004</option>
              <option value="2005"
                <% if (cardExpiryYear.equals("2005")) {
                    found = true; %> selected <% } %>
                >2005</option>
              <option value="2006"
                <% if (cardExpiryYear.equals("2006")) {
                    found = true; %> selected <% } %>
                >2006</option>
              <% if (!found) { %>
              <option value="<%= cardExpiryYear %>" selected>
                <%= cardExpiryYear%></option>
              <% } %>
            </select>



    <asp:dropdownlist id="listMonth" runat="server">
    <asp:listitem>01</asp:listitem>
    <asp:listitem>02</asp:listitem>
    <asp:listitem>03</asp:listitem>
    <asp:listitem>04</asp:listitem>
    <asp:listitem>05</asp:listitem>
    <asp:listitem>06</asp:listitem>
    <asp:listitem>07</asp:listitem>
    <asp:listitem>08</asp:listitem>
    <asp:listitem>09</asp:listitem>
    <asp:listitem>10</asp:listitem>
    <asp:listitem>11</asp:listitem>
    <asp:listitem>12</asp:listitem>
    </asp:dropdownlist>

    <asp:dropdownlist id="listYear" runat="server">
    <asp:listitem>2002</asp:listitem>
    <asp:listitem>2003</asp:listitem>
    <asp:listitem>2004</asp:listitem>
    <asp:listitem>2005</asp:listitem>
    <asp:listitem>2006</asp:listitem>
    </asp:dropdownlist>
  123. Oh, not the bloody EJB debate again![ Go to top ]

    I have an inside source that informed me that there was a brownout during the J2EE benchmark and that's why it turned out slower...merely an electrical problem (Of course this wasn't in the report).

    I, too, am insulted that I wasn't personally consulted prior to this being published. What kind of company would publish a benchmark without contacting me first?...Perhaps a M$ company? I was on a Hailstorm protected site just last night and guess who owns TSS? That's right, one Bill Gates. I also unearthed several pseudonyms that Bill used to buy 51% of the shares in Sun solely to give the appearance of competition. And I thought holy water was only necessary when the MSDN conferences came to town. I am NOT paranoid, it's just that Bill is out to get me.

    As for the Lines of Code. Anyone wide-eyed youngster in kneepants knows that if you take the utility methods and put them inside a helper class you'll get a 700% reduction in the total LOC. Who really cares about developer time anyway? It's not like it's the most important part of the cost equation. With hardware so cheap why not just throw hardware at the problem? That's why I built an extra server room...just so I can stack Solaris boxes to the sky.

    Here I was thinking it was the use of stored procedures that skewed the initial Pet Store results. Now we all know it's the use of BMP instead of CMP.

    If it turns out that TSS is not owned by Bill, then we all know this is just a ploy by TSS to get the M$ developers' attention. They're already working on a revision where J2EE comes out on top, but they're waiting for funding to pay for the app servers and a mathematician to come along and count the LOC. Also, this time they're actually waiting until I approve of it before they publish.

    And to those of you crying about us J2EEers not publishing our own benchmarks, why must we publish a benchmark of our own when we can just punch holes in the benchmarks of others?

    Perhaps most importantly, the report fails to mention at what altitude these results were taken? Anybody want to wager they were measured at over 5000 ft?

    We all know J2EE is faster and better than .NET. Just because .NET came out 6-7 years later doesn't mean it's better. In fact, it means it's worse. Why do we need a benchmark to prove it? Let's just trust our intuition for once. I, for one, will never even look at a line of M$ code. With all these people asking "WWJD?", all I hear from the M$ camp is "WWBD?".

    1...2...Billy's coming for you...3...4...better lock the door...5...6...grab the crucifix..
  124. I'll remind the audience again, TheServerSide (TSS) is simply reporting the news here, much like javaworld or onjava.com.

    TheServerSide was not a participant in this experiement, we did not run them tests, we are not even hosting the report. We're simply linking a piece of news.

    Would you blame TheServerSide if this news about TMC's experiment was first posted on onjava.com?

    We have nothing to do with these resuls, other than being owned by TMC, where TSS is a completely independent business unit consisting of 5 people who work on TSS full time.

    thanks,

    Floyd
  125. Oh, not the bloody EJB debate again![ Go to top ]

    Does TSS have the resources (possibly with commitment from BEAS or ORCL also) to re-implement Java PetStore 1.3.1 on WebLogic 7.0.1/OC4J 9.0.3 using EJB 2.0 CMP? It will be a huge win if you guys can pull that off!
  126. Im sorry Floyd, but you can't escape like this.
    I find suspicious that all TheServerside gurus (Rijkard,Floyd) are so quiets.
    Guys, you are J2EE evangelist (writing books, giving seminars etc.) and you shut your mouth adter publising such results!!
    How r u oing to promote to promote J2EE platform when your site publish articles that says that EJB sucks, but .NET rocks?
    Sorry guy, you may loose a bit of your credibility here.
    Its your duty to react to this analysis


  127. "I find suspicious that all TheServerside gurus (Rijkard,Floyd) are so quiets."

    Rickard is not quiet.

    http://dreambean.com/petstore.html
  128. I changed my mind![ Go to top ]

    Somewhere earlier in this thread I said that thanks to the TSS (or now, TMC) for being honest. I changed my mind. It seems that the benchmark is not accurate enough. Hope to see a true benchmark as soon as possible. It's very disappointing to see a Java-side company has done such a wrong competition.

  129. Boycott TSS and TMC[ Go to top ]

    We should boycott TSS and TMC unless they redo their benchmark tests under the supervision of representatives from Bea, Sun, IBM, and Oracle and the others. They say our tests was not apples-to-apples because we had not enough money! So it would be better for you not to run the tests anyway.
  130. I changed my mind![ Go to top ]

    i am sure you would like to see a benchmark where j2ee is 200 times faster than .net.


    if u cant respect a third party's observations , dispute it with facts.
  131. Posted by Floyd Marinescu 2002-10-29 21:58:35.0.

    > I'll remind the audience again, TheServerSide (TSS) is
    > simply reporting the news here, much like javaworld or
    > onjava.com.
    >
    > TheServerSide was not a participant in this experiement,
    > we did not run them tests, we are not even hosting the
    > report. We're simply linking a piece of news.
    >
    > Would you blame TheServerSide if this news about TMC's
    > experiment was first posted on onjava.com?
    >
    > We have nothing to do with these resuls, other than being > owned by TMC, where TSS is a completely independent
    > business unit consisting of 5 people who work on TSS full
    > time.
    >
    > thanks,
    >
    > Floyd

    Floyd,

    With the greatest respect, to imply that TheServerSide is in any way independent from the The Middleware Company is at the very least being economical with the truth... Do they not (as the advert on the side bar of this web-page states) build, run and sponser this site?

    Other sites, such as Javaworld or onjava.com, would surely have editorial freedom to comment adversely if they perceived there were issues with The Middleware Company's J2EE PetStore application implementation or the fairness of the comparison. TSS obviously does not have / desire this freedom.

    Are you trying to tell me that Rupert Murdoch has no control over the editorial content of his newspapers???

    Also, you seem to be deliberately evading the real question:

    Why did The Middleware Company (supposedly) pick a fight that they were never going to win? J2EE (especially using entity beans with the additional layer of abstraction they entail) was never going to outperform .NET running on the Wintel platform.

    Are you trying to tell me that the TMC did not know, or at least suspect the final outcome of this comparison before they undertook it?

    If they did not realise this, then their technical abilities cannot be held in very high regard... (but I think we know this is not the case)

    However, if they did know/guess the outcome in advance, then the only possible reason for conducting this comparison could be to help Microsoft promote .NET for their own financial gains. If this is the case then it is a fallacy to say that TMC "were trying very hard to make J2EE win".

    Either way it just does not add up.

    And, by the way, did TMC approach the leading J2EE app server vendors with an offer to compete in this comparison? - I'm guessing that TMC did not or else they would have stated in the report that their offers had been declined.

    Just this issue alone is enough to cast TMC in a very bad light.

    I would be most interested to hear the thoughts of the J2EE app server vendors on this report...
  132. Oh, not the bloody EJB debate again![ Go to top ]

    <quote ref="http://www.middleware-company.com/company/companyIndex.shtml">
    The Middleware Company began in 1998 when we envisioned a better future for Java projects. At the time, we noticed many Enterprise Java projects were failing, because:

    The wrong decisions were made on a project, or
    The team lacked the skills.

    Our people are the world's leading authorities in middleware technologies, and have been through the tough issues before. We guide developers towards avoiding common potholes, and they learn to make sound architectural judgments in their projects. This reduces projects' risk levels
    </quote>

    I guess TMC just shot it self on the head :D
  133. This is a very interesting article which is supposed to
    represent the J2EE application servers that have been
    extensively optimized for performance and scalability. It appears as if they were using servers that have been
    optimized if one does not download the test code.

    After the flaws in testing were pointed out,
    they are stating the facts in FAQ. Why don't they mention these in the main report? Well!! Now they imply
    that they were short of time to perform meaningful comparison.
    They are guilty of misrepresenting J2EE servers in the report. They basically misused the popularity
    they gained through this web site.
     


    >* Local interfaces: Did we use them? If not, why not?
    >App Server B did not support EJB 2.0 so we could not use
    >local interfaces to do an apples to apples comparison.

    Very amusing "apples to apples" comparison!! Isn't this comparison supposed to be
    between .Net and J2EE rather than two J2EE appservers you arbitrarily picked up?

    You conveniently chose an older version of J2EE (thanks to your "apples to apples" stuff) and compared against the
    brand new Beta technology.

    >* Entity beans: Was it fair to use EJB or entity beans
    >for J2EE?
    >Why not use a stateless model as in .NET?

    >The original Sun petstore was built around entity beans.
    >To rip them out would have completely gutted the application.
    >There was not the time nor the money to do that.
    -------------------------------------------------
    Yeah! This is a great reason.

    >Note that entity beans don't always result in bad performance.
    >Sometimes you can get higher performance with entity beans
    >by making them read-only, for example, to reduce the
    >number of database hits. We did optimize our entity beans
    >substantially to minimize database calls.

    Sure! "Entity beans don't always result in bad performance."
    Why didn't you use EJB2.0/CMP entity beans if you truly believe in what you said?

    >* CMP vs BMP: Why was BMP chosen? Was CMP tried?
    >We did not try CMP, and so we cannot testify for sure
    >about whether
    >BMP or CMP would have been faster.

    You do not have to testify this. Any one with latest Appserver can figure this out. On Weblogic 6.1/7.0,
    you can see better performance for CMP compared with BMP.
    There are even published test results that can be seen at
    "http://www.amazon.com/exec/obidos/tg/detail/-/1904284000/qid=1035955208/sr=8-3/ref=sr_8_3/102-9974615-1452940?v=glance&n=507846".


    > App Server B did not support EJB 2.0 and thus the new
    >CMP model.
    >It was not possible for us to do an apples-to-apples
    >comparison if we used the new CMP 2.0 model.

    Well! Here comes your mysterious "apples-to-apples" stuff
    for choosing something bad for J2EE model to start with.
     (of course you do not have the money and time
    issues as far as J2EE concerned).

    > As for the famous N+1 database calls problem: We tried to
    >optimize around this problem by returning the key
    >rather than
    >doing the database query.

     You know you are not solving this N+1 problem.
    Don't you?

    >* Oracle vs SQL Server: People are insinuating that
    >SQL Server should have been used as the J2EE database.
    >What is our defense?
    >This was a JDBC driver issue. We tried a multitude of
    >drivers, and were not able to find an adequate JDBC driver >for SQL Server
    >that gave us anything like the performance of the Oracle
    >driver that we used.

    Why didn't you choose Oracle in both cases?
    What happened to your "apples-to-apples" stuff?

    >So, I think yes it would have been better to re-write
    >from scratch but I don't think we'd have achieved a
    > significantly greater increase in performance.

    Thank you very much! This statement gives us so much relief
    and we trust you. And of course you do not have time and money.


    BTW, why didn?t you consider JBOSS as were comparing the license costs?

    >* If you were to run the app on appservers A and B without
    >using EJBs as many TheServerSide.com members have suggested,
    >would it not run as fast as or a little faster than
    >the .NET app? It very well might have. We don?t know. Some
    > of us think that without EJBs the J2EE app would have
    >clearly beaten .NET,
    >but others who feel that for this particular app, the .NET
    >version would have beaten the J2EE version even without
    > EJBs. We may conduct this experiment in the future.

    Yeah! right! You want to write another report.
    Please let it be done by some one who has money and time!

    >TMC stands behind the results of the tests that were conducted.

    Go ahead stand behind the results!
    Thanks for providing the test code.
    We can interpret the results you are standing behind.
  134. <Quote>
    Some of us think that without EJBs the J2EE app would have clearly beaten .NET, but others who feel that for this particular app, the .NET version would have beaten the J2EE version even without EJBs. We may conduct this experiment in the future.
    </Quote>

    The last line in particular of the above quote made by Floyd is in my opinion indications of the prevarication that this report was deliberatly setup to imply. It's a sad time when a J2EE consulting firm regresses to issuing FUD against J2EE in an attempt to level the playing field for them to develop a better and faster j2ee development pushed by them to prove thier worth.

    Go back to the meeting room guys and re-think this one. Time to pull the plug??
  135. EXCELLENT parody - wonder how many recognized it as such?
  136. Oh, not the bloody EJB debate again![ Go to top ]

    <quote>
    The only comment by Rickard I'd take issue with is that he accuses TMC "lying"
    </quote>

    C'mon!!! If they're not lying, there are options:

    1) They have to accept that they don't know this job (although they claim to be the experts in this area) so it's not that they did things on purpose but because they didn't know how to do it properly,

    2) Accept that they got paid by MSFT.

    So, which one?!?
  137. Oh, not the bloody EJB debate again![ Go to top ]

    Yagiz: "If they're not lying, there are options: 1) They have to accept that they don't know this job (although they claim to be the experts in this area) so it's not that they did things on purpose but because they didn't know how to do it properly, 2) Accept that they got paid by MSFT."

    Don't be so quick to impune the credibility of TMC. While more up-front disclosure should have been made, the fault is not with the efforts, but with the test itself. As one example, distributed transactions with XA are significantly more expensive than in COM+ with a pair of Microsoft SQL Server instances. Why was that added to the test if "there was no time"? Let's analyze whose interest it was in to make such changes to the test:

    1. TMC: No. They said they weren't paid for their time, right? Why would they suggest a big change to the application that would take more of their time?

    2. J2EE Server "A": No. Server "A" is Weblogic, which implements XA "correctly," but in doing so has to log and flush TX state changes to disk, thus slowing it down by a huge factor.

    3. J2EE Server "B": No. Server "B" doesn't work well at all with XA, completely flunking the test.

    4. Oracle: No. XA doesn't make Oracle look any faster.

    5. Microsoft: Yes. Notice that when J2EE had to use Oracle because Microsoft's SQL Server JDBC driver didn't work, the .NET platform didn't switch to using Oracle. It doesn't take a genius to figure out that Microsoft stacked the deck from the start.

    This is just one example, but it's a good start.

    Now, before anyone turns their frustration toward Microsoft, remember, server "B" didn't work. It didn't work. That's completely unacceptable. So what if the test was stacked against open standards. So what if the servers selected may not have been the best two for the test. So what if they used JDK 1.3.x instead of 1.4.x. The J2EE server "B" didn't friggin' work!

    Secondly, why can't we get a working JDBC driver for Microsoft SQL Server? Is it acceptable that a database vendor (Microsoft) not support JDBC? What kind of business would sabotage their own products to spite the standard enterprise computing platform? (Rhetorical question.) No wonder we never see Microsoft SQL Server in financial services and insurance companies. If you can't learn to play nice with others, then stay home.

    Third, why is XA (to be specific, recoverable transaction logging) so slow? OK, so J2EE can beat .NET (yes, even with entity beans, and yes, even on Windows) if you turn off XA and make some other optimizations. How good is that if a known requirement for certain systems (XA) is so slow? I'm not convinced that taking the "home court" (tightly coupled) Microsoft SQL Server advantage away from .NET would make all the difference. Let's say we re-ran the tests with one Oracle and one DB/2 server in the mix (for both .NET and J2EE.) Would .NET be that much slower (assuming it would even work), or does the XA implementation (WebLogic and server "B") just need a little bit more love? Will NIO in 1.4 help that logging for example? Would a native implementation help (ala WebLogic's custom native IO library for sockets)?

    There are two ways to treat these disgraceful results. We can burn TMC/Microsoft/BEA/"B"/whomever at the stake and make excuses all day long, or we can push for improvements in these products. While we know the contest was fixed by design, it highlights issues that need to be addressed, and those issues (like server "B" failing miserably to work) can't be blamed on Microsoft, and probably can't be blamed on TMC either.

    For a rather paltry sum of money, Microsoft won this round, and don't underestimate the size of the win. That they almost got away with it, without any direct tie-in to themselves, is shocking. This is no different from the tactics that we've seen them use in other parts of the market, and if an unethical approach from Microsoft comes as a surprise to us, then we're the idiots. However, and let there be no mistake on this point, we can also see in this that Microsoft is worried -- even terrified -- about its major and continuing losses in this space. They aren't trying to widen a lead; they're trying to stop the bleeding. It's not time for Java and J2EE to go on the defensive; it's time to turn up the heat.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  138. Nicely put Cameron, but I fail to see why we should not question the credibility of TMC for this one. I am still hoping that this move was a deliberate move to make it possible to conduct other benchmarks (for an example using all the improvements and optimizations presented in this thread).

    If TMC would implement basic improvements in a fairly simple rewrite using CMP2.0 mechanisms and/or Stateless/DAO and/or something else and allow J2EE application server vendors to tune the deployment and re-run the tests, TMC's credibility would certainly be a lot better. It is time for BEA, IBM and Oracle to step up to the plate and sponsor this kind of test and thus prove that J2EE does not have to look this bad.

    I am not able to predict any test outcomes but the comparison would certainly be a lot more credible in my books. The only problem is that the EXACT same conditions (hardware, network etc.) would need to be duplicated for the MS crowd not to scream. Can this be done since the lab was provided by MS and located in Seatle(?).
  139. Oh, not the bloody EJB debate again![ Go to top ]

    I'm really not worried about these results. Server A is more reliable than anything MSFT could put out, especially with all the transactional stuff it's doing.

    What *I* want to see is clusters. Find the best Java cluster and run it against a single machine running Windows/.NET. MSFT is only beginning to find out the clustering is the only way to get reliable and available systems. Linux and the J2EE vendors have been involved with clustering/caching long before MSFT ever noticed it needed a server-side story.

    $.02
    Steve
  140. Oh, not the bloody EJB debate again![ Go to top ]

    If you're going to cluster WebLogic then allow the MS side to do the same thing. After all, it's only fair - plus MS and WL charge more for clustering, which could be a good way to see how price/performance differs.

    MS recommends scaling out .NET using multiple app servers, so I would consider it valid comparision.
  141. Greetings Cameron - I've always enjoyed your input to this forum! I must pose a question to you ... would you praise Dr. Gleyzer if he posted findings that cited Coherence as being inferior in every way to a well-marketed competitor - even if the context expressed the truth? I truly feel this may be the most damaging communication I've seen in this conflict and, to a great extent, TMC should be held accountable. Clearly J2EE practices must improve and we are all responsible for its success - but politically speaking, this document may have caused irreparable damage. Regards - Bruce
  142. Bruce: "Greetings Cameron - I've always enjoyed your input to this forum!"

    Flattery will get you nowhere. (That is so untrue, of course, but I felt like I had to say it. ;-)

    Bruce: "I must pose a question to you ..."

    I cannot answer that hypothetical question; I simply do not know how to.

    Bruce: "I truly feel this may be the most damaging communication I've seen in this conflict and, to a great extent, TMC should be held accountable. Clearly J2EE practices must improve and we are all responsible for its success - but politically speaking, this document may have caused irreparable damage."

    In his many battles in North Africa, when Rommel felt that his flank was falling, he would make a decision to either press ahead, knowing that the enemy was no longer in front, or fall back, knowing that his flank protected something even more important. Never did he turn his fire on his own tanks. I think you'd be pretty surprised if you knew the whole story behind this test, but it isn't my story to tell. Needless to say, I have every confidence in the motives and efforts of the those TMC consultants, and I wish the published report didn't appear so damning to the work that they performed.

    I agree that the PR damage to J2EE is horrendous, and would have not have been so blatant had the proper disclosures been made up front by TMC, but since this represents Microsoft's best efforts at designing a test to show one of the few areas in which .NET can outperform J2EE (not to mention the best efforts to leverage a respected name in the industry to publish Microsoft's own test ;-), it would seem that there is little more that they can gain from it. They're already starting to lose whatever short-term gains that they hoped for as the industry in general and vendors "A" and "B" in particular wake up. Microsoft, as usual, is playing with fire, and in the end it will get burned.

    Always forward.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  143. Thank-you for your thoughtful reply (even though things did not turn out so well for Rommel and his integrity ultimately led to his unfortunate accident). I have counted on authors from TMC for so much of my own work - I have held, and continue to hold, the highest regard for their mentorship. My question is more one of good business sense - it is difficult for me to reconcile a decision to make public such findings (check the main MS page www.microsoft.com). Take care - Bruce
  144. Oh, not the bloody EJB debate again![ Go to top ]

    Cameron,

    "I think you'd be pretty surprised if you knew the whole story behind this test, but it isn't my story to tell."

    I think we'd all be pretty surprised if we knew the whole story behind *any* political action in general, but I guess we are just people with no specific access to non-disclosable information. All we can see is the PR, the report and the code to draw our conclusions. Pardon us, we're not in the know...

    If you don't want to disclose anything, you should not even mention the existence of a whole story behind this test.

    Yann
  145. Oh, not the bloody EJB debate again![ Go to top ]

    The FAQ posted on TMC is only another layer of lies. If App Server B doesn't support feature X and Y and X are Y have considerable advantages why use App Server B?
    Why claim that CMP is not clearly faster than BMP when it is published in a book written by TMC employee.
    This FAQ is just PR that dismiss even more the credibility of TSS and TMC.
  146. ./ed

    I sure hope that TSS cluster is working ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  147. Oh, not the bloody EJB debate again![ Go to top ]

    Cameron..

    "./ed"?

    Slashdot not dotslash...

    /.ed

    sheesh....
  148. Oh, not the bloody EJB debate again![ Go to top ]

    Michael: "Slashdot not dotslash..."

    I'm a bit dyslexic at times.

    The interesting thing is that when gotdotnet.com was slashdotted, it crashed and burned. ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  149. Fast J2EE[ Go to top ]

    <quote>
    CMP's can be fast when employing caching but you cannot trust the data on a web page not to be dirty because
    </quote>

    Extending your point: the issue isn't just 'the web tier'. Rich Clients hit the same issue, only worse, because they tend to have more data cached on the client system. So the issue is the interaction between a transactional system (the app server) and a non-transactional system (the client).

    Sean
  150. Fast J2EE[ Go to top ]

      Very interesting benchmark,

    I'm happy the java community is opening eyes !

    I work with Java since 3 years now. I begin work on J2EE application with EJB. Now I only use JSP/Servlet and a MVC framework (struts, custom or other) and a simple custom persistance layer directly based on JDBC (not EJB neither JDO based). Since, performances issues has never been a real problem for our specifics needs (web application server IPlanet, Weblogic, WAS, Tomcat).

    I'm very curious to know, if it's possible, what's the cost (in terme of benchmark) of the EJB based persistance layer in the Java PetStore ? (versus of a simple JDBC persistance layer ?)

    Thank's

    Farid.
  151. Fast J2EE[ Go to top ]

    <quote>
    Use
    - JSP/Servlets
    - low-level JDBC drivers
    - Your own class libraries
    - that's it!
    </quote>

    For a simple Petstore style application, I completely agree. If just productivity is the question in vertical apps, then an alternative is to use vertical frameworks (such as commerce, personalisation, portals). MS Commerce Server 2000 was quite bad, so now that everybody is doing CRM, they asked Siebel to write this one for them :-)

    But people have choice. Also, enterprise solutions are more than shopping apps. I think since the start-ups are gone, there are few big scale projects that have the priviledge to start stuff from scratch. Integration capabilities are key in big enterprise apps. Java probably still leads there.
  152. Fast J2EE[ Go to top ]

    <Scott>
    Don't use
    - EJB
    - JDO
    - Struts or other layer-adding MVC framework
    - JavaBeans, which depend on expensive reflection
    </Scott>

    Not sure I follow you here. I can see why you would suggest avoiding EJBs. Unless you need a good deal of the services they provide, you can do without. That is, if you don't need distributed objects (and *many* apps don't) and you don't have tranacactions or your transactions are trivial, then by all means don't use EJB.

    As for JDO - I still haven't seen much happening here, so I won't/can't comment on that.

    But not using frameworks - why? First of all, they don't add any significant overhead. Second, they provide a framework that somebody is going to end up writing anyway. Third, even *if* they slowed things down (which they don't) - speed isn't everything. Minimizing complexities and dependencies is often just as, if not more, important. (Sidebar - are open source frameworks like Struts available for .NET?)

    Finally...avoiding JavaBeans? Why? This a staple of JSP development. Plus, as of 1.4, reflective method calls are not an order of magnitude slower than compiled method calls like they used to be. IIRC, they are about twice as slow. Used correctly, JavaBeans *do* have a pay off.

    And one more thing, I get *so* tired of everybody beating the "this app server is faster than that app server" or "this technology is faster that that technology" drum. So what? Does this really matter in many circumstances? Very often, app A can do 1000 tasks/second while app B can only do 500 tasks/second. But if 10 tasks/second is all that is required, does it really matter which one you go with if speed is the only concern?

    My $.02

    Ryan
  153. "And one more thing, I get *so* tired of everybody beating the "this app server is faster than that app server" or "this technology is faster that that technology" drum. So what? Does this really matter in many circumstances? Very often, app A can do 1000 tasks/second while app B can only do 500 tasks/second. But if 10 tasks/second is all that is required, does it really matter which one you go with if speed is the only concern? "

    Totally agree, that's when factors like lines of code, easy of configuration and such become important. But according to the report, .NET beats J2EE even harder on this issues.
  154. Fast J2EE[ Go to top ]

    <Ryan>
    As for JDO - I still haven't seen much happening here, so I won't/can't comment on that.
    </Ryan>

    Please, take a look to Exadel JDO.

    Best,
    Igor Shabalov.
  155. Fast J2EE[ Go to top ]

    The "castor jdo" project has a strong following. http://www.castor.org/


    Anthony
  156. Fast J2EE[ Go to top ]

    the use of a framework like struts is very benefit :
    the separation between the data, the presentation and control;

    if we don't use ejb than we last the midelware services that would take us important time to code
  157. Fast J2EE[ Go to top ]

    And how many MORE lines of code will you consume by not using these frameworks, etc?

    By doing so, you just increased the complexity and cost of your J2EE app. Where is the value in that?
  158. I'm not sure I understand this correctly. Is it true that no one can compare against .NET in terms of performance or any other criteria, if that comparison shows .NET to be inferior?

    Where is this stated (ie. MS licensing agreement, etc.)?


    Just curious.
  159. Greetings,

    Very, very interesting benchmarks report. There maybe only one conclusion from reading this report - .NET rules!

    One thing that you would not agree with is OS Environments. I would expect to see J2EE running on OS and hardware other then Windows and PC compatible. In case of .NET Pet Store you may see that all components of the Application well integrated with OS: IIS, C#, COM+, DAO. But in case of J2EE all integration was encapsulated in JVM. Probably, this difference in benchmark results is a price for platform independence.

    Hopefully, Sun or IBM will provide another benchmarks where they will utilize their own OS and hardware (or even DBMS). Unfortunately, price for solution will be more expensive then .NET, but at least they may beat performance results. But on other side maybe Sun and IBM knew that .NET superior and they do not want to show their benchmarks.

    Best regards,

    Taras
  160. For some of us, there still isn't a real choice available. If you are forced to access (or better yet, deploy on) mainframes or midrange, .Net is really not an option yet. Will it catch up? Possibly, but I'm still waiting for the .Net implementation that will run on IBM iseries or zseries (AS/400 or Mainframe), give me all the features I currently get with J2EE, and give me more than 1 transaction per hour (note the sarcasm). If that happens, I'll be very impressed.
    For small internal applications that are going to be deployed to Windows anyway, I've already been evaluating for a while. Each platform has it's strengths, just have to pick the one that best fits the problem at hand.
  161. 1. 2 man-week vs. 10 man-weeks. Right, but as I understand, in order to choose the best deployment for the J2EE paltform , there were tested 4 JVMS(Sun, JRockit, etc), few other JDBC drivers, 2 OSs(Linux and Windows) and 2 different app servers were used (with different configuration requirements, etc).

    2. Performance: the jre1.4 was not used, even the report states that it performs with 50% better with some components.

    3. Prices: besides the fact the .NET Server price was not published yet, did you made any price computation taking into consideration the MS future price policy? Or any free J2EE implementations?

    Cheers,
    Horia
  162. And, by the way, I think you should've test with JBOSS or any other insted of app server B. It is a shame (as the price) and I bet it's WAS.
  163. Regarding your Point 1:

    OK, so the argument is that having many choices (as opposed to the monolithic "one" offered by Microsoft) in tools/technologies is a good thing, and therefore a justified reason for taking 10 weeks to choose and configure the "optimal" combination of choices?

    After taking all of that time to choose the optimal combination, you then step into the benchmark arena and still get your a** thoroughly kicked, while writing 7 times more code than the "single choice" required??? What is the point of having all of these great choices?

    What alarmed me the most about these results, and I haven't seen this mentioned in any other threads, is the absolutely consistent peformance .NET has, no matter how many CPUs, how many users, etc. The graph line is almost a straight 45 degree angle--as the load increases, the throughput increases. Java performance either petered out as loads increased, or the application failed altogether. Others have been suggesting that maybe using different DBMSs had alot to do with the results. The absolute consistency and uniformity of the .NET results leads me to believe that .NET will outperfom Java in this configuration, no matter what DBMS is used, and that the underlying plumbing and infrastructure is fundamentally more sound than the JRE (at least in the configuration tested).

    McNealy: Wake up. You got your a** kicked. Stop disputing the validity of the results and talking about how nasty Microsoft is and focus on fixing the problem. If you don't, by 2007 .NET will be to Java what Windows was to OS/2, and what IE is now to Netscape.
  164. Well, in this case I can see a strange combination for 2007:

    first, .net takes over j2ee
    Mozilla takes over IE
    Linux takes over Windows
    x86 dissapears
    Microsoft turns to linux

    How funny the life can be! Let's just enjoy it!

    cheers
  165. Ouch!
  166. I think that it's very important to stress a few things about the J2EE implementation for these benchmarks:

    (1) The configuration / tuning time was based on evaluating many different products. This is very different than evaluating all of the options for a specific vendor, as is the case with .NET.

    (2) The Java Pet Store doesn't use an off-the-shelf framework. Part of .NET's appeal is that it has a lot of existing components to handle a lot of work for you, and ASP.NET really is a combination of a servlet engine, a user interface component model (with a basic set of components), and a whole web app framework. In the J2EE world, this would be something like Struts and some pre-existing user interface components / tag libraries plus the servlet engine. This obviously makes a big difference in the number of lines of code.

    (3) I really do think EJB may be overkill for this type of application, but the main benefit it provides is the distributed transaction support. In general, I definitely think EJB has more overhead than the comparative Microsoft approach, and for many applications, it isn't necessary. I think it provides a lot, however, in real-world integration scenarios.

    (4) I wonder if the cost comparison was a little too simplistic, since it doesn't take into account total cost of ownership. What about supporting the .NET solution over time?

    I know some of you have mentioned these things before, but I think it's important to emphasize exactly what the differences are.

    Great job, Middleware.

    Kito D. Mann
    Virtua, Inc.
  167. Thanks a lot for this report.

    If I can guess the Server A is WL
    (considering JRocket was mentioned).

    The Server B can be oc4j or WS.
    They mentioned Oracle drivers, but
    current version of oc4j supports 1.4.

    Similar to questions in previous
    posts, I would like if database
    times were directly measured or
    somehow estimated. Database can
    take from 5% to 95% of responce time.
    As bare minimum database cpu utilization
    can tell something.

    THanks again,

    AlexV.
  168. I don't think TSS has done a good job with this comparison, since we're still benchmarking apples against oranges and fairness is still an issue. Crux of the matter is usage of entity beans in J2EE Pet Shop, which, if I'm not mistaking, lack a direct analog in .NET framework. I could bet money on the fact that if J2EE pet store was reiplemented without EJB's at all or with Session bean only, the performance would increase at least two-fold and both applications would be on more equal footing. Plus, to have a more level ground database backends should be the same across both implementations. I'm not trying to shoot a messenger, but TSS folks should do a better job at perfoming fair and objective comparisons! Redo it!
  169. time to switch to .net! any tips on how to get started?
  170. The best site I know of for learning .NET is http://www.asp.net. You can download a free tool called Web Matrix that supports simple ASP development. Not as fully featured as Visual Studio, but the price is right. You can also find sample code, whitepapers, and so forth at the site.

    It's good that the discussion is moving away from religion and toward an analysis of the data. I look forward to future iterations of these types of comparisons.
  171. Ouch...
    I just when quickly though all the diagram, the results doedn't really looks good for J2EE.
    I think the J2EE world need to address the issues presented by this report because it can be a major weapon for MS evangelists.

    Great job to the ServerSide
  172. At first, congratulations on this benchmark. It is really time "we" (the Java community) open our eyes...
    The results were approximately as I expected them, J2EE to be quite a bit slower. Well, J2EE performed a bit worse than I expected I must admit, but I think with some tuning (JDK 1.4.1,...) it could perform somewhat better (they also used .NET 1.1 RC).
    On the AppServers, I'd also bet on A=WebLogic and B=WebSphere.

    What I really wanted to say is that I don't think Java by itself is slower than .NET/C# (I mean bytecode on JVM and MSIL on its VM performs roughly equal), I think its the J2EE framework that has much more overhead (just consider JNDI... I don't think there is anything alike with such overhead in .NET).
    Honestly I didn't look at the Middleware Company PetStore, but it really seems to me that the LOC comparison favours .NET, I cannot imagine that you can do the same with so much fewer lines...
    If I got that correctly they used EJB 1.1? Why that? I guess 2.0 would really reduce LOC and increase performance(?)

    I'm also with the poster who said that he'd also like an apples to apples comparison, not "best practices vs. best practices" (huh, not "I'd like this one better", I want both :-).

    A petstore shootout site would really be cool, where both communities develop a (or even several different) petstores on their respective platform... I think this would also greatly serve both platforms in finding strategies, patterns, ... (many tips, patterns, best practices etc. on both platforms are often at least partially based on assumptions, at least this is my impression).

    Thoughts?

    regards,

    Messi
  173. The comparison is apples to oranges. Specifically, the .NET solution is ASP pages making direct database calls, while the J2EE solution utilizes a full business tier utilizing EJBs. This also helps account for the LOC difference.

    There are reasons for architecting with a business tier, but there is a performance penalty associated with doing so.

    For a relevant performance comparison, strip out the EJBs and have the JSPs make database calls directly (perhaps leveraging off of the Oracle JSP custom tags).
  174. Jim, the .NET solution does have a complete, object oriented business tier. None of the ASP pages are making direct database calls. This was clearly indicated from the start of the report and audited by the Middleware company.
  175. The business tier used for the .NET arechitecture is only logical (and captive to the ASPs). The business tier used for J2EE is both logical and physical.

    For a fair comparison, use only a logical business tier in the Web container in the J2EE architecture.

    What's interesting is that with .NET, there is no option to physically separate the business tier. In J2EE you have flexibility to do either depending on the non-functional system requirements.
  176. <quote>What's interesting is that with .NET, there is no option to physically separate the business tier. In J2EE you have flexibility to do either depending on the non-functional system requirements.</quote>

    Huh? Are you talking about physically separating the frontend from the business logic? The .NET pet shop is a bad example because they don't do this BUT it is possible in .NET. Use remoting or web services - but you do have to design it that way from the start (in much the same way that you need to be pro-active about not putting business logic code in JSP's)

    With a project I am currently working on, it is set-up so that we can at compile time decide whether to use a remote or local facade for the middle tier.
  177. <Q>With a project I am currently working on, it is set-up so that we can at compile time decide whether to use a remote or local facade for the middle tier.
    </Q>
    How about deploy time? We are doing this. When the code runs it then is discovered how to communicate. Pretty simple.
  178. I told ya, EJBs suck!

    Don't use them. Period. Most J2EE patterns in this web-site are patches to EJB issues.

    Use Servlets, MDBs, MAYBE Session EJBs.
  179. Totally agree. Add these pattern to an already complex environment and you end up with a nightmare to test and
    mantain. See Allen Hollubs article about them
    in the Java Solutions supplement of C/C++ User Journal
  180. TSS fell into M$'s Trap.
    I hope next time they ask for help to Oracle,BEA and IBM.
    So, the vendors can provide truly optimizations.
  181. JDK 1.4[ Go to top ]

    Hi,

    Most J2EE servers are targeted for JDK 1.3. I wonder what the performance implications would be when doing (partial) rewrites of io critical code and using JDK 1.4 java.nio (buffers, non-blocking io, ...) classes. Even though the java.io package in JDK 1.4 takes advantage (behind the scenes) of java.nio capabilities, there should be an even greater advantage when explicitly using the java.nio package. For example: behind every io operation like JNDI, RMI stuff, ... a ServerSocket.accept() construction is used, which scales not as good as using a Selector.

    The .NET runtime has better integration with Windows and hence has many advantages when doing io stuff. The JVM is an abstraction on top of the OS resulting in a great deal of copying from buffer to buffer when doing io operations.

    I mean J2EE is mostly doing io stuff...
  182.  Heh the M$ trolls are so happy !!! But is . By the way this Pet Store is over kill to be written in EJB I think the Middleware people know that just try this with servlet, and datasource and see how it does. I bet then Java will be faster. Using precompiled store proc in Database does help a lot that what .Not used. One more thing I dont think either WAS or BEA runs on JDK 1.4 and if you force it like you did it here then it would have issues. I dont think there is any major vendor that supports jdk 1.4 for their EJB container I would like this to be answered by the Middleware people to prove me wrong. Sorry went to wrong post.
  183. I don't think we want to see an apples-to-apples comparision here, because what we want to see is a web application that delivers the appropriate functionality that uses the strengths of each platform: In .net, it's the tight integration between MS products, and on Java it's the flexability and openness (as in, you can use different app server proviers to host the application). I agree that EJB is completely mis-applied for this applicaton for the following reasons:

    1) JTS can handle the distributed transactions for the database updates, and I don't believe this requires the data access components to be EJBs. Removing EJBs from the picture removes the JNI overhead, and container management overhead. I think this app is straight forward enough that having complex container manaement is not worth the overhead.

    2) The database updates on the application are simple enough that they could be coded through standard SQL querries or even stored procs. We don't need the Entity Bean overhead here to manage the persistance.

    3) There doesn't seem to be a need for container level security, the data access objects (or business tier) can limit what orders/customer information is returned to the client without all the overhead of defining access control lists, etc.

    I think Sun has a pride problem because they aren't willing to take a black eye (by depricating EBJs) in favor over trying to make it work. In theory, EJBs sound great, with all the caching capabilities, etc, but in practice, it just doesn't quite deliver. Lots of things sound good in theory (such as a dynamic compiler) but when it comes down to the implementation, the proof is in the pudding.

    It's my hope that someone (maybe middleware co again?) will come forth and produce 1 more petstore, and this time, use stored procs , limit to session EJBs (if they really need that container services), and a simplified architecture build for speed, and solve the PetStore problem (which is, allowing users to buy products) without putting so much extra baggage on it that it can be easily changed into a full featured CRM solution. Simplicity is divine....

    -Chris
  184. Chris Knoll, I agree with your comments.

    However, you have to imagine who this benchmark is important for--if benchmarks are important at all. Is this benchmark important for the small/medium business? Or is it directed to the enterprise? I don't believe it is geared to the former since for the former, with a smaller user base and smaller requirements, .NET or simplified J2EE will do just fine.

    Considering the scale and load they are placing on the implementations, I believe this benchmark is geared to the enterprise, with very high scalability requirements. Unfortunately, the application is a Petstore, so while the implementations (using EJB or COM+ distributed transactions) are clearly overkill for just a pet store, you have to take a leap of faith and imagine that this is maybe an online bookstore, or something else with much larger requirements and 10s of thousands of users.

    So yes, EJBs are overkill for this specific project (just a pet store) and you can speed up Petstore dramatically without it. However, EJBs have long been the J2EE staple for enterprise class scalability in distributed systems. We have people in my own company developing using EJBs because it should make us more scalable.

    So, in short, I do want a benchmark applicable to enterprise class designs. If there are J2EE frameworks out there that do not have the overhead of EJBs, but provide the same level of scalability, and also offer cacheing, and distributed transactions, etc... I'd like to see those benchmarked also.
  185. So, in short, I do want a benchmark applicable to enterprise class designs. If there are J2EE frameworks out there that do not have the overhead of EJBs, but provide the same level of scalability, and also offer cacheing, and distributed transactions, etc... I'd like to see those benchmarked also.


    Servlets + Apache Avalon. It's just one of the options.

    Pet Store is to me the example of how NOT to develop the application. Using EJBs for that appl is a poor deisgn. It's a pure tutorial type of application.

    So, if you decided to use the tutorial sample to gauge the performance, then you ought to do the same with .NET. And it's simply impossible due to the current state of .NET. That's why the whole test is waste of time.

    Wanna compare the performance? Use servlets + some lihgtweight component framework and see what happens.
  186. I think there are several points to be made here.

        First, performance is a measure of raw transaction throughput. A highly-scalable system is one whose performance changes very little under load. A system can be highly-scalable without performing well. I know most people on this site know the difference, but we tend to forget it at times in the heat of the moment.

        An EJB system can be highly-scalable. EJBs can also perform well, assuming that the system being implemented requires a significant subset of the features of EJB. If it doesn't, then a system implemented without EJBs will perform much better. If it does, then few custom-developed frameworks will succesfully compete with EJB 2.x in maturity, performance, or scalability.

        As many people have said, this isn't an apples-to-apples comparison. It is still a very useful comparison, because this is the kind of bake-off that we will see in a competitive environment. TMC has provided a real service to the Java community, and (as always) the dicussion here on TSS is interesting and thought-provoking. I'm working with a client where this paper is generating a lot of buzz. So.... thanks, TMC!

        Also, someone mentioned the word arrogance WRT the response in the Java community to .NET, and I couldn't agree more. We in the Java community need to take this stuff seriously unless we want to find ourselves in a niche market. My two cents....
  187. [Hi Dave!]

    As long as we are drawing distinctions, we may as well point out that it is NOT necessary to prove, or even assert, that Microsoft products are slow or fast, or buggy or robust, in order to build a case for or against using them. Most of the reasons to use--or to avoid--Microsoft products are not technical.
  188. While entity EJBs might be the wrong approach for this application, there are other application types out there which aren't web based and for which (I personally) find entity EJBs to be of service (warehousing, certain EAI, etc) - and they scale and perform wonderfully, certainly at the EJB 2.0 level. If the world were all web apps, perhaps deprecating EJB might be a reasonable idea - but that's not the way the world is (mine anyway!). Why not work to a.) Clarify the roll of entity EJBs and b.) make the spec and implementation better? I know that many people I have seen build their entity beans by reverse engineering their database and trying to use EJBs specifically for persistance - which doesn't always work too well as we all know and to me isn't what they were designed for. If people need persistance only, there are plenty of other directions you can take.

    But anyway, as regards this silly Petstore application, I agree with another poster who suggested a Petstore shoot-out site. Mostly for fun I would guess - something for lots of people to post messages about. No doubt someone can use various aspects of J2EE at least to provide .NET a run for its money, and I would suspect that people could beat it, too.

    Cheers
    Ray
  189. I believe server B is websphere. If you define a EntityBean with CMP and check how many layers/classes get generated for persistence, you know why that server perform poorly.
    I don't know how good Java compares with C# at VM level, but for sure, if you redo the petstore with some low overhead data persistent framework, result will be much better.

    -Wanchun
  190. 1- Why did you use more and more CPUs instead of more and more single CPU machines, as recommanded by all the major AppServer vendors ? Would it be because real clustering is difficult to realize with .Net

    This leads me to my point 2 : I'm tired to listen that EJBs sucks, because they just do not. Writing an EJB CMP 2.0 is quite fast, specially if you use some smart IDE (anyway, you got to type the names of your data objects fields while designing your class diagram, so it would be stupid if at the same time your CMP wasn't generated...), and if you don't use Local Interfaces, you automatically get a perfectly scalable data object taking care of all the distributed transactions, of fail-over, load-balancing, ...etc.

    And point 3 : I don't give a f.. of performances if I know that when I will need to handle more requests I'll just have to easily plug another machine (can you do that with .Net?). Anyways since the very beginning of Java, performance is not the point !!!

    Oh one last thing : I use to work on a big sized sport web-site fully Microsoft designed : FIVE guys were there doing nothing but rebooting the machines because of replication problems between the boxes on which ran SQL Server (while some guys from Microsoft came many times and tried to solve the problems...) There were always problems and problems, the WebSite was hacked several times (IIS 4.0, remember?), ..etc. Maybe this kind of issue should enter in the price/performance metrics...

    Just after that, I turned to Java and J2EE.

    Philippe
  191. To add to postings about overhead of EJB, JNDI ? etc? you do not need to use all best practices for every application.
    So far MS owns Intel platform but it could change in future, but I do not think it will change with Windows platform. You could not beat performance of tightly integrated products with layered products. MS will always take advantage of windows (was not there some lawsuits ).
    it seems that the rules of the game were more favorable to .NET
    In all fairness it should be named:
    New J2EE vs .NET Performance Comparison Performed on the Wintel platform

    But in spite of this it is a great job by TSS it helped to clarify a lot of issues at least for me.

    oleg
  192. It might sound exotic in this thread but I think EJB 2.0 with local interfaces and CMP is a good thing.

    Look at all those projects left unmaintainable written in a style I call the M$ VB style (you can do that in Java too).

    The real problem is most developers can't deal with the high entry level J2EE affords. Do you really want to write all dumb SQL statements yourself (INSERT INTO, UPDATE, etc) ? This is boring and blows up the code with useless statements.

    EJB's provide a standard way of coding and creates programs that one can maintain. I don't think there are many developers creating a better framework themselves. There are not many Rickards in this world.

  193. I want to commend The Middleware Company for investing the time and resources in performing this test. I have performed tests on a similar scale and would suggest the following optimizations for anyone faced with similar issues.

    use generational garbage collection using at least 50% of memory for the young generation
    use about 512MB - 1GB of ram per CPU if possible
    use the thin driver, it multithreads better

    Thread counts are really less important than the CPU utilization, were the cPUs maxed in all tiers? If not check the db, if it is maxed then beef it up with more ram or cpus, if not, the app servers should be near 100% or they are bottlnecked, do stack dumps to see where they are hung up.

    It would be nice to see the cpu and io usage at the application and database tiers so that we can understand if there were any remaining bottlenecks in the code.

    Hope to continue the dialog, java can run fast too, but there are a lot of traps to fall into ;)
  194. The results of this analysis are conceivable, albeit somewhat surprising - certainly alarming. It would appear that every effort was made to keep the test on a level playing field. I am, however, almost surprised to see the Middleware Company post results that read more like a concession speech than an objective analysis. I was curious to visit http://www.middleware-company.com/j2eedotnetbench/report.shtml where I was asked if I may need help deciding between .NET and J2EE? Is the Middleware Company shifting gears - or am I a gloomy, incurable cynic?
  195. exactly what i was thinking. why would a company post something seriously hurting its own business???
  196. If A + X > C + Y and we don't know X and Y we cannot say A > C.
     
    If .NET + MSSQL is faster than J2EE + Oracle and we don't know the database configuration/contribution we cannot say .NET is faster than J2EE.
     
    The Middleware Company hosts ecPerf results and is very well aware what consitutes a test disclosure and how important a database contribution is. E.g., HP/BEA disclosure at http://www2.theserverside.com/ecperf/resultslinks/HP-ECperf-FDR.pdf describes both hardware, OS, and Oracle configuration.
     
    Oracle DBAs will correct me if I am wrong, but AFAIK Oracle performance on W2K could vary in a wide range depending on tuning (e.g. have a small number of rollback segments - from top of my head). Without knowing database configuration we really cannot assume Oracle and MS SQL Server perform identically (X=Y).
     
    And there is a very simple way to eliminate the unknown - run J2EE and .NET against both Oracle and MS SQL Server. It would also provide very valueable information about portability and compatibility, I bet our customers would love to see how well .NET works with Oracle. TMC had all pieces in place and ready and hasn't done that.
     
    Additionally, one of the key metrics is CPU utilization on app. servers and database. Microsoft recognized it and provided disclosure in their tests. TMC didn't.

    Sergey


  197. Will,

    I have a couple of questions:

    1) Was there any analysis done of where the performance was going (ie profiling)?

    It seems to me that there is some bottleneck in the J2EE application itself (or the db). I would not expect such a performance delta as we see here between what are two fairly similar platforms.

    2) Just a clarification really:

    Did the entity beans make any use of caching?

    The report does not specify what kind of J2EE caching was used (while there were details of the .net caching). The only mention was that the same level of caching was achieved with both apps.

    What caching was used? Read-Only beans in weblogic (not sure what Websphere's eq: is) or optimistic locking (both support this)?

    3) Did you try running the J2EE app against the SQLServer database for a sanity check?

    Cheers,
    Nick
  198. Please stop all the nonsense.

    First of all, let's come down and don't get personal or emotional. I know that it is very hard not to be because we personally have invested so much in learning one or the other. We have to really get away from this kind of disccussion/or arguments.

    First, we have to admit that Micrsoft has done a great job to make PC really user friendly. I remember my experience years ago when I used a word processor. The M$ word is now much better and easier to use. Microsoft has done a great job in pushing for user-oriented rather than programmer oriented software development.

    When it comes the field that J2EE is for, Microsoft is playing a catchup game. Don't be fooled by claims from any party. It is simple that there is a motivation behind a claim although it may or may not be honest and objective. For instance, why would the midleware company spend time and resources on "J2EE vs .NET Performance Comparison Performed"? The guy in the midleware company knows pretty well that .NET is not even a matured technology. On the other hand, I doubt if you can really be serious about comparing the performance between a language designed to run on many different platforms with one designed specifically for M$ platform. How can you, for instance, compare the speed between a car and a train?

    It looks reasonable to assume which technology, J2EE or .NET, comes out of this battle as a winner (maybe no winner at all), will depend on users rather than on programmers.

    In my opinion, this kind of arguments would not really serve J2EE good, if any. Here are a few personal opinions of mine if we really like to see J2EE advanced:

    1) present the business advantages that the platform independence of J2EE provides. Having systems locked in by one company is like to put a rope around your neck. Business people are smart guys. They know that it would be a bad idea to put a rope around their neck. One other advantage is that the flexibility from platform independence is to make best use of your existing hardware resources under the circumstance. For instance, if a business management knows that they need to have an application but it is impossible to anticipate the number of users at the time, they can still go ahead to develop the application using J2EE and deployed on a machine that is not fully used, be it a Windows NT/200/XP, or Unix or Linux. The business management has the choice of deciding migration to an adequate machine whenever it needs. They are the people in charge, not Bill Gates or somebody else.

    2) emphasize the importance of a matured technology. For a production system, nobody in their right mind would risk to use an technology that has never been tested in the real world by not-related users. I myself is pretty amazed by reports that claimed so and so % of applications will use .NET. It is puzzling why would a company like to become an experimental object of another company's technology unless it was paid for it. I remember somebody said somewhere that .NET would not have any chance of being known if it was not from Microsoft. We all know that is very true. However, the maturity of a technology does not depends on which company it comes from. How many years has passed and how many players are behnid before java was taken seriously for enterprise strength applications? Microsoft is great in getting their stuffs known to the world, one way or another. There is nothing wrong (perhaps except by posting butterfly posters for MSN8 around the middle town of NYC without any permission) because they are a business.

    3) take and present the advantage of rich open source java/J2EE resources. The open source java/J2EE projects are really very valuable and great. Excellent examples are Apache's list of java/J2EE/XML projects, forgiving me not to mention others since it is simply too many and not my focus here. Business people is doubtful (rightly so!)that there are sophisticated software out there for free. We need to let them know that it is the case and show them the license attached to the software if they question. The richness of the open source projects is a great asset that no one company can offer. It is my personal opinion that the quality of codes is generally or at least in many cases better from the open source projects. This is not to say that programmers not coding for open source projects aren't good programmers. Many (/or most) of them are excellent. However, a programmer on a non-open source project often has to make a choice when facing very tight deadline or over-worked. Codes from a close-door project do not have opportunities of being scrutinized by so many outsiders with no conflict of interest. When you use those open source projects effectively, it greatly reduces time for development and testing. For one thing I am sure of is that .NET doesn't come any closer in this respect.

    4) get a clear mind of understanding EJB. I read so much about people complaining about the complexity and poor performance of EJB. My opinion is that in such cases people often used EJB for the sake of using EJB per se. They complain about the technology rather than themselves. That may be the problem with some of these guys. You need to know exactly where and when you should or should not use a technology. You need pay very close attention on what problem you want to solve and weigh on different means you can use. You list their cons and pros before you settle down on one of them. If you used a hammer in a place where a screwdriver should be used, don't blame the hammer.

    5) welcome the challenge from .NET. I really think that it is good to the community and business users for J2EE to have challenge from some equivalently mighty player. We got one. We should consider it a previlige or an honor being challenged by .NET from Microsoft even if there is smearing stuffs out there. Microsoft does not go out and spends so much money to put a battle with other technologies of this kind. This challenge from M$ itself proves J2EE is a great technology.



    Apology in advance if some word here offend someone out there.
  199. Thankyou for this usefull motivation.
    I am afraid of the good Microsoft salesmen who
    can run away with this report by influencing
    management. We as developers can use such comments
    to give a more realistic picture.
    We must remember that there are a place for both these
    technologies and that they can coexist through Web Services.

    I want to see people start using the best database out there
    which is UDB from IBM.


  200. Great work - thanks for making it available.

    A few queries/clarifications...

    For J2EE version, what type of Entity Beans, commit options, pool and cache sizes were used? Were these optimised?

    For .NET version, how is persistence managed (sorry, don't know much about .NET)? i.e. is there a .NET Entity bean equivalent? How does .NET manage data load/store (and is it comparable to the way Entity bean persistence is managed?)

    I guess everyone knows that using Stateless Session Beans and DAOs/JDBC is substantially faster than Entity Beans in general, particulary for bulk row reading etc - we've observed double the throughput for our benchmark application and some products. This would tend to close the gap between J2EE and .NET :-)

    The other problem is the J2EE product selection. A and B may be "typical" or even leading J2EE products, but this doesn't mean they are necessarily the fastest J2EE products around. Based on an educated guess at what they are, I believe we have seen and evaluated another product which is twice as fast as A.

    If I've guessed right for product B I'm puzzled that it performed so poorly cf A (except for SOAP tests of course), and had problems under high loads etc.
    This seems like a configuration issue to me - again, if I've guessed right we've seen similar problems in the past with it, but were able to fix them (hint: one of the symptoms were excessive page faults).

    Regards,

    Paul.
  201. Perhaps I've answered my own question: From the MS site (snippet below) it seems that the Pet Store 2.0 .NET version uses the equivalent of Session Beans, and Java Pet Store uses BMP Entity Beans! Is this correct? In which case the comparison is more like "Apples and Peanuts":

    The Sun Java Pet Store makes heavy use of Bean Managed Persistence (BMP) in its middle-tier Enterprise Java Beans (EJB). Essentially, this provides objects a mechanism to persist their state to the database. The .NET Pet Shop takes a different approach. The middle-tier components make specific database calls for each business method invoked. Version two varies from version one in that for .NET Pet Shop Version 2, we have moved the SQL code to the middle tier rather than residing in the database as stored procedures. Using stored procedures at the database level is well established as a best practice for distributed applications. It provides a cleaner separation of code from the middle-tier and also helps clarify the transaction context and scope. In the original Pet Shop the stored procedures had only contained basic queries; with all the business logic kept in the middle tier .NET classes. With the new version of Pet Shop, we wanted to show that the performance of the application did not originate from the use of stored procedures, and that the underlying .NET framework used in the web and middle tiers was the source of the .NET Pet Shop's superior performance.
  202. hi Paul,
    You, and others, have made a couple of comments about the inferiority of entity beans to other approaches. Most of these statements seem to be based on a misunderstanding of the feature/benefits of entity beans:

    1. EJBs in general, and entity beans specifically, allow much greater app scalability than stored procs without needing to cluster a DB. (plus *numerous* other advantages -The StoredProcasaurus is a fossil of the client-server era.)

    2. Entity beans support dynamic queries (at least in WLS 6.x and 7.0)

    3. the assertion that "Stateless Session Beans and DAOs/JDBC is substantially faster than Entity Beans in general" is true in some cases (esp. without using ejb caching) but is less than 30-50% overhead in many cases. Your experience most likely indicates that the entity bean is not the appropriate way to model your application (again, if caching is not beneficial).

    4. EJB Maintainability, Quality of Service, re-use, portability, security, transactionality and tunability tend to outweigh raw performance in overall lifecycle costs...more so as applications become more complex.

    (Moore's Law and Murphy's Law are both on the side of EJB.)

    Matt

  203. Conclusion aren't really new.

    Research done at Rice University comparing Servlet based vs EJB architecture draw similar conclusions. You can find the research here:

    http://www.cs.rice.edu/CS/Systems/DynaServer/

    Section 6.1 of the paper "Performance and scalability of EJB applications" states it pretty clearly:

    Session beans offer performance comparable to the Java servlets-only implementation. All other EJB-based implementation methods fare considerably worse. Session facade beans and EJB 2.0 local interfaces are more than a factor of 2 slower than session beans implementations. Implementation based solely on entity beans experience and even bigger performance drop, regardless of whether they use CMP or BMP.

    There you have it, crystal clear, the performance you see difference you see in this report is in essence the difference between the 2 architectures, session beans vs entity beans. A Java based session beans implementation should perform comparably to a .NET implementation.

    EJB is a flawed concept, and many java developers have been saying so for years (see http://www.softwarereality.com/programming/ejb/index.jsp for a long list of problems). So, its about time that Sun, IBM, BEA and Oracle get real. It's time for a rethink of the architecture with out the baggage of OTM.

  204. Carlos,

    I could not agree more. I'm currently in the process of writing yet another set of benchmark applications based on Rubis for WebLogic and Oracle 9iAS application servers on top of MySQL and Oracle 9i databases to come up with objective conclusions regarding persistence mechanisms.

    Persistence mechanisms tested are:
    - JSP/SQL
    - JSP/DAO/SQL
    - JSP/session beans/DAO/SQL
    - JSP/session beans/BMP 2.0/DAO/SQL
    - JSP/session beans/CMP 2.0/SQL

    Next tests will include a web service layer. Other tests will compare Struts/non-Struts architecture. Finally, JDO and an object database (Versant Enjin) should be tested.

    We want to evaluate among others the overhead of entity beans and understand whether or not it is bearable.

    Can't wait to read "J2EE Performance Testing with BEA WebLogic Server" from Peter Zadrozny, who apparently comes up with conclusions that are quite unexpected: he advocates the use of CMP 2.0 versus other persistence mechanisms. Can't wait to read that book really...

    Yann
  205. Yann,

    Maybe we should all look at this report as being both a challenge and an opportunity.

    Clearly, there are many ways to "skin the cat" in the Java world. The main difficulty of anyone trying to make comparisons however is that its expensive to put together a test environment. www.ibatis.com showed a pet store implementation that was comparable to the .NET version, however a performance comparison was never done.

    Now that we have Microsoft's Tuned Pet Store benchmark, the entire java community should start taking cracks a blowing it away. Java strength is in choice. I can choose to implement Pet Store in a single vm, running say Resin and maybe an in memory database HSQL an I'm sure without a doubt its gooing to blow the socks of the .net implementation. Sure, its not fair, but everyone seems to be making their own rules with regard to the Pet Store comparison.

    So, folks, go ahead, grab either TSS's petstore implementation or www.ibatis.com, play around with the many Java technolgies out there, and lets just see how microsoft's .NET will fair.

    For starters, for a web container, I believer Resin to be the fastest. For a app server, I've heard good reviews about orionserver. There are tons of caching products like OC4J, Tangosol and Gemstone. Tons of OO mapping tools, like TopLink, BC4J, OJB, Hibernate etc. If going direct JDBC is your choice there are caching JDBC drivers.

    So folks, if the Pet Store is what these people want, let's give them a kick ass one!

    Here's my idea, Resin + Prevayler, kick ass fast, almost negligible code, fair comparison for you?
  206. Carlos,

    "Maybe we should all look at this report as being both a challenge and an opportunity."

    Definitely. A real-life architecture with those requirements would probably not use BMP 1.1 and remote entity beans. I doubt it would even use entity beans at all. I guess it could be interesting to see how a real life J2EE solution compares to a real-life .NET application. I imagine .NET would still outperform J2EE through native calls but the difference would certainly not be that huge. There is now a clear opportunity to use a real-life J2EE architecture and compare it to .NET. I'm not sure it would be that much of a challenge though... perhaps for the web services part...

    And for many of us, I think it is a good time to learn humility too. :) This report was badly needed in that way: I bet the J2EE community will take the shock and bounce now.

    In any case, this is a great report and I would like to congratulate (and thank) the people who put it all together. Great work!

    Yann
  207. Go to Ted Osborne's web site (http://www.tedosborne.com) and download the source code and PowerPoint presentation.
  208. Carlos,
    The myth you( and others) are operating under is that
    JBOSS/Jonas' are comparable to the commercial products...

    The rice report even states:
    " Container design ...has a direct impact on performance with entity beans."

    CSIRO (http://www.cmis.csiro.au/adsat/jboss.htm) determined that JBOSS SB performance is almost 350% faster than JBoss CMP Bean Performance! AND that JBOSS Session bean performance was "about half of the leaders".

    (That means that Weblogic CMP beans could be faster than JBOSS session beans -not even taking into account WebLogic’s caching abilities for Read-only and Read-mostly CMP beans, field grouping optimization, etc. which would increase the performance gap even more.)

    later,
    Matt
  209. Shooting J2ee & .Net petstore[ Go to top ]

    Ok.. what about a web site dedicated to the both,J2ee and .Net version of Petstore where both communities try to prove what they claim about their technology.
    I can do that
    Faisal



  210. Paul,
    You are quite right - all of the database access is via BMP.

    Why BMP over CMP?
    Its little wonder code volume count is so high.
    How is it an all-apples comparison when compared to the Session-Bean approach used in the .net app?

    Also, with regard to performance, I have had a quick look at the code and the Weblogic descriptors and I have some additional questions:
    Will the settings used (below) work for BMP beans?
    I thought they were for CMP - any Weblogic guru's care to comment? Cedric? How will WLS treat these settings for a BMP Bean? Will the outcome be the same as for CMP?

    <concurrency-strategy>ReadOnly</concurrency-strategy>
    and
    <concurrency-strategy>Optimistic</concurrency-strategy>


    In general, this result doesnt look right.

    A) The difference between the two J2EE appservers seems too large. This doesnt tally with either the ECPerf or the CSIRO benchmark results. (CSIRO is also done on identical hardware)

    B) The difference between the .net and the J2ee application seem to be amount to more than Virtual Machine / appserver differences. Until we get some profiling info, we are not going to know.


    Regards,
    Nick
  211. The WebLogic-specific DD element makes no difference BMP or CMP WRT the ejbLoad() callbacks by the container (ejbLoad is not called before the timeout is reached). However, since BMP, not CMP is used by these guys, the n+1 database calls (vs 1+1) overhead would kill any hope of performance, even the beans are tagged as ReadOnly.
  212. <quote>
    However, since BMP, not CMP is used by these guys, the n+1 database calls (vs 1+1) overhead would kill any hope of performance, even the beans are tagged as ReadOnly.
    </quote>

    Yes, WLS7 CMP is much much faster than BMP.
    CMP provides:
    - store optimization.
    - solution for the N+1 problem on finder methods.
    - optimistic concurrency and caching.

    If you can accept optimistic characteristics, please try
    concurrency-strategy=Optimstic,
    cache-between-transactions=True,
    in your applications.
    It is acceptable for almost all the web appliations.

    Using CMP with avobe options will also reduce LOC greatly.
    You do not have to use even Read-Mostly ptterns.
  213. SUGGESTIONS to the developers at the Middleware Company:

    First, add a disclaimer to your report clearly stating you are using an older version of the EJB technology.

    Then, get WebLogic to fund a rewrite of your app, using WebLogic 7.0.1, JDK1.4.1, and as much WLS-proprietary features as possible. DEFINATELY use EJB 2.0 CMP ReadOnly entity beans. Rerun the benchmark tests, and publish the new result.

    Thanks.
  214. Hang on, Something doesnt look right here...[ Go to top ]

    ...looks like a lot of people are good
    in suggestions about what TMC/TSS must
    do. There is old saying:
    "Everybody is the best strateg looking
    at the battle from the hill".
  215. First, I've built reasonably complex web apps in the past year with both j2ee and .net. There are some great advantages to both. .NET has a really nice, integrated web development environment. I'm not just talking about drawing things; I'm talking about really nice debugging and error messaging. They also have a nice thing called ViewState which can persist objects behind the scenes, encrypted, on a web page. If you don't feel like getting rid of objects and you can deal with the performance overhead, it's a nice option.

    That said, I prefer working with j2ee. I have more options. I can choose from countless dao frameworks, web frameworks, etc. I can use EJB if appropriate. I have asynchronous messaging at my disposal (.NET has NOTHING like JMS and that's a major setback). The web application I recently built is running for nothing but the cost of the web server. I'm running mysql, tomcat, and openjms, and not paying a cent for any of it. I'm running it on red hat 7.2 and not paying for that.

    Some thoughts about this comparison:
    1) The middleware company has been around for a while and I recall a great deal of info they were pushing a few years ago talking about how the COM+ stack was superior to the J2ee stack in its early stages. My impression is that their business is in MS-based development. Take that for what it's worth. I'm not saying that their article wasn't done with best intentions, but if they don't themselves sell anything j2ee related and they put together a doc on how that doesn't matter because what they do provide is better, you draw your own conclusions.
    2) EJBs should not have been used at all in this comparison. They are totally unnecessary for this type of application. You need tomcat, struts, and a persistence mechanism. I recommend a lightweight OR tool like hibernate (hibernate.sourceforge.net). I know what the various mechanisms for .net DAO are and hibernate is superior in every way. It ends up being direct jdbc calls but the developer writes config files. THAT'S IT. No code except to handle sessions for accessing the store to get or write your objects. No "fieldA = field_a" crap. None. I would guess that it works out to less lines of code by a LONG shot if you remove EJB and use something like hibernate. I would guess that the speed is comparable, as well. But I'd like to see. For those who say that EJBs are useful in some cases, I agree, but that's addressed next:
    3) There are cases where EJBs are useful, from a management and scalability standpoint, or for dealing with integration, remoting, or other complex issues. A web app is not such as case. For those cases, though, does .NET have an equivalent solution? I don't think there is one available. There is no equivalent to the connector api, or JMS, etc., etc., etc.
    4) Can someone please, please do an apples to apples comparison, preferably using open source software on the Java side.

    Bottom line, both products are great. .NET is very powerful in the low to mid range, especially if you have an investment in MS software and talent. I just don't think that it has anything close to some of the high-end enterprise features that J2ee offers. J2EE is just a more complete enterprise stack. And this comparison is still not the "nail in the coffin" that .NET supporters would like to make it because it's still not a straight comparison. Even if the J2EE stack were slower in a more fair comparison, the cost, freedom from vendors, and freedom of available, proven solutions across the board (patterns, software, hardware choice, etc.) would be enough to sell me on J2EE.

    Disclosure: I am a contractor. My bread-making contract right now is in .NET. Take that for what you will.
  216. What is MSMQ if not JMS before JMS came out?

    What I don't get is how everyone says EJB's should only be used for "big-ass, complex apps". If it can't work for simple apps then how can it scale to work for much larger, complicated systems? It either scales (up and down) or it doesn't.

    Oracle can be used for simple or difficult tasks. Same with Linux.

    Now, I will admit that in terms of development time & complexity, that is a good issue over whether to use EJB's or not. After all, poorly trained programmers will use EJB's to their disadvantage.

  217. This is the closest, even though not exact, comparison between a J2EE application and a .NET application to date, with very impressive results for .NET that will hopefully draw a lot of attention in the J2EE camp and spark better attention to performance and productivity (specifically, development environment) issues in J2EE.

    Here are a few quick remarks on deficiencies, in my view, in this comparison:

    - Use of different databases for .NET and J2EE.

    - .NET Enterprise Services components rely on COM+ unmanaged code behind the scene, which, theoretically, should be faster than the coming managed code implementation and does not use benefits of Garbage Collection. The equivalent in Java would be using JNI.

    For database persistence layer BMP (Bean-Managed Persistence) 1.1 was used - 1st, 2.0 BMP specification is considerably optimized; 2nd, BMP calls require more code on the developer's part and are significantely slower for finder methods (n+1 vs 1+1 for CMP if I'm not mistaken).

    There is no equivalent of BMP in .NET framework - thus, it should not have been used at all for the purposes of this comparison.

    For the cost of the hardware/software setup: J2EE can be reliably run using free open source servers that have proven themselves robust and secure: Apache, Linux, Tomcat, etc. Although, the expertise level required to properly use such a setup has to be significantely higher than for using .NET servers; thus, possible higher costs of the staff.

    I would definately like to see JMS vs MSMQ performance and implementation complexity comparison.

    All in all, I am very impressed with the results for .NET framework.
  218. What is MSMQ if not JMS before JMS came out?


    Message queuing APIs have been around a lot longer than either Java or Windows NT, so it's safe to say that both are reimplementations of earlier works.

    > What I don't get is how everyone says EJB's should only be
    > used for "big-ass, complex apps".

    Simple. Performance is usually measured in "Big O" notation. Quick sort takes O(n*log(n)) time to sort a list while bubble sort takes O(n^2) time to sort a list. It's clear that quick sort scales much better than bubble sort for large values of n. However, for small values of n (n<6), bubble sort is better. Why? Simply because there's a large constant time associated with the setup of a quick sort. That setup time is insignificant when the size of the list is large enough, but it is significant for small values of n.

    If the fixed overhead of Entity EJB's is larger than the performance gain of using Entity EJB's, EJB's shouldn't be used. In small apps, the performance gain is less, so the overhead would be large relative to it.

    > If it can't work for
    > simple apps then how can it scale to work for much larger,
    > complicated systems? It either scales (up and down) or it
    > doesn't.

    Overhead in the wrong places can have a dramatic effect on scalability. Consider the example of copying a string. A naive C++ of this would be as follows:

    for(int i=0; i<=strlen(mystring); i++) buffer[i]=mystring[i];

    Can you see what's wrong with it? The string copy should be a linear time operation, however, the above example copies the string in quadratic time. It definitely doesn't scale well.

    The key problem is that the "trivial" operation of finding the length is used continually by the copying routine. This results in quadratic behaviour.

    It's very easy to create the same sort of nonscalability if you use the wrong J2EE (or .NET) technology in the wrong location in your code.
  219. What the F**K?! I downloaded the code and the first thing that strikes me is that the Entitybeans are all Remote?!
    This thing is optimized?! H**l no!
  220. Note remote entity beans? This smacks of having to resort to remotes because of the lack of support in the J2EE application servers tested.

    WebSphere 4 certainly doesn't support them, 5 does but from what I understand is only on early access, and can only support DB2, which would rule it out of this test?

    In order to do a fair comparison between the J2EE app servers, full J2EE 1.3 would have been required by both wouldn't it?

    Also, I thought that WebLogic 6+ was clever enough to detect if only getters were called on entities so avoiding unnecessary db writes

  221. Remote sounds about right for what COM+ is doing. To make a call to COM+ from the webserver (IIS) the request needs to leave my process and find in the registry where the component is that it needs to make a call to. In this case the components are host on the same machine, but it's still on out of process call with marshalling and other overhead.
  222. Actually no, COM/+ can run in or out of process, and anyone that was going to optimize an app that used COM making it call in process would be the first place to start.
  223. What is MSMQ if not JMS before JMS came out?


    1.) MSMQ is only supports Point to Point messaging
    2.) It is coupled to the Windows Domain Structure (AD in Windows 2000)

    As an individual who works with both, I can say MSMQ doesn't hold a candle to JMS.
  224. What is MSMQ if not JMS before JMS came out?


    Nathan: "1.) MSMQ is only supports Point to Point messaging 2.) It is coupled to the Windows Domain Structure (AD in Windows 2000) As an individual who works with both, I can say MSMQ doesn't hold a candle to JMS."

    Furthermore, JMS is a spec, an API. You could implement the JMS API (or parts of it) to run over MSMQ if you wanted to. It's already been implemented over dozens of other messaging systems, including the messaging systems that MSMQ was a copy of.

    Interestingly, early implementations of some of those messaging systems were around when Microsoft was incorporated back in the '70s. Software never dies, it just fades away ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  225. I've been looking at MSMQ 3.0 recently.

    I thought that MSMQ didn't support pub/sub either, but from reading the docs, it appears that they now do support it.

    They call it "multiple-destination" messaging.

    I have yet to prototype with it (need to get Active Directory set up first).

    From reading the docs, it looks pretty impressive. You can do sophisticated 2PC, routing etc.

    The rub is you have to use Active Directory and MSMQ 3.0 is "unsafe code".

    This shouldn't matter for small impls, but I read on the discussion forum that this can become a problem when you are talking about 1M messages/hour etc.

    It would be great if MSMQ would implement the JMS spec so Java could bang into it. There still would be a problem on the other end of the queue, however, in talking back to Java. The only solution I have come up with is having a MSMQ trigger fire a web service consumer (implemented with C#, VB.NET, etc.) that bangs into a Java web service.

    It will be very interesting to watch the messaging area develop. True interop with messaging would be great.

    Mike
  226. How would the result be if one used TowerJ instead ?
  227. Conclusion:

    .NET is *much* better, faster, cheaper than EJB.

    J2EE w/o EJB is *much* better, faster, cheaper than .NET.

    Where is a good example of Good practices of J2EE w/o EJB

    Answer:
    basicPortal.sf.net, that is the URL.

    Uses Struts, regular JavaBeans, Struts and a great DAO design pattern.
    It is just a JavaBean with RowSet, similar to http://www.javaworld.com/javaworld/jw-02-2001/jw-0202-cachedrow.html.

    basicPortal.sf.net is best practices, it is open source, it is free, it is MVC, did I say great DAO using RowSet (not ResultSet) very fast in production loads, and very simple.

    Linux has more Server Side Market Share than MS.
    Apache has more Market Share than MS. (netcraft.com)
    Why go with a 2nd place team:

    Tomcat from Jakarta.apache.org = Free, no run time or resale, you can recompile it and resell
    BasicPortal = Free, Ditto.
    PostgreSQL.org = Free, Stable, Fast, no run time cost.
    Linux = FREE ? RedHat 8.3 is a free download, as required by Linux license, RedHat can charge for the CDs.

    (If you have a site that has 40,000 concurrent users it is to expensive to use MS SQL, you must must use the free PostgreSQL)

    Training on Struts was voted as best hand on training by JDJ by baseBeans.com.

    So basicPortal is faster, and cheaper. Great training, very little code.

    And there is plenty of open source competition. Commons-SQL.
    Ibatis, etc. Support on mail lists like Struts is best money can buy (except it?s free)

    Linux! Struts! Tomcat! pgSQL! And roll your own beans with RowSet or DAO open source.

    I challenge any .NET design to go against open source performance and cost, Pet Cemetary or other. What does it take to test real world J2EE w/o EJB?

    .V
  228. I have do an extensive benchmark of various measures of performance or SUBs. For my test I choosed a red Ford SUB and two other green SUB from different vendors, we will call it SUB A and SUB B. After extensive reseach the re Ford SUB outperforms the SUB A and SUB B.

    My conclusion, if you want to buy an SUB buy a red one, they way outperform green ones. The maker and models of the SUB A and SUB B should be completely irrelevant to you.
  229. The authentic SUNW Java PetStore had gone EJB 2.0 CMP with local interfaces since version 1.3. For what reason the Middleware company J2EE "gurus" decided to turn back the clock and implemented an "optimized" PetStore using BMP with DAO? For BMP, how come Gene Chuang's Fat Key pattern is not used? I strongly urge the benchmark team redo their homework and rewrite their app using SUNW's Java PetStore 1.3 as their codebase!
  230. <Eric Ma>
    The authentic SUNW Java PetStore had gone EJB 2.0 CMP with local interfaces since version 1.3. For what reason the Middleware company J2EE "gurus" decided to turn back the clock and implemented an "optimized" PetStore using BMP with DAO?
    <Eric Ma>

    Cos MS does not have an equivelent for CMP. (ObjectSpaces might change this). The .NET Petstore implmentation uses handcoded SQL. thats same as using BMP with DAO. I think thats why TSS went for BMP.
  231. The authentic SUNW Java PetStore had gone EJB 2.0 CMP

    >> with local interfaces since version 1.3. For what reason
    >> the Middleware company J2EE "gurus" decided to turn back
    >> the clock and implemented an "optimized" PetStore using
    >> BMP with DAO?

    >Cos MS does not have an equivelent for CMP. (ObjectSpaces
    >might change this). The .NET Petstore implmentation uses
    >handcoded SQL. thats same as using BMP with DAO. I think
    >thats why TSS went for BMP.

    If I understand you correctly you are saying that, for the sake of a fair and honest comparison(!), the J2EE PetStore should not be allowed to use any J2EE features that might improve performance if those features do not exist in .NET?! And, by logical extension, that the .NET PetStore uses no features that are proprietary to .NET? - I think you'll find it does...

    I think you (and the TheServerSide.com) are deliberately missing the point. .NET and Microsoft's .NET best practices (as used in the .NET PetStore) are obviously aimed at performance (on one OS and preferably against one particular database). J2EE and Sun's J2EE design patterns / best practices are aimed at creating portable applications that are robust and easy to maintain.

    There is no way you would use entity beans (BMP or CMP) if you were aiming for outright performance alone. Hard-coded (sorry, dynamic) SQL is always going to be faster than entity beans because entity beans add another layer of abstraction. However, entity beans are not included in J2EE because they provide faster performance, they are there to provide a standard way to persist your domain model in an OO fashion and to encapsulate the persistence mechanism.

    The benefit of entity beans is that they allow developers to create more maintainable and robust code which, especially on larger projects, is usually much more important than the sort of performance differential seen in this (seemingly rigged in MS .NET's favour) comparison.

    But then let's face it, this was never set up to be a fair fight over the issues that really matter!
  232. Let's face it. It doesn't matter about the underlying technology. It matters about the performance. If you really wants Java to compare with .NET, Come on, let's hard code everything and forget about J2EE so that you can compare.

    Any real benefit from J2EE? Longer code.... Slower performance.... "Better abstraction" Who care?
  233. Let's face it. It doesn't matter about the underlying

    > technology. It matters about the performance. If you
    > really wants Java to compare with .NET, Come on, let's
    > hard code everything and forget about J2EE so that you
    > can compare.

    > Any real benefit from J2EE? Longer code.... Slower
    > performance.... "Better abstraction" Who care?

    I am confused! Is this a J2EE fan being ironic, or a .NET stooge who has no knowledge of computer science??? Perhaps I should answer in a similar vein:

    > Let's face it. It doesn't matter about the underlying
    > technology. It matters about the performance. If you
    > really wants Java to compare with .NET, Come on, let's
    > hard code everything and forget about J2EE so that you
    > can compare.
    If outright performance is so important to you, and hard-coding rocks your boat, then have you considered writing your apps in machine code rather than .NET?

    However, if you need more performance but also want robust and maintainable code then (now, how's this for an idea?!) you can use J2EE instead of .NET and just add some more hardware (which, by the way, is ever so cheap compared to development, management, and maintenance required by poorly architected applications). And if the Wintel architecture stops scaling? - You can leave Wintel behind and move to a more performant OS/Hardware platform.

    > Any real benefit from J2EE?
    Er... More robust and maintainable code, portability to other OS/Hardware platforms, no vendor lock-in, etc.

    Personally, I find that not directly or indirectly contributing to a company which has been proven in the courts to be guilty of underhand business practices gives one a certain sense of moral integrity. Then there is the self-respect that comes from actually understanding distributed programming rather than just clicking on a wizard and hoping it works and running crying to technical support if it doesn't. There is also a sense of satisfaction in knowing that your well-crafted J2EE app using Design Patterns (don't be scared!) will be used for years by grateful users, rather than suddenly having to be scrapped/re-written when MS have another brain-wave and break compatibility with your existing code (e.g. VB6 -> VB.NET).

    > "Better abstraction" Who care?
    Perhaps I could suggest you check out Amazon.com for some beginner's guides to Object Oriented programming.

    If I can be of any further assistance in your quest for enlightenment then please let me know...
  234. Inconsistency[ Go to top ]

    Cos MS does not have an equivelent for CMP. (ObjectSpaces

    > might change this). The .NET Petstore implmentation uses
    > handcoded SQL. thats same as using BMP with DAO. I think
    > thats why TSS went for BMP.

    Does .NET have anything like entity beans? From what I know, it doesn't -- you have to use DCOM services to do that. So by your reasoning, they shouldn't have used BMP either. At the very least, if they used BMP, they needed to convert the remote interfaces into more efficient local interfaces.

    From my reading, the .NET example is a lot closer to Struts+JSP+JDBC (or JDO) than EJBs. So technologically, the J2EE petstore doesn't match either.

    The key question is, what is TSS trying to measure?

    If they're trying to show the best practises as illustrated in PetStore, they need to use CMP and local interfaces.

    If they are trying to compare .NET and J2EE, they need to use comparible technologies.

    You can't have it both ways.
  235. Inconsistency[ Go to top ]

    <Robert Devi>
    Does .NET have anything like entity beans? From what I know, it doesn't -- you have to use DCOM services to do that. So by your reasoning, they shouldn't have used BMP either. At the very least, if they used BMP, they needed to convert the remote interfaces into more efficient local interfaces.
    <Robert Devi>

    I have not gone though the .NET Petstore code. but i understand they use COM+ servieces. (are they?). If thats the case they have all the disadvantages of remote interfaces. beside local interfaces are not always outperform remote interfaces. (I think app server optimizations do a better job in this).. More about this topic in this thead.

    http://www2.theserverside.com/home/thread.jsp?thread_id=10856

  236. GLUE saves the day![ Go to top ]

    Looks like GLUE, written in Java beats .NET in its own game.

    http://www.themindelectric.com/glue/index.html?../products.header.html&developers.html&../bottom.html

    Click on the "benchmarks" link.

    That is hosting webservices.
  237. Inconsistency[ Go to top ]

    <pratheep>
    > I have not gone though the .NET Petstore code. but i
    > understand they use COM+ servieces. (are they?). If thats
    > the case they have all the disadvantages of remote
    > interfaces.
    </pratheep>
    I've been out of the loop with COM for a while, but from what I remember, COM's overhead is little more than a shared library load. There is marshalling if you're using a language like C#, but the marshalling is to the local server. Basically, COM+ is analogous to EJB Local interfaces.

    <pratheep>
    > beside local interfaces are not always outperform remote
    > interfaces. (I think app server optimizations do a better
    > job in this).. More about this topic in this thead.
    </pratheep>

    True. It seems like some J2EE servers optimize out the difference between local and remote interfaces. Under light loads, you shouldn't notice much of a difference.

    But is this the case here? Without knowing the software or the versions of the J2EE servers being compared, it's impossible to say. The only thing I do know is that the RedHat Linux version mentioned in the report is two versions behind (RedHat Linux currently is at 8.0 and the previous version was at 7.3) and the Entity bean model being used (EJB 1.1 BMP) is also fairly old and the pet store example they wrote is modelled after an old version. Given this all this, I'd expect that old versions of the appserver are also being compared. Perhaps they wrote the examples using BMP because their version didn't support EJB 2.0 CMP. Does this old J2EE server version have remote interface optimization? Only the Middleware Company knows.

    Given all this, I'd say that the only way to remove all doubt is to use local interfaces.
  238. Inconsistency[ Go to top ]

    .NET System.EnterpriseServices (COM+) provide functionality equivalent to Entity Beans.

    They do not have CMP, but then again that should be a disadvantage to .NET not J2EE.

    From MS...
    "COM+ services, such as automatic transactions or Queued Components (QC), are configured declaratively. You apply service-related attributes at design time and create instances of classes that use those services. Some services can flow from one object to another. For example, an object configured to require a transaction can extend that transaction to a second object if the second object also supports or requires transactions."

    Therefore, the .NET example here is not just Struts+JSP+JDBC.

    The Java vendors should be able to use either CMP or BMP depending on which one performs better. If you remove Entity Beans entirely, then .NET should also be allowed to remove the usage of COM+ services (which do add overhead).


    BTW, WebLogic will treat remote interfaces as local if within the same JVM, this was true back in version 5.1.

  239. Inconsistency[ Go to top ]

    .NET System.EnterpriseServices (COM+) provide functionality >equivalent to Entity Beans.


    >They do not have CMP, but then again that should be a >disadvantage to .NET not J2EE.

    >From MS...
    >"COM+ services, such as automatic transactions or Queued >Components (QC), are configured declaratively. You apply >service-related attributes at design time and create >instances >of classes that use those services. Some services >can flow >from one object to another. For example, an object >configured >to require a transaction can extend that >transaction to a >second object if the second object also >supports or requires >transactions."

    Lets take both the compiled code as is for .Net and Java we have here and run it in Linux and lets see which one runs better. We know most of enterprise application runs on UNIX or Linux.

  240. I downloaded the code, and had a look at the Java implementation, and one thing is clear; they're pretty silly about exception handling. There are gazillions of cases like this:

        } catch (OrderDAODBUpdateException se) {
            context.setRollbackOnly();
            throw new CreateException (se.getMessage());
        } catch (OrderDAOAppException oapp) {
            context.setRollbackOnly();
            throw new OrderAppException(oapp.getMessage());
        } catch (OrderDAOSysException osys) {
            context.setRollbackOnly();
            throw new EJBException(osys.getMessage());
        }

    That explains some of the code overhead. Not all, of course, but some.

    Mats
  241. Is there a technical explanation?[ Go to top ]

    So what are they saying is the reason for the difference in performance? I would be reluctant to believe whatever numbers these guys produce if it is not backed up by some kind of technical explanation.


  242. "- 2 man-weeks verses 10 man-weeks to tune for performance"

    In practice, the performance difference would be much greater because expert tuning like this, Oracle et all, is beyond the average J2EE programmers/admin capability.

    Second, I think it strange that the hardware cost is included in the cost. In practice you already have the hardware.

    Third, in my opinion the second most important issue is:

    "- 2096 lines for .NET verses 14,004 lines for J2EE" with what it means for maintainability and development cost.

    Forth, the strangest thing with this whole discussion is that does this experienced group (the TSS community) never download and try anything on their own?

    Fifth, why are J2EE (and EJB) so often used in trivial projects that would have been much better off by using something simple like PHP or Perl?


    Regards
    Rolf Tollerud
     
  243. And how can I get the source code for PetShop?
    Not a msi file, just a source, like it was
    available for PetShop 1.5.2

    Thank you,
     Dmitry Namiot
     Coldbeans
  244. Looks like they were down again for an hour.

    Geez, my NT4 servers are more stable... ;-)

    Maybe the size of this thread is swamping them...(hint hint)
  245. And how can I get the source code for PetShop?

    >Not a msi file, just a source, like it was
    >available for PetShop 1.5.2

    I got a w2k machine. First of all, I need to install .Net framework, which require MDAC (Microsoft Data Access Component). Ok, it is equivalent to installing JVM and JDBC driver. After installed these, I run the msi and ... it requires MSSQL server. No need for benchmark, .Net Petstore is a loser for it cannot run on other DBMS!

    Does anybody have mssql server and get the petstore successfully installed can share the source code? (if license is allowed)
  246. Sam,

    Give me an e-mail address and I will mail it to you.(only 350k zipped)
  247. .Net Source Code[ Go to top ]

    Just run the msi install and when it asks for MSSQL settings, go to the file system and make a copy of the installed directory. When the installer undo the installation, it isn't smart enough to remove the copy also. :)
  248. I've read more of the J2EE code now, and some code can be shaved off by making various closeConnection(), closeResultSet() and closeStatement() functions static, and put them in a JDBC helper class.

    My guess is that 100-200 LOC can be saved that way, very easily. Yes, yes, I know, doesn't make that much of a difference, but I'm just letting everyone know. I assume others can find similar places to save code, and it may all add up to be significant.

    Mats
  249. Whatever!
  250. How did they come up with the number of lines of code? Did they use JavaNCSS, or something else?

    Mats
  251. OK guys....time to wake up.

    When the Middleware Company, the company that hosts this site, is telling you that J2EE is a piece of sh*t...why dont you listen?

    The excuses are over.....the comparisons are fair...and even "Your J2EE Community" could not make J2EE look good.

    I tried Java this year... The tools are junk..the code is dog slow...and the platform is a cluster-f*ck of config files. Take a look at .net. Hate Microsoft all you want, but the stuff works...right out of the box.

    Give the .net platform a shot. See why MS developers dont convert to J2EE. Great tools, fast code...no downtime...and you dont need a PHD to get an aspx page to come up.

    My company implemented a JSP/Servlet solution, and a VB/ASP COM solution in parallel. Guess which one is a support nightmare? Guess which one crawls? Guess which one grinds to a halt for no reason? I am sure you guessed by now...

    Its not too late. Abandon the crap shareware pipe dream....before Sun goes under.
  252. Look for .NET 1.1 to come out soon. Looks like .net 1.0 is already the Java-killer.
  253. Yup, it's time to wakeup. I mean not to wakeup and switch to .Net but time to wakeup and reduce J2EE's complexity.

    I think this benchmark clearly triggered the countdown for the death of EJB. I'm so happy about it, I'm specially happy because no customer or coworker or consultant will recommend EJB any more, the hype is over!

    Btw, am I right that they didn't use Struts or any other web framework? And why not ejb2 cmp2? And why XA datasources?

    Use javabeans, a nice OR mapping frameowrk like Hibernate, a nice web framework like webwork or Tapestry, employ caching with Tangosol Coherence and/or use Quartz for scheduling, use Jakarta Commons-SQL for the listing behavior, use a cool IDE like IDEA and XDoclet for attributl programming and code generation, employ Junit and mockobjects, and a fabulous build system like Maven, and be sure the app will be fast/manageable/reliable/maintainable/easy-to-code. This is imho a best practice, not the crappy best practices from Sun. Out of the box J2EE recommended by Sun/IBM/etc just sucks. They just want to sell you something priceier.

    Ara.
  254. Talking about complexity: all these names EJB, XDoclet, XA, .Net, JTA, J2EE, COM+, CMP, ADO, JAX, JCP, JDO, etc, etc, etc remind me how simple is life in a client/server achitecture.
  255. "I tried Java this year"

    Let me tell u one sad story.
    Last year I worked for one company that have a teams with a lot of M$ experiense. They thought to move their asp site to Java. Guess what was the result?

    Yes, the guys that "tried Java this year" did nothing.
  256. Tom,

    <quote>
    When the Middleware Company, the company that hosts this site, is telling you that J2EE is a piece of sh*t...why dont you listen?
    </quote>

    Because they haven't said any such thing. They have found in the particular scenario they have chosen, that .net performs better than J2EE. Only those who do not know J2EE will suggest that .NET is therefore better.

    <quote>
    The excuses are over.....the comparisons are fair...and even "Your J2EE Community" could not make J2EE look good.
    </quote>

    I'm not sure that the J2EE community would have build the application the way TMC built it. Now I know that's somewhat confusing for the MS community, given there is one vendor, one platform and one way to build something with one tool, but with J2EE you have a variety of ways to approach a given problem. TMC chose one of those paths. If they hadn't tried to constrain their J2EE approach to copying what they thought MS was doing and instead used J2EE in its own right, I have no doubt that the two apps would be on a more equal footing.

    <quote>
    I tried Java this year... The tools are junk..the code is dog slow...and the platform is a cluster-f*ck of config files. Take a look at .net. Hate Microsoft all you want, but the stuff works...right out of the box.
    </quote>

    Those who are used to the MS paradigm will not find Java to their liking. There aren't the drag-n-drool GUI tools and when faced with actually writing some enterprise code, it can be a daunting task. However Java can be quite fast programmed properly and the config files required with J2EE are hardly rocket science. You shouldn't use your lack of understanding of Java/J2EE as a basis to spread FUD.

    <quote>
    Give the .net platform a shot. See why MS developers dont convert to J2EE. Great tools, fast code...no downtime...and you dont need a PHD to get an aspx page to come up.
    </quote>

    While I can try (and have) .net on my home PC, I work in a heterogeneous environment at my client sites, with our application server boxes being Unix based - and try as I might, I can't make .net work in that environment. I can't disagree with your statement on great tools and fast code, but "no downtime" is not an experience I have had (or have currently) when MS is part of the picture. Oviously, if you (and MS programmers in general) feel you need a PhD to work with Java/J2EE, that is indeed a good reason to not have MS programmers switch over.

    <quote>
    My company implemented a JSP/Servlet solution, and a VB/ASP COM solution in parallel. Guess which one is a support nightmare? Guess which one crawls? Guess which one grinds to a halt for no reason? I am sure you guessed by now...
    </quote>

    Your company's lack of experience and skill in implementing Java/J2EE solutions isn't a reason to come here and spread your FUD.

    Cheers
    Ray
  257. <quote>
    Those who are used to the MS paradigm will not find Java to their liking. There aren't the drag-n-drool GUI tools and when faced with actually writing some enterprise code, it can be a daunting task. However Java can be quite fast programmed properly and the config files required with J2EE are hardly rocket science. You shouldn't use your lack of understanding of Java/J2EE as a basis to spread FUD.
    </quote>

    How long has J2EE been around? And they can't come up with tools to match MS? That's an excuse - not a answer. In my mind there is no reason why the tools should not be on the same level, yet there is no doubt, at least in my mind, that they are not (they are getting there - IDEA for example).

    A bunch of people have also pointed out that performance of Java will automatically be less than .NET on the windows platform. Well, I understand what you mean - but why has not more innovation happened to make it faster on key platforms? Sure, there will be some overhead, but I think the performance should be closer.

    I think people can sit around and complain that the tests aren't fair, specifically:
    - That the tests are not an appropriate usage of J2EE capabilities.
    - That the tests contain differences that may have a impact (judged as major or minor).
    - That the tests are poorly coded in some way.
    - That the whole thing is bias because of political or payment reasons.

    These are fine and valid complaints, however, it's pretty clear the MS has a performance advantage. That's natural - it's a newer technology that has created it to battle J2EE.

    This doesn't sound the death of J2EE for .NET - just means that innovation needs to happen. This is no different than saying that Microsoft needs to work on security.

    There is so much defensiveness around these issues - take the report as some interesting information that can point to areas where J2EE can be improved.

    Damian
  258. Damian,

    <quote>
    How long has J2EE been around? And they can't come up with tools to match MS? That's an excuse - not a answer. In my mind there is no reason why the tools should not be on the same level, yet there is no doubt, at least in my mind, that they are not (they are getting there - IDEA for example).
    </quote>

    I guess my point is that I don't particulary miss MS tools and find that I am very productive with my particular arsenol of tools (of which IDEA is a part). It's not particularly important to me that I have VB.J2EE or some such. Would I like better tools? Sure, who wouldn't? Is it crucial? I personally don't think so.

    And sorry to sound defensive but so many have come into this and have interpreted the results of this test as proof positive that J2EE can't scale and can't perform. I know for a fact it scales and performs very well. Like you, I agree that the report should be taken as some interesting bit of information and that everyone should realize that more innovation needs to take place in J2EE land (should be true for any technological infrastructure). But that's about it, really. It's hard to go further than that with the results.

    Cheers
    Ray
  259. And they can't come up with tools to match MS?


    They can. For example Eclipse/WSAD. You may not like them but they are just as good if not better. Built in refactoring, AOP, Finding of classes, etc. Things VS.Net doesn't have. And it will run on Linux, Windows, Mac and Unix.
  260. Haven't tried .net, but went to Java because I worked with MFC for several years. Talk about a lousy framework... The only thing that saved it was the comparison to the sdk.

    Actually .net is a bit misleading. I bet we can beat that performance by doing the thing in C++ (actually I never found ISAPI programming to be that bad, but that is a different subject ;)), of course that is really what MS is doing here isn't it? Why don't we just get rid of the CLR overhead and write something we can compile to native code? Then we won't even have the COM runtime overhead.

    One thing MS did get right is async io, we didn't get this in Java until 1.4, and I am sure that it wasn't in the app servers used in this test.

    Java still needs work, but it is a coherent platform, and it is true that not every computer runs Windows.
  261. N+1 BMP optimizations Performed[ Go to top ]

    "This time, the Middleware Company has re-coded the J2EE Petstore and optimized the implementation for performance."

    The Middleware Company should be embarrassed if BMP is
    the best optimization they can do. It is well known that
    BMP finders result in n+1 database calls:
    http://c2.com/cgi/wiki?EntityBmpFinders



    "The new implementation has been extensively optimized for performance and scalability, and tested in two leading J2EE application servers."

    It would have been interesting to see: Solaris/Intel,
    Linux,JBoss results.

    Dave Hunter
  262. Would it be possible to make a pure source version of the .Net PetStore available? I don't want to install neither ISS nor .Net on my machine.

    Thanks.

  263. Well, congratulations to The Middleware Company for a very thorough and unbiased report. You are now the leading middleware analyst in the world. It must have taken a lot of courage to publish these findings. I am certain that Apache-Jakarta foundation will take up their axe and come up with something better than EJB.

    "You can fool all the people some of the time and some of the people all the time, but you can't fool all the people all of the time."

    Regards
    Rolf Tollerud
  264. W A K E U P ! !
    EntityBeans with RemoteInterfaces! I don't know what the h**l were the CrapWare guys thinking but this thing surely isn't optimized!
  265. <Java Guy>
    EntityBeans with RemoteInterfaces! I don't know what the h**l were the CrapWare guys thinking but this thing surely isn't optimized!
    <Java Guy>

    Any decent java appserver will automatically detect whether its a localcall/remote call and optimizes it. So i dont think it had any affect on the performence.
  266. Don't believe this entitybeans are crap fud!
    I got so annoyed by this benchmark that I wrote a test to compare server dataobject 'gets' with entity layer and without entity layer on our J2EE app.

    The dataobjects are quite complex, they contain data from 7 database tables. This is a real world scenario, an application that needs to have both GUI and Web interface.

    All the entity beans are BMP, since that complex stuff doesn't scale to CMP or anything, and SQL code is easy to optimize (basic rule: many simple queries are much faster than one complex with lots of joins, if u don't believe, try..)

    All entity beans use DAO (data access objects) so this makes it really easy to test it without entity cache.

    The remote session facade has a method like this:
    List getDataObject(List ids);
    that fetches the dataobjects with given ids. Normally I iterate through ids and call findByPrimary() and then getData() to fill result array.

    To test entityless implementation I added method
    List getDataObjectsFast(List ids);
    in which I directly connect to the database (of course through the transaction handler) and call the DAO's load method for each id.

    The small test database contained 200000 records in several tables (yes, it's small - we normally have 10 000 000 + records) and these records 'translated' to approx 24000 dataobjects.

    So to read those dataobjects the database (tested with mysql 4.0.4 beta with JConnector beta 3) would have to perform 200000 reads and the server would need to construct 24000 entity beans in entity scenario.

    Here are the results (clocked at the server end)
    run entity [sek] entityless [sek]
    1.st 105 82
    2.nd 5,3 81
    3.rd 4,7 82
    4.th 4,7 81
    5.th 4,7 82

    So the 1.st time the entityless implementation is 28% faster
    than the entity implementation, but after the cache is fully utilized the entity implementation is 1700% faster. Over 17 times faster!!

    Without entity cache the server can do approx 2500 database ops / sek. Non optimized MySQL & non optimized JBoss running on my P4 1,6GHz laptop. Not bad, kudos to MySQL and well written JConnector beta! drivers..

    But with entity cache turned on the same hw delivers
    approx 43000 database ops / sek!!! Non optimized (both MySQL and JBoss 3.0.3 in default settings)

    And remember that for each antitybean read method I call first connect to datasource then do the read and then disconnect, whereas entityless implementation needs only one connect/disconnect pair.

    Can u get 43000 database ops / sek from your .NET server on off the shelf laptop?
    But that in your .NET pipe and smoke it!
  267. Forgot to tell that:
    - the EntityBeans all use Local interfaces
    - every method has transaction attribute required
    - MySQL tables are InnoDB type (supports transactions and foreign keys)
    - JBOSS commit option was A (since our app is the only app accessing the db)
    - JBOSS VM was set to operate on 64MB (min/max memory)
  268. So, when do you guys open theserverside.net, your .NET community portal ? :)

    Yann
  269. yeah!
    let the J(i)had begin!!!!
  270. First of all many thanks to the middleware company for performing the benchmark, great work guys. Now the interpretation seems quite difficult ;-) IMO the results show that _based_on_the_official_recommendations_ .NET performs way better then J2EE-Petstore - no doubt. However one has to ask to ask what does this mean ? First of all let me point out that the pet store was developed by people from SUN engineering - having worked for the SUN Java Centers, let me assure you that thoes guys do a great job BUT they are no practitioners and do apparently not take into consideration a lot of common knowledge. We always felt bad about the fact that engineering takes over this role instead of having real world architects (that have been burnt) give recommendations. Unfortunately JPS is considered the "reference architecture" - that has an implication: People will try to understand it and reproduce its architecture.
    The comparison is .NET vs J2EE is therefore fair to some extent...

    I do read quite some articles on TSS and see some mindbogling architects out there. There are cool frameworks (struts,jato,webworks ...) tons of persitence tools (OJB, Hibernate ...) and people making great applications with them. For large scale projects however one has to base on standards and unfortunately pet store is considered to be it for J2EE. Therefore companies expect their people to understand and adopt it. We cannot expect to have the freedom to choose something "non standard" or to have the luck of working together with excellent people. John Doe is supposed to know "pet store like" architectures - that's SUN's fault since they seem not to be able to push a reasonable architecture. Again the comparison is fair from this point of view.

    I would prefer J2EE over .NET right now in case I'd be free to choose the architecture and enjoy the privilege to work with skilled programmers. But ask yourself: How many times have you been lucky to work with people understanding the architectures outside of the standard ? M$ gives their programmers an architecute that will be adequate and therefore succesful most of the time and that's quite an achievement. Nevertheless, they will fail IMHO in complex large scale projects where you need skilled professionals anyway - given the current superiority (IMHO) of the Java toolbox as a whole .NET will not succeed there. At least not right away - M$ is on the right track conquering the "mass market" right now. They will be smart enough to extend their technology stack over time in order to cover more complex areas too. Let's be honest: That's they way one should do it. I'm really sorry for my former employer
    :-((


  271. On page 16 you say that you acheived a 50% throughput increase by using Jdk1.4 but still chose to compare using Jdk1.3. Why not use an app server that allowed you to get the best results since you framed this as a performance comparision???

    >>In tuning the two application servers, the first points that were considered were the choice

    of Java Runtime Environment (JRE) and JDBC driver to be used. In this, four JREs were

    tested: SunSoft 1.3 and 1.4, JRockit 3.1 and IBM 1.3.

    On application server A, SunSoft 1.4 was found to be by far the fastest for this

    application, offering up to 50% better throughput in the benchmark than SunSoft 1.3.

    However, because of compatibility reasons SunSoft 1.4, while it could be used for the

    Web application and distributed transaction benchmarks, could not be used on J2EE

    Application Server A for the Web service testing so the JRockit JRE was used in this

    case. One important factor in the performance of these JREs is the availability of

    concurrent garbage collection. If this feature is not available, the pauses for garbage

    collection become excessive and can lead to &#147;connection refused&#148; errors under high load

    as application server processing stops during garbage collections. J2EE Application

    Server A benefited from the fact that SunSoft JVM 1.4, which provided much optimized

    garbage collection over 1.3, worked with the product (except in the Web Service test).
    <



    I'm especially puzzled since you had a Micrsoft Solutions Provider write the Petstore for .Net and then had a Microsoft employee spend two weeks optimizing it.

    From page 14:


    >>Time spent on .NET tuning and configuration optimizations prior to

    testing

    After developing the .NET Pet Shop 2.0 implementation (the application was actually

    developed by a California-based Microsoft Solution Provider), a single Microsoft

    employee spent roughly 2 weeks on trial runs and final optimizations including

    configuration settings prior to the Middleware testing (total 2 man-weeks).
  272. One thing I noticed in the report is that it appears that the 1.4 JRE was used on Server A for the normal testing, but 1.3 was used for the web services testing. Check out Page 51. I don't think anyone has actually mentioned this yet -- all the messages I found about 1.4 usage so far in this thread are of the "they really should have used 1.4" variety.
  273. One thing I noticed in the report is that it appears that

    >the 1.4 JRE was used on Server A for the normal testing,
    >but 1.3 was used for the web services testing. Check out
    >Page 51. I don't think anyone has actually mentioned this
    >yet -- all the messages I found about 1.4 usage so far in
    >this thread are of the "they really should have used 1.4"
    >variety.

    I don't think so. Weblogic 6 and 7 do not support JDK 1.4 at all.
  274. Well, then, the report itself is wrong, because it says on page 51 that that is what they used. They also mention that they used 1.3 for the web services portion since it didn't work with 1.4, and they used 1.3 with Server B because it didn't work with 1.4 either. They also say on page 16 that "J2EE Application Server A benefitted from the fact that SunSoft JVM 1.4, which provided much optimized garbage collection over 1.3, worked with the product (except in the Web Services test)."

    So, even if 1.4 isn't officially supported on "Server A", it does appear that they used it and found that its performance was higher than 1.3.
  275. Well, then, the report itself is wrong, because it says on

    >page 51 that that is what they used. They also mention that
    >they used 1.3 for the web services portion since it didn't
    >work with 1.4, and they used 1.3 with Server B because it
    >didn't work with 1.4 either. They also say on page 16 that
    >"J2EE Application Server A benefitted from the fact that
    >SunSoft JVM 1.4, which provided much optimized garbage
    >collection over 1.3, worked with the product (except in the
    >Web Services test)."

    >So, even if 1.4 isn't officially supported on "Server A", it
    >does appear that they used it and found that its performance
    >was higher than 1.3.

    I am not happy with this but I assure that you and the report are right. I have read page 16 and 51 again and I think they *did* use JDK 1.4 for Server A. And Server A is WebLogic 7 (the readme in the petstore source tells how to make it work in WL 7).

    I can't make WL7 work with JDK1.4 myself some months ago. Maybe they are genius or I'm stupid. Or maybe WL7 SP1 / WL on w2k supports JDK 1.4.
  276. See code for User Interface:

    .NET 1002 lines
    J2EE 5567 lines

    But why do not use existing taglibs ? No, it is not about
    JSTL. For example Coldtags (www.servletsuite.com/jsp.htm) provides a lot of ASP.NET similar tags.

    From our own experience some of them are even more convenient than .NET analogues. At least this disadvantage could be eliminated.
  277. One question is that if, in the .NET implementation, you built up dynamic SQL and executed that rather than using Stored Procedures, why wasn't the J2EE dbaccess done in a similar way.

    I appreciate that TSS has done a lot of work, including evalutation testing etc, but surely the application should be written to be equivalent to each other, with non-app server programs written and executed on each platform(.NET and J2EE) to provide language and runtime execution comparisons.

    Also just because in took two man weeks to evaluate/configure and tune .NET does not make it _better_ when the sheer options available for J2EE (plus two app-servers used for deployment after the configure stage) is obviously going to make systems selections more complicated and drawn-out . I also agree that the same database should have been used.

    What TSS should now do is re-write the J2EE version to strip it down to the kind of layers and components that .NET is using - until the systems are roughly equivalent in terms of the layering and like-for-like technologies (JDBC<-> ADO.net) this argument will nopt come to a definitive end. Also use jdk1.4

    Calum
  278. FUD[ Go to top ]

    Thanks to tss for spreading fud.
    First you make an broken comparision (BMP, etc), then the Microsoft Licensing terms make it illegal for everybody else to publish a better benchmark. Well Done! I really wonder what was the motivation behind this. Playing nice with microsoft to get consulting jobs from them?

    peace
     chris
  279. FUD[ Go to top ]

    First you make an broken comparision (BMP, etc),

    > then the Microsoft Licensing terms make it illegal
    > for everybody else to publish a better benchmark.
    > Well Done! I really wonder what was the motivation
    > behind this. Playing nice with microsoft to
    > get consulting jobs from them?

    Yeah, that's also my impression. Look at the hardware farm they've used. Look at the menpower and time they have spend. And look at the link from MS to their benchmark. Do you think that a Ma & Pa company like Middleware can handle it without some "sponsering"?

    Look closely. It's all about money, guys.

    Ey, Middleware, shame on you. You've just sold your mother. Nauseous.
  280. FUD[ Go to top ]

    First you make an broken comparision (BMP, etc),

    >> then the Microsoft Licensing terms make it illegal
    >> for everybody else to publish a better benchmark.
    >> Well Done! I really wonder what was the motivation
    >> behind this. Playing nice with microsoft to
    >> get consulting jobs from them?

    > Yeah, that's also my impression. Look at the hardware
    > farm they've used. Look at the menpower and time they
    > have spend. And look at the link from MS to their
    > benchmark. Do you think that a Ma & Pa company like
    > Middleware can handle it without some "sponsering"?

    I could not agree more. Also to add to the unfairness of this comparison:

    1. Microsoft designed and coded their .NET PetStore application and a Microsoft employee performed the configuration / performance tuning. To my mind, whether or not The Middleware Company were really trying to beat .NET's performance figures (which given the use of entity beans seems unlikely), it would have been much fairer to have given the two J2EE vendors a chance to provide their own PetStore apps and configure / performance-tune them.

    2. The Middleware Company provide performance figures on .NET 1.1 RC1 and Windows .NET Server RC1. These are not fully fledged production products. There were no signs of any beta products from the J2EE app server vendors, nor did they run the J2EE app servers on Windows .NET Server RC1. This is obviously heavily biased in MS's favour.

    One has to wonder why The Middleware Company (who provide Java/J2EE training and consultancy) would publish a report that ON THE SURFACE is so damning to J2EE... Of all the differentiating factors with which they could have compared .NET and J2EE, they chose performance. Let's face it, J2EE (even if it was properly optimised for performance) was NEVER going to beat .NET on performance while running on an Wintel platform because .NET is designed to run natively on that platform and Java/J2EE is not! The Middleware Company new before they started what the result was going to be. Like several other posters, I wonder whether it is perhaps because The Middleware Company are about to diversify into .NET training/consultancy with some MS money behind them...

    I mean, how much do you think it would be worth to MS to have a high profile, supposedly pro Java/J2EE, web-site post a report that shows in .NET in a most positive light? Given MS's absolutely huge .NET marketing budget, is it outside the realms of possibility that MS (apart from developing the .NET PetStore application, sending an employee to configure and performance-tune it, etc) have provided other incentives?

    And why are some posters acting like this report is a major victory for .NET? Personally I was surprised (given the reality that the J2EE PetStore was not optimised for performance,and that the tests were all run on the Wintel platform) as to how small the performance differential was between J2EE App Server A and .NET 1.0 (less than a factor of 2 on pretty much every test and sometimes a lot less).

    Most importantly though, J2EE's advantages over .NET greatly outweigh any alleged performance disadvantage. J2EE provides more robust and maintainable code, and no vendor lock-in (portability to other App Server / OS / Hardware platform). It is an easy matter to increase J2EE performance simply by adding better/more hardware, and hardware costs for most projects are negligible compared to development, management, and maintenance costs. Therefore it is these areas that any relevent comparison should be focusing on...
  281. FUD[ Go to top ]

    I don't think marketing had anything to do with this test / comparison. And you mention the difference between AppServer A and .Net 1.0... did you fail to notice that AppServer B couldn't even complete the test. And if you are going to look at all four with an honest eye than you need to realize that .Net Server crushed all of the participants.

    Your rose-colored glasses would have been a lovely sight in the Soviet Union. How many bushels of wheat did we produce? Oh I'd say a trillion trillion bushels, sir. When in fact all of the wheat was burned down while trying to crush some peasants in revolt.

    Wake up and realize that J2EE / Java has some issues to confront, instead of running away and blaming the TMC.
  282. FUD[ Go to top ]

    <quote>
    Wake up and realize that J2EE / Java has some issues to confront, instead of running away and blaming the TMC.
    </quote>

    OK! I left my home door unlocked and when I came back everything was robbed. Now you're blaming me because I left my door unlocked and you say it's all my fault but is't that freaking robber any guilty at all?
  283. FUD[ Go to top ]

    Yagiz,

    What robber do you mean?

    MS published a report made by independent part that showed that .NET was 28 times faster which in retrospect was more or less the correct. For this they were strongly attacked and seriously ridiculed.

    Oracle then published their own not-by-any-third-part results where they claim that they had made a few changes and were roughly 20 times faster than the .NET solution. They didn’t release any code or test scripts.

    And as Mark says, that would mean that they had made a few changes to accomplish a gain of 20x28=560 times increase. That Is What I Call A Nice Performance Jump. For that they were not attacked or ridiculed in any way.

    In fact think the comparison of J2EE/EJB with the old Soviet Union 5-year plans is very apt.

    Regards
    Rolf Tollerud
  284. FUD[ Go to top ]

    This is soooo funny I see more M$ trolls supporting this report than the TMC & Middleware guys. Does it smell fishy here ......;)
  285. FUD[ Go to top ]

    Rolf,

    I have two questions:

    1. Do you work as a .NET consultant ?

    2. What the hell do you know about the Soviet Union ?

    Regards,
    Alex
  286. FUD[ Go to top ]

    Rolf,

    MS was ridiculed because they did the ridiculous - they took an application that was never meant to be benchmarked and they benchmarked it. It was a simplistic marketing ploy that worked on some people. Oracle perpetuated it. TMC is perpetuating it. It showed little then, and little now - except to people such as yourself. And it has started this whole silly thread - every few months, somebody takes the petstore and benchmarks it. As someone who works with J2EE and not .net (and I can't since I don't work in an all-MS-all-the-time environment), I know that the J2EE infrastructure performs quite well. I know the petstore doesn't. That fact is about as irrelevant to me as you can possibly get because it isn't terribly important in the grand scheme of things, except to divert myself from work to participate in this discussion - which I do enjoy. A double standard I know.

    I am happy to see that TMC is taking suggestions on how to re-run the test. While kicking the pants out of .NET on a revised Petstore application doesn't really show anything - it will be amusing to see the MS trolls squirm for excuses.

    Cheers
    Ray
  287. <b>a priori:</b>
    <ol>
    <li>MS refuses to allow independent benchmarks of .NET.
    <P>
    <li>MS agrees to pay TMC to do this benchmark in MS labs.
    <P>
    </ol>
    <P>
    <b>a posteriori:</b>
    <P>
    Q. Clearly MS put restrictions/requirements on the test. What conditions did MS put on TMC?
    <P>
    After reviewing the code and the test conditions, it's clear that either a) this test was purposely rigged using old Java and slow choices, designed to slam Java (likely for financial gain) or b) the people at TMC who built that app are just not very good (those who can't do, teach?).
    <P>
    It has to be one or the other. The Pet Store app is not well written. So it's either a lack of skill or deliberate.
    <P>
    ********************************************************
    <P>
    And remember, gentlemen -- if the .NET developers are un-knowledgable enough to believe this, it's just another win for the Java world.
    <P>
    As long as they are ignorant of the truth, they can't compete.
    <P>
    Can't you just see them at some jobsite, when they see a well-written Java solution scream?
    <P>
    "But, but, but -- I read on the Microsoft site that Java is slow?!? This can't be?"
  288. Oops, sorry about the tags, I thought this board could handle HTML.

    My bad.
  289. "But, but, but -- I read on the Microsoft site that Java is slow?!? This can't be?"


    Actually, I read it at TheServerSide.

    Sorry.. couldn't help myself.
  290. Actually, I read it at TheServerSide. <

    :-D

    I'm assuming 90% of the MS-only folks are finding this via the link on MS's developer page, hence the high number of MS-only people commenting on this thread.

    Altho I'd say that this report does not say that Java is slow, and that's the whole point!

    It's funny, in every other case, if someone posts a benchmark for an app they wrote and it turns out slow, typically that is considered an indictment of the programmers, not the technology.

    A Java petstore isn't slow, the Java Petstore written by TMC turned out slow.

    And since TMC did this with Microsoft funding, and according to Microsoft's requirements, and run in Microsoft's lab . . .

    So the question is, why was TMC's effort so slow? What can TMC's people learn from this? What can the folks here teach the teachers?
  291. Drop this bashing and grow up[ Go to top ]

    Alexandar

    Yes, even if I would not call me a .NET consultant I do believe in the future of .NET.
    Yes, being Scandinavian I actually know the old Soviet Union quite well. They were our neighbors.

    Ray,

    I enjoy a good fight and do not expect to win every time. I do not want everybody to instantly switch to .NET. I only ask to take away the arrogance, the ridicule, the snottyness, the accusations of criminal or immoral activities, and so on and let the best system win.

    It irritates me this constant bashing of VB programmers for example. VB never was indented for web, it was a system to build desktop applications. Such applications are not needed in any great quantity anymore, but if you should need a -Window desktop application- there are still no better way to do it, in productivity, maintenance and performance it still outperforms any other system today. And I made more money with my VB application than most of you guys in this forum can dream of.

    I only want the Java/Unix/Linux users to accept that .NET server (clustered or not), with .NET Framework is a formidable adversary on par with anything else to day, including mainframes. Drop this bashing and grow up. You need some improvment on your own you know.

    Regards
    Rolf Tollerud
  292. Drop this bashing and grow up[ Go to top ]

    Alexandar has your number.

    >>"if you should need a -Window desktop application- there are still no better way to do it..."

    Rolf, you lost more credibility than TMC with this statement!
    I fully understand now, why J2EE would overwhelm you - it is too hard for non-technical programmers.

    I've been using Delphi for a long time; I guess that's why I picked up J2EE pretty easy...

    I hope you are not from Norway.
  293. Drop this bashing and grow up[ Go to top ]

    Rolf,

    < Yes, even if I would not call me a .NET consultant I do believe in the future of .NET.
    Yes, being Scandinavian I actually know the old Soviet Union quite well. They were our neighbors. />

    Have you created any .NET applications that are in production now ? From your postings I got the idea that MS products/people are way better than the rest of the world .So I just wonder why you are working with java then ?

    You know for the USSR from the newspapers or you actually lived there ? Just curios.

    Regards,
    Alex
  294. Drop this bashing and grow up[ Go to top ]

    Alexandar,

    The poor people who were actually living in the old Soviet Union knew no more about the reality than the citizens of North Korea know today. They actually believed in the 5 year plans.

    But someday they woke up to lost illusions, just like the J2EE guys one day will do too..

    Regards
    Rolf Tollerud
  295. Drop this bashing and grow up[ Go to top ]

    Rolf,

    You didn't answer my questions .

    Regards,
    Alex
  296. Drop this bashing and grow up[ Go to top ]

    <qoute>
    But someday they woke up to lost illusions, just like the J2EE guys one day will do too..
    </quote>

    Says who??? Rasputin?

    Seriously, where do you get your 5 year forecasts? Do you read them from your coffee cup as they do in Middle the East or from a highly placed spirit talking during ghost hunting sessions?

    Ti di, ti di! Ti di, ti di! (Please play here Twilight Zone Soundtrack)
  297. Java.NET[ Go to top ]

    Java.NET! I will definitly be interested to know more about .. or this looks like the best solution for J2ee developers
    Faisal
  298. Talking to spirits[ Go to top ]

    Yagiz,

    <Q>Seriously, where do you get your 5 year forecasts? Do you read them from your coffee cup as they do in Middle the East or from a highly placed spirit talking during ghost hunting sessions?</Q>

    I tried talk to the highly placed spirits but they didn’t want to talk with me!
    My educated guess is more based on things like personal experience and benchmark tests but also on reading stuff like this:

    "Organize inter-service transfers according to use case from known domain objects into a coarse-grained Composite." :-)

    Regards
    Rolf Tollerud
  299. Have the J2 vendors step up already...![ Go to top ]

    J2EE has been out for the longest time - so you would think J2 vendors should be 'enterprise-ready' to take on an enterprise wannabe like MS. Shouldn't they be ready, given SUun and the rest of the J2EE vendors are so vocal usually, and should have come out sooner and more publicly to refute this and provide $ and resources to get it done right. One sure would expect them to. It really doesn't cost that much money for these billion dollar companies, and for MS to do this and releasing all the source code (Oracle released a perf doc but never released source that stood 3rd party scrutiny), only for people to vent excuses, bashing MiddlewareCo, which before was a respected J2 consultancy until this piece of bad news pierced everyone's dreams....(sob sob)

    People may go back thousand times to optimize J2 to beat .NET but end of day this requires time and money, and if you apply enough of both you'll eventually get what you want somehow someway. The real world doesn't give you a fountain of time and money for techies to go back forever for redo's. Middleware was given a lot more time to do whatever they want than MS to put up a *functionally equivalent* app. IBM and Co. could have put their poker chips on the table and put $$. Complain all you want but end of day I would like IBM, Oracle and the rest to come out NOW (actually a year ago) already with their skins/wallets/Java geniuses to support this so J2 can undeniably put their best foot forward, stop copping-out by playing silent and dumb and let a bunch of zealots cop out some more for them (plus they usually talk so much, boast that Java is the solution for everything short of world peace, and bash MS like nobody else you'd think they'd have more pride and be squashing .NET by now).

    So anyways, I say again, you would suppose since MS is not enterprise ready, it shouldn't be all that hard and take all that long. Three's a charm, right?
  300. Silly questions[ Go to top ]

    Portability that big catch-word. Someone stand up and testify (amen) to how much of an impact to their company portability has had to the bottom line. Who has actually ported, what % of folks? And what's wrong with a vendor developing on one platform and putting all their $ into something to make it the best and, dare I say it, affordable vs. have something that works unevenly on most platforms and you pay more for less? Is it easy or practical to port from IBM to BEA; aren't they their own proprietary implementation of J2 anyways? And if so, what advantage does that give you? In the real world, isn't portability really a straw man for people to justify bigger, more complex and spiffier, not better? Course, let's not get into those fatter paychecks...

    Choice of vendors. So choice of vendors means cheaper software right? Cheaper high-volume hardware for sure? Performance (w/o having to hire in those dozen consultants that stay on until the next layoff round and get rehired again in the upcoming CY budget cycle)? Lower labor costs? So how much does Websphere cost again...and it can't be that half of IBM's revenues comes from consulting, can it?

    Open vs. proprietary. IS sun open really? Do they represent the last bastion standing up for open standards and these days open source? I'd say from SPARC to J2EE, Sun controls something, ok let's call them standards to avoid a fist-fight, and they nominally control J2EE which IBM and the rest fracture to their own benefit.

    But let's say we all do go to this wonderful new world and embrace open source everything and it's a free for all 5 years hence - enterprise software/dev goes open source and evil MS croaks and IBM migrates all s/w to open source and the industry competes on who provides free or near-free software w/ the most funcionality. Will any of us on this board have a job then or do we work for free and barter for a living: I write 50,000 lines of Java code for a good Proliant server or an Infiniti? Or do we switch jobs to chip design and low level microcoding where people can still charge for hardware, until of course a new subscription-based revenue model takes the world over by storm and we all get communally-built hardware for free subsidized a bit by our subscription $'s?

    Isn't Java just a cool language blown out of proportion? Is it really the 'end all be all' development platform that nobody can come close to touching? Is J2EE that much better than .NET? Is it all anyone will ever need? Isn't it true that sometimes .NET is better and other times J2? One can only wonder...

    Though I may not know the answers to these silly questions, I am darn sure that Java is better warm than cold.
  301. Silly questions[ Go to top ]

    The Who:

    >>Isn't Java just a cool language blown out of proportion? Is it really the 'end all be all' development platform that nobody can come close to touching? <


    It was cool enough for Microsoft to drop their "Windows Only" propaganda and duplicate Java and create .Net.

    So in spite of your spin job.... Microsoft had to change their game to play to the new tune. If you can admit that then you are delusional.

    Nobody really trusts Microsoft anymore....especially after this latest ridiculous stunt. They just keep proving to us that the choice of Java for freedom was the right choice.

    I can build on any platform...Windows included if I use Java. If I use .Net I am limited and much worse I am LOCKED IN.

    No thanks. Java=Freedom. That very freedom busted this latest ploy to lie to the community because we have been trained to look under every bush where MS is concerned.

    Face it; Microsoft's biggest enemy is themselves. This sort of stuff creates the distrust.
  302. Silly questions[ Go to top ]

    "Microsoft had to change their game to play to the new tune. If you can admit that then you are delusional."

    My sentiments exactly!

    "That very freedom busted this latest ploy to lie to the community because we have been trained to look under every bush where MS is concerned."

    Talk about delusional!
  303. Any thing going on for Retest.[ Go to top ]

    Is any thing been done for retesting that was proposed by TMC. Also in the Options put forward by TMC will the Java app haa to use all the features of J2EE.
    I think for the application as small as Petstore, we do not need EJB. I will use EJB only if there is enough Business complexity demanding it.
    If we are allowed to put a Java app with simple design as MS Petshop then we can get under 3000 lines of code for similar functionality. I will use Struts/EntityEngine(ofbiz.org) and we can get number of Hand written code application code well under 3000 lines. Also the Performance will be lot better.
  304. Silly questions[ Go to top ]

    First it was IBM in the 80's, back when Apple was bigger than MS. IBM was the evil empire then. You were locked in to big iron, until the 'PC' revolutionized and freed the masses from that. MS then was an upstart; now that MS is big and successful too, you and your camp of folks who like to shoot down #1's (it's a fatalistic American trait I don't get, e.g. I don't like Tiger Woods because he wins too much) will distrust and bogeyman MS forever no matter.

    .NET copied Java. Uh huh, and Apple copied Xerox Parc, and Windows copied Apple, and JTS/EJB copied COM/COM+ (the latter was out two years before). Sun was smart initially and let J2EE ride the Java language tidal wave. It 'innovatively' wraps a single set of unified APIs around a single language that is better factored while OO than what's avail in C++ and VB runtimes while easier to code against (course those Java creators left Javasoft afterwards cuz Sun was just a great innovative company to work for). Then they start cramming down people's throats that J2EE is multiplatform and Java the only language you'd ever need that you can WORA, which is when they started losing me. So I guess I'm satanic if I write on a platform with couple languages. But w/ J2EE I can't use another language (e.g. Perl) if I want to do text manipulation, just those renowned speed demon Java string classes. Yet, I can port from one plat to another. How often to I port? How often do I use one or more languages. What do I do, I'm kinda lost with those choices. I'd say porting happens less.

    They did provide a unified API around a single (albeit nice and elegant) language. Is that the last innovation permitted in IT? You mean .NET can't come out with inline attribute support, built-in XML serialization, and a common type system for multilanguages? I'm pretty sure you think MS is obviously guilty by being the first and only to build on past improvements of others. But then Java was a hybrid of C++ and VB and Smalltalk. C++ was a leech of C and K&R. Where do we end? Do we bash because we as a society generally improve incrementally and MS is doing that w/ C# and a CLR optimized for one platform? Was Java a totally revolutionary and stand-alone deal noone can touch or improve upon? Does Sun get bashed for trying to sell Forte as a multilanguage IDE (when it's two separate IDEs under one shell)? Hmm...

    You say trust and freedom and innovation = SUN.Java and MS is just a cheating, lying, money-sucking Borg. I say the evidence speaks for itself - at least them borgs give you affordable prices, easy to use tools and products packed with more and run faster each vnext, and general purpose enough so that more people can benefit than less. To me, the 'freedom fighters' you support are high maintenance - expensive, specialized, and you pay piecemeal for every feature that only a few can master yet complain when they don't make enough $ charging more and more for their products that people find less and less value paying for.

    But then, again I forgot, at least you're free to port from one plat to another when you need to. If that's freedom, it must be an abstract higher level of nirvana few can comprehend or ever hope to achieve.
  305. Silly questions[ Go to top ]

    *****************
    and JTS/EJB copied COM/COM+ (the latter was out two years before).
    *****************
    I think you are wrong here. JTS/EJB is extention of CORBA technlogies and also "so is COM/COM+ which is called the HELL by its creator"
  306. Silly questions[ Go to top ]

    The Who:

    <quote>
    How often to I port? How often do I use one or more languages.
    </quote>

    How often do I build enterprise applications on non-MS-based servers in my fortune 500 clients? All the time. How often can I run .NET on those non-MS-based servers? Never.

    J2EE is very fast, scalable, secure, multi-platform (i.e. real world), has a variety of pricing structures, is easy to work with, has great and continuously improving tools, competition,integrates well with all sorts of legacy systems, the list goes on....

    Please, if you like .NET, continue programming in that environment. No one thinks you're satanic. No one is spending a great deal of effort trying to convert you or any other MS person into a J2EE developer. There seems to be an effort on the part of the MS community to convert J2EE programmers to .NET, of course. But the same tired story of 'it's too hard, it's too slow, it's etc' gets old and wears on one, especially when it isn't true. And especially at the end of a long day ;-)

    Cheers
    Ray
  307. Silly questions[ Go to top ]

    Hmmm... after all that jabbering .Net is not portable . Sorry no marketing bluff will change my mind.
  308. Silly questions[ Go to top ]

    "Portability that big catch-word. Someone stand up and testify (amen) to how much of an impact to their company portability has had to the bottom line. Who has actually ported, what % of folks?"

    Lots of Microsoft customers have, most to Java. (The Java customers don't have to port.)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  309. Silly questions[ Go to top ]

    <pissed myself>

    Good one Cameron
  310. Drop this bashing and grow up[ Go to top ]

    Rolf -

    <quote>
    And I made more money with my VB application than most of you guys in this forum can dream of.
    </quote>

    I don't know - I can dream of quite a lot of money!

    <quote>
     Drop this bashing and grow up. You need some improvment on your own you know.
    </quote>

    And thanks for the helpful psychological assessment!


    Cheers
    Ray
  311. Drop this bashing and grow up[ Go to top ]

    Ray,

    You’re welcome. I try as best I can!

    Regards
    Rolf Tollerud
  312. FUD[ Go to top ]

    "MS was ridiculed because they did the ridiculous - they took an application that was never meant to be benchmarked and they benchmarked it."

    Why post a sample that was never supposed to be performant? Weak, WEAK argument......

    "it will be amusing to see the MS trolls squirm for excuses."

    How ironic......who's squirming for excuses right now?

    d.
  313. FUD[ Go to top ]

    David,

    <quote>
    Why post a sample that was never supposed to be performant? Weak, WEAK argument......
    </quote>

    I don't know - I didn't post it. Doesn't change the fact that if I had taken the "performance tests" from MS, Oracle or TMC into any sort of technology decision meeting and presented the results I would have been laughed out of the room.

    <quote>
    How ironic......who's squirming for excuses right now?
    </quote>

    Those who are easily swayed by petstore performance tests, perhaps? I don't think any excuse needs to be given.

    Cheers
    Ray
  314. FUD[ Go to top ]

    "Why post a sample that was never supposed to be performant? Weak, WEAK argument......"

    If you can't see the point, then I don't think you're in a position to call any argument weak.

    Why spend time testing and tweaking (for performance) an application that is only meant to serve as a blueprint for others to follow? Granted, ideally the application would be architected for performance, even for example code, but it's not required.
  315. FUD[ Go to top ]

    <"Why post a sample that was never supposed to be performant? Weak, WEAK argument......"

    If you can't see the point, then I don't think you're in a position to call any argument weak.

    Why spend time testing and tweaking (for performance) an application that is only meant to serve as a blueprint for others to follow? Granted, ideally the application would be architected for performance, even for example code, but it's not required. >

    You're right...I don't see the point. Why create a blueprint application, one you promote as the way customers should build applications, if you don't think it will be a good performing application? I've just never understood this side of the argument against the PetShop "war".

    I'm not trying to slam you on this....i'm slamming whoever created the PetStore in the first place.....if I took the time to create a blueprint for my customers, I'd want it to be a solid, well performing application blueprint....otherwise, what's the point?

    d.
  316. FUD - Petshop as design pattern[ Go to top ]

    When I want an example of how to implement a technology, I expect that the example uses appropriate design patterns and good component and class architecture derived from those patterns.

    If the intent was to demonstrate, then *add on* functionality and put in the JavaDoc what you want to illustrate. What kind of idiot throws stuff into a design because it is cool...oh, yea, consultants, that's right ;-)

    Did MS write Use Cases for their Petshop that we could leverage so that we could do a proper PetShop design? (I'll bet Sun didn't write any)

    It would be informative to run it on Linux/Intel, Solaris/Intel and Solaris/Solaris and with 1.3.x and 1.4.x.

    That is one hellova test matrix...Maybe MS will help to fund it...I'd find rich irony if MS paid for it and it did turn out to be faster.

    Oh, yea, don't forget a couple runs with OptimizeIt or JProbe...


    Maybe someone who is a QA expert could some withthe test design.


    this is a community, right?
  317. FUD[ Go to top ]

    good performing application? I've just never

    > understood this side of the argument

    Because you're probably missing the right skills mate!

    The fastest doesn't mean the best. "Speed of execution" is only one of the decision criteria (and it's been the only argument of MSFT against Java) in the enterprise.

    You mentioned a "good performing application". According to what? This is decided in the system requirements. For example, if you develop for an insurance company, the requirement may be to process 2 claims per second. If you can develop an application that can process 2.5 claims per second, it is a "good performing application".

    I hope the picture is getting clearer now...
  318. FUD[ Go to top ]

    <Because you're probably missing the right skills mate! >

    Ouch....I was playing pretty nice.......ok, here goes...

    <The fastest doesn't mean the best. "Speed of execution" is only one of the decision criteria (and it's been the only argument of MSFT against Java) in the enterprise. >

    Then I guess you will continue to be happy with J2EE <smile>. And you've got numbers to make yourself feel good.....for me, i'll be picking another "blueprint" because I prefer not to take chances on blueprints that don't scale......

    <I hope the picture is getting clearer now... >

    Much......G'day......(mate...)

    d.
  319. FUD[ Go to top ]

    <Because you're probably missing the right skills mate! >

    Ouch....I was playing pretty nice.......ok, here goes...

    <The fastest doesn't mean the best. "Speed of execution" is only one of the decision criteria (and it's been the only argument of MSFT against Java) in the enterprise. >

    Then I guess you will continue to be happy with J2EE <smile>. And you've got numbers to make yourself feel good.....for me, i'll be picking another "blueprint" because I prefer not to take chances on blueprints that don't scale......

    <I hope the picture is getting clearer now... >

    Much......G'day......(mate...)

    d.
  320. FUD[ Go to top ]

    David,

    <quote>
    Then I guess you will continue to be happy with J2EE <smile>. And you've got numbers to make yourself feel good.....for me, i'll be picking another "blueprint" because I prefer not to take chances on blueprints that don't scale......
    </quote>

    J2EE is highly scalable. I know - I've built it and seen it. Others have built and seen it. It works in all sorts of real world scenarios and in real world hardware and OS combinations. None that I have been a part of have been built off of the Petstore application (see Nick's posting as to what the Petstore represents).

    I do not know .NET, I do not work in environments where it is possible to use it. So I don't know how it scales in real world setups. MS's Petstore might be good enough for you and might represent how you should build apps in .NET. More power to you.

    However, I do know that those who use and know J2EE would not use the J2EE Petstore as a total blueprint. One can pull a piece off here or there and use it if desired, but since the range of enterprise applications addressed by the J2EE infrastructure goes far beyond web stores, it is difficult to provide any real blue print. Is that an excuse for the Petstore? No - as I have said, the Petstore doesn't need any excuses, it is what it is.

    However, we rely on our knowledge of design patterns, OOD, query techniques and the Java language itself, among other things, coupled with very productive tools from a variety of vendors, to build highly scalable apps.

    You can buy into the MS/TMC/Oracle battles of the petstores if you want. But please don't confuse it with the real world. You would be doing yourself and others a disservice.

    Cheers
    Ray
  321. lol[ Go to top ]

    Wow i cant recall having had so much fun since last years elections.

    gotta love marketting babble

    <quote>
    we have been contacted by several research firms in the U.S. and Europe who want to work together with us conduct more such experiments under different conditions (some of which have been suggested on TheServerSide.com) and also several appserver vendors who want to work with us on future performance comparisons. We have been pleasantly surprised at how constructive these organizations are being.
    </quote>

    To me this reads something like the *cough* experts at The middleware company seriously flunked their exam and people from all over the industry are lining up to give them somewhat of a clue because they seriously hurt the j2ee brand with their benchmark

    I'll take rickards "tell it like it is" way of adressing stuff over this any time.

    btw am I they only one who thinks it's hilarous that IBM's ehm I mean B's "industrial strength" appserver cant even run a simple petstore properly?
  322. lol[ Go to top ]

    jelmer: "btw am I they only one who thinks it's hilarous that IBM's ehm I mean B's "industrial strength" appserver cant even run a simple petstore properly?"

    While "B" couldn't run the application, it wasn't a "simple petstore." The "simple petstore" application wasn't built to do distributed (XA) transactions. That was a "feature" that was added to show how much better Microsoft SQL Server (not using XA distributed transactions) would run to give a huge edge to .NET.

    However, it's not Microsoft's fault that server "B" didn't work. That just sucks.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  323. lol[ Go to top ]

    <btw am I they only one who thinks it's hilarous that IBM's ehm I mean B's "industrial strength" appserver cant even run a simple petstore properly? >

    Nope, it took me 4 days to deploy a realworld app into it... Just for the record, took me 2 hours in server A, and a little more that 1 hour in servers J and O (use your imagination for the last two)
  324. The purpose of blueprints[ Go to top ]

    <>
    I'm not trying to slam you on this....i'm slamming whoever created the PetStore in the first place.....if I took the time to create a blueprint for my customers, I'd want it to be a solid, well performing application blueprint....otherwise, what's the point
    </>

    The point of the Petshop is to:
    a) Exercise/demonstrate as much of the API's as possible and how they might be used (in a small, noddy application)

    b) Suggest design patterns that can be used. (though perhaps overkill for a small, noddy application) (e.g exception handling)

    c) Suggest a project layout and build mechanism that can be employed.

    As a result, there are copious amounts of cut&paste coding (to try and make each sample as self-contained as possible)

    THE PETSHOP IS NOT MEANT TO BE A TEMPLATE FOR YOUR PROJECT... you are allowed to exercise some grey-matter ;-)

    Obviously, if you were going to write a serious application (as Microsoft did with their Petstore) you would make sure that you had more common libraries and refactored out the cutandpastis code. Obviously too, you would profile your application to find where the performance is going. It seems obviously that Microsoft did this and TMC didnt.

    Certainly, starting with the PetStore as a base for a benchmark application is nonsense. What I dont understand is why TMC thought it would ever be a good idea...
    Perhaps, at the point when BEA and IBM declined to be involved (in this shambles) it should have been a subtle clue...???

    Anyway, the damage of this report is done.
    The fact that TMC are labelled as a "leading J2EE consultancy" lends this absurd benchmark much more legitimacy than it deserves.

    -Nick
  325. The purpose of blueprints[ Go to top ]

    You see, that's the problem - a lot of people out there does take it as the template (well, "blueprint" comes pretty close to suggesting that, doesn't it?).
    IMNSHO it would have been much, much better, to provide a separate, not connected examples of the individual patterns/practices (I would still question some of them though) ala GoF patterns than this - and maybe come up with an example _good_ J2EE app, using the only the appropriate best practices, where they apply.
    Regards,
    Vlad
  326. FUD (no, an opportunity!)[ Go to top ]

    Thanks to tss for spreading fud.

    > First you make an broken comparision (BMP, etc), then the
    > Microsoft Licensing terms make it illegal for everybody
    > else to publish a better benchmark. Well Done!

    Are you all blind or what???

    TSS has effectively given you a way to circumvent Microsoft's licencing terms.

    Let me spell it out for you:

    1. TSS compares the .NET implementation to their own version of J2EE PetStore. They also made the source code available.

    2. You implement a better implementation, benchmark it and compare it to TSS's J2EE version.

    3. You publish the results saying something like like: "my implementation ran X% faster than TSS's, used Y% less code, blah blah blah... Oh, and by the way, here's a link that compares TSS's implementation to .NET so that you can infer how my solution compares to .NET"

    That's it! You have a reference implementation of a .NET app that was legally benchmarked against a comparable reference J2EE app, one which you can legally compare with.

    So, without any hint of sarcasm, Well Done TSS! Well Done indeed!
  327. FUD (no, an opportunity!)[ Go to top ]

    Zevel,

    Hit it right on the head!

    Run the TSS PetStore implementation against your own and you can legally compare w/ .NET.

    Excellent observation.
  328. FUD (no, an opportunity!)[ Go to top ]

    Guys, please keep in mind the editor's note at the top of this post. TSS was not involved in this project. All we did is announce the news and provide a thread of discussion, just like Javaworld or onjava.com would do.

    Floyd
  329. FUD (no, an opportunity!)[ Go to top ]

    Floyd,

    You are the writer of the excellent "EJB Best Practices" book. Therefore I would be very interested in knowing your opinion about this benchmark and the quality of the application as far as 'optimisation' is concerned. Did you have time to look at the code yet ?

    Yann
  330. FUD (no, an opportunity!)[ Go to top ]

    Thanks for your intervention Floyd. I'd be very curious to hear more from you on this thread! I have followed the teachings of yourself and your colleagues at MiddleWare - could you do us all the honor of saying more about this benchmark? Is there truly an opportunity here?
  331. FUD (no, an opportunity!)[ Go to top ]

    It seems like serverside shot Java ,J2ee and itself right in the leg for no reasons. If u check their Perstore version u can see it is far form perfect ...This is not fair especially when every developer knows that J2ee is a SUPERIOR than .Net in its data modelling,security etc...
    This test means nothing to me .
    Faisal
    Birmingahm,UK
  332. FUD (no, an opportunity!)[ Go to top ]

    Michael,

    Nearly ever vendors EULA requires that

    1) The software not be used in a public benchmark and that
    2) The software is provided for "personal use". Benchmarks are generally not considered personal use.

    For instance, if you ran a benchmark for WebLogic or IBM wihtout their permission and then published the results listing of BEA or IBM I promise you a cease and desist order from that vendor pretty damned quick.

    Such EULA restrictions are not MSFT specific.

    Dave Wolf
    Personified Technologies LLC
  333. FUD (no, an opportunity!)[ Go to top ]

    Hey,

    I hate to see the J2ee folks on the list become Mac like fanatics where they make statements like "J2ee is better so I don't care about test results ...".

    Let's not be silly here, all these things are tools and the bottom line is that test after test shows that the .NET platform does things quicker and with much less code as compared to J2ee with EJB. I have never used EJB but from what I see, I think it is big stupid mess. It would seem that the FACT .NET is consistently beating J2ee + EJB seems to validate my point of view.

    Beyond that the IDE (VS) kicks ass and ASP.net has many things that makes a JSP guy like myself cry. I use Java and JSP but have also done things with .NET when it was in RC1. I use JSP for fear of vendor lock, but then again many app-server vendors are implementing container specific functionality...:)
  334. FUD (no, an opportunity!)[ Go to top ]

    <Q>Beyond that the IDE (VS) kicks ass and ASP.net has many things that makes a JSP guy like myself cry. I use Java and JSP but have also done things with .NET when it was in RC1. I use JSP for fear of vendor lock, but then again many app-server vendors are implementing container specific functionality...:)
    </Q>

    Check out Eclipse/WSAD for great Java IDE. And as for JSPs - check out the Echo framework (discussion on this site). I've done some ASP.Net and yes it is cool but it made me cry for other reasons (not good). Why do I need to think paging and remembering state and etc.
  335. FUD (no, an opportunity!)[ Go to top ]

    back up your superior argument with some stats and details
  336. FUD (no, an opportunity!)[ Go to top ]

    Bruce, Yann,

     I'd love to, but in this case I think its best that I (and TheServerSide) continue to maintain neutrality.

     What I do know is that TMC did its best to optimize J2EE given their resources.

    Floyd
  337. FUD (no, an opportunity!)[ Go to top ]

    WWJD?
  338. FUD (no, an opportunity!)[ Go to top ]



    Floyd,

    I can not remember last time I was so entertained. Who needs Cinema, Stand up, Seinfeld? This is really an incredible site. Have you considered charging for it?

    Regards
    Rolf Tollerud
  339. FUD (no, an opportunity!)[ Go to top ]

    Guys, please keep in mind the editor's note at the top

    > of this post. TSS was not involved in this project. All
    > we did is announce the news and provide a thread of
    > discussion, just like Javaworld or onjava.com would do.

    Floyd,

    With the greatest respect, to imply that TheServerSide is independent from the The Middleware Company is surely misleading... Do they not (as the advert on the side bar of this web-page states) build, run and sponser this site?

    Other sites, such as Javaworld or onjava.com, would surely have editorial freedom to comment adversely if they perceived there were issues with The Middleware Company's J2EE PetStore application implementation or the fairness of the comparison. TSS obviously does not have / desire this freedom.

    Why were RC (/beta) Microsoft products used in the comparison when TMC were did not attempt (/were unable?) to get beta (or even, seemingly, current) app server releases from J2EE vendors.

    > * Entity beans: Was it fair to use EJB or entity beans
    > for J2EE? Why not use a stateless model as in .NET?
    >
    > The original Sun petstore was built around entity beans.
    > To rip them out would have completely gutted the
    > application. There was not the time nor the money to do
    > that.
    >
    > * CMP vs BMP: Why was BMP chosen? Was CMP tried?
    > We did not try CMP, and so we cannot testify for sure
    > about whether BMP or CMP would have been faster.
    >
    > The original Sun petstore used BMP. There were numerous
    > helper classes being called. It would be more time and
    > money than we had to gut the application and redo it with
    > CMP.

    You seem to be saying that TMC did not have time and money available to rewrite these parts of the application and therefore perform some of the, judging from previous posters, most obvious optimisations. How can you possibly expect us to believe that this was ever a fair fight (and not just an insult to our intelligence) when on the other hand we have one of the largest companies in the world throwing their almost limitless resources and immense .NET marketing budget behind their contender?

    > App Server B did not support EJB 2.0 and thus the new CMP
    > model. It was not possible for us to do an apples-to-
    > apples comparison if we used the new CMP 2.0 model.

    Surely the comparison was between J2EE and .NET as the J2EE app server vendor's were not even named. Why chose an app server that does not support the latest specs? Why go the lowest common denominator route and drag App Server A down with App Server B?

    But the question most people seem to be asking is "WHY???"

    Why did TMC (supposedly) pick a fight that they were never going to win? J2EE (especially using entity beans with the additional layer of abstraction they entail) was never going to outperform .NET running on the Wintel platform. All things considered, I'm surprised it came as close as it did.

    And why did TMC focus on performance to such an extent? This problem can be solved by adding bigger/better hardware - and hardware costs usually make up a relatively small proportion of the total spend for enterprise-level applications, when compared to development, management and maintenance.

    Because, if the TMC did not know, or at least suspect the final outcome of this comparison before they undertook it, then I don't think that reflects too favourably on their abilities.
  340. Does(did) anyone think that MS software wouldn't work better with their own platform? It had better.

    This only proves that if you exclusively, now and forever, will use Windows (client, server, pda, etc.) and don't mind paying MS for everything at their price then just use .Net. Yeah, there are some things missing current from VS.Net and .Net but one can live without them. It may be a little painful now and then (ok some of them really bother me).

    If you don't know that Windows will be your only platform or you don't want to be tied to MS or Windows then [MS].Net is not a choice. Please, no one chime in about Mono. It is not MS.Net. We don't know how well real and useful [Mono].Net programs will run on Linux. We don't know if real (not 'Hello World') programs written for one platform will run on the other. History says NOT (not for lack of Mono trying).

    When it comes down to it is Java successfully being used in environments where scallability and speed matter? Yes. So it comes down to who do you want to be your 'master'.
  341. I don't yet have seen the code but...

    if it's EJb 1.1 (without Local interfaces) it's clearly paying the remoteness price. Being 2.0 compliant app servers officially around from more than 1 year (the final spec is of September 2001, and a lot of app servers were ready almost immediately), I cannot understand why they're comparing .Net 1.1 (not released), with EJb 1.1 (an outdated spec)

    What is strange it's the claim this should be an "optimized" j2ee app. To me, if you don't use local interfaces (or use smart middleware ;)) this benchmark is only partially useful.

    I hope we will see something better than this in a short time.

    P.S. and... where is a clustering benchmark comparison?

    uL


  342. ...conspiracy theory...[ Go to top ]

    ....MTC made this report to drag MS into
    "easy win" it payed for. In a week MTC will publish
    results with Struts/POJO/DAO on the sqlserver and
    MS will be down. New results will be oficially
    compared only with original EJB results to avoid
    legal disclosure restrictions from MS...
  343. ...conspiracy theory...[ Go to top ]

    errata: TMC... sorry.
  344. <quote>
    I cannot understand why they're comparing .Net 1.1 (not released) with EJB 1.1 (an outdated spec).
    </quote>

    I also cannot understand this choice. Including EJB 1.1 in the comparison may be nice for demonstrating the evolution of J2EE, but it only makes sense in conjunction with EJB 2.0.

    Of course, why use Entity Beans at all? 1.1 or 2.0, remote or local, you always have the overhead of managed components, i.e. method call interception by the container. A fair comparison should use a decent, straightforward persistence toolkit, be it JDO or an open source favourite.

    Anyway, TMC's comparison is by no means fair. Note that I wouldn't be surprised if a .NET solution still outperformed a well-architected J2EE solution. But the difference is definitely not that high.

    IMHO TMC puts its reputation at stake. Looks like one has to do its own unsponsored and unbiased benchmark for valuable numbers. Actually, we have both J2EE and .NET knowhow in our team. If we just had the time...

    Juergen
  345. Juergen,

    I have to agree with you here. They've compared a .NET 1.1(not released) version with an outdated version of EJB. It would only be fair then to compare with the latest and "greatest" version of the other vendors servers.

    Now, of course there's an open question about whether Entity Beans should have been used or not. However, its common knowledge that you do take a hit using Entity Beans, in an enterprise environment that is taken in to account and balanced against other issues such as maintainability.

  346. Thanks for showing the real facts !!!!

    It was good to see the Java crowd taking this in right spirits. Cheers guys.

    It was really a big shock to me after reading all the boastings of ORACLE over .Net etc.. Though my brain accepts the results but my heart fails to do :-(

    Look at the results from a layman's perspective..
    - J2EE( with A or B appserver) & .Net
    - J2EE needs more man months to configure than .Net
    - On the same H/W J2EE throughput is less than that of .Net
    - J2EE price/performance is very high compared to .Net

    To put in a sentence.
    J2EE is complex, very expensive and less scalable.

    Why on earth will a person will invest or go in for J2EE if on any grounds ( as per report ) J2EE is not at all better than .Net.

    Funny isn't it !!!

    Around 1999 the EBAY site was down for a few hours. It costed them close to 5 Million to establish the trust back( that was big money for them) . Only after that they started to focus on getting a 24/7 reliable system.<ref> The Perfect Store - Inside eBay by Adam Cohen </ref>

    So it is not just performance !!!

    How can one get the best of both worlds. Abstractions at all levels ( H/W,OS,location,Database etc ) and at the same time very high performance.

    So we have accept the basic fact that Abstractions come at a price. At any point of time J2EE based system( even with bare jsps and servlets ) cannot scale at par with .Net on Windows !!!.

    But why is even the price of J2EE stack is not at all at par with .Net application stack.
    Licensing costs etc.

    Where did J2EE went wrong !!!
    It tried to make too many people happy. OS Vendors, Database Vendors, Transaction Monitor Vendors etc. All of them tried to arrive at level where there can have a level playing field but kept performance at the backburner !!!

    At the heart of J2EE, they think that real world systems are inherently complex. They need complex distributed transactions, complex security etc.But in real world i don't feel that is what is happening. The same 80-20% rule seems to be working. J2EE focussed on the 20 % mob whereas Microsoft focussed on the 80%.

    Now what should have happened is this. Sun should taken a very serious role in categorizing J2EE in two streams J2EE-Rabbit and J2EE-Monster.
    All the Vendor implementations should fall under any of this category.

    J2EE-Rabbit:
    -----------
    Standardise upon the the standard pieces.
    This stack should have all the components that is required to start building and deploying a fairly decent system( let us figure out the metrics )..

    Build - Ant
    JVM - JDK XXX on Linux & Windows
    Frameworks - Struts / Expresso / etc
    IDE - Eclipse / NetBeans / IntelliJ
    O/R - hibernate / cocobase
    Web Tier - JSTL, CFM( Java Version ), VRML etc
    Client Tier - Swing & SWT clients through ( XUL )
    H/W - x86
    Patterns - Abstract things to real world needs and recommend them.
    Knowledge - Presently


    So I can directly buy a J2EE-Rabbit stack from a Vendor and start my development.


    J2EE-Monster:
    -------------
    Target this for users who cannot run their business on a X86 architecture. People who use want all the big machines !!!. Here do all the necessary J2EE certifications to ensure application server portability etc.


    Now by doing this we can easily keep Microsoft based technologies at bay. Then it is an apple to apple comparison. Pull out a Vendor who proclaims he has the Best J2EE-Rabbit and compare that with .Net and publish the results. Then it makes sense to the buyer to buy which one depending on his business needs.

    I would bet if we have J2EE-Rabbit stacks from say Weblogic, Websphere & OpenSource. Then we have really true competition.

    Now the user has real choices to go in for !!!.


    They can do a real world comparison of a J2EE-Rabbit Stack with .Net and publish the results. I would guarantee the results would be like this

    Vendor A - For exampleJ2EE app provider like ( Weblogic, Oracle, IBM etc )

    This would be the results:

    Performance : .Net > J2EE-Rabbit Vendor A > Opensource-Rabbit
    Price : Opensource-Rabbit > .Net > J2EE-Rabbit Vendor A
    Vendor Lockin : .Net >> Vendor A > Opensource-Rabbit
    Multi-Platform : Opensource-Rabbit = Vendor A
    Extensibility : Opensource > Vendor A >> .Net


    thanks
    my $0.02 :-)
  347. Interesting. I was just raving about lack of good comparisons on another site, and now here it is. This is the first good attempt to unveil some of the issues concerning J2EE vs. .NET.

    Now, I would like to see a better comparison. To me it seems that J2EE side had to give .NET a head start because .NET does not have some features of J2EE and the J2EE Petshop has to have somekind of woodlegged implementation to compensate.

    Now, let us see a real comparison where J2EE is geared towards performance. Leave the BMP away and use stateless session beans and a technology like Hibernate or just use JDBC. Then start using JDK 1.4. because .NET is using 1.1. Surely you can squeeze a version of the app servers from the vendors that run with JDK 1.4. Have engineers of each app. server tune their own stuff. Use the same database.

    You have been cutting too much slack for the MS people to come in to the test - J2EE runs with older technology, bad solutions for this type of app while MS guys get to choose the best solution and play their game like they want to. Granted, MS has to give up using the stored procedures for this comparison to be fair.

    I bet you would see those bars evening out in the charts if the changes that I suggested would be made. I still think that would be a more fair comparison. Too bad we will never see those results published. MS just won the publicity wars and they will hold on to this and use it well.

    --
    Tero
  348. Well,

    The Middleware Java Pet Store design and architecture really sucks like an INVERTED HURICANE (to paraphrase Martin Fowler):

    A) BMP is a VERY bad idea. First, it performs bad (a lot is written on the web about that, remember only ?n+1 DB call per finder method problem?), and second, it requires writing all JDBC code by hand (so it does not improve developer productivity).

    B) Forget EJB 1.1 CMP, when EJB 2.0 local interfaces promise more performance (although I believe both vendor A and vendor B have implemented optimizations on local calls; if not, then this benchmark is far from fair).

    And the deployment environment is not ideal:

    C) Don?t use vendor A (WebLogic, my guess) nor vendor B (WebSphere, my guess) J2EE Server. Use vendor C (best ECPerf performer, Oracle) and vendor D (fastest EJB server, look around and you will find it).

    I believe that by eliminating entity EJBs in Pet Store application, by using Sun?s JDK 1.4.1 VM and by deploying to the fastest J2EE Servers Java Pet Sore can catch MS?s one.
  349. hey guys...

    this group is so defensive and "flighty" about any perceived weakness in the Java platform...and so shit scared about MS...

    don't be so easily intimidated...


    Java kicks heavy butt!

    you want to talk about performance?

    as if it really mattered considering the nature of most modern business apps...

    but, ok, we can discuss performance...

    consider this article about the new IBM PowerPC 970, this chip is going to scream...and IBM is almost certainly going to port Linux to this chip....

    http://arstechnica.com/cpu/02q2/ppc970/ppc970-1.html

    4x, 8x CPU servers...with state of the art RISC chips, low power consumption, state of the art branch prediction, yada yada, all at workstation-level prices...

    and a kick ass, open source, OS, to ride on top...


    oh...but that's right....

    you sold yourself to the .Net platform, you bought into the hype, and forgot about first principles, so this wouldn't be an option for you...

    oh well, too bad, so sad...

    we use Java you see, and we can turn on a dime, we can make decisions, and we can even change our minds if it suits us...

    you, on the other hand, are gonna have to wait around awhile, and settle for 32-bit P4 crap, or Xeon crap, until Intel finally gets its Itanium strategy together, and delivers an affordable, and fast, 64-bit CPU...

    this will happen, no doubt, eventually, and then you .Net guys can enter the 21st century along with the rest of us...

    but of course, if Itanium turns out to rock, well, we can just flip on over...all it takes is one shiny new JVM port to Itanium...

    and then of course their is the absolutely huge, expansive, selection of open source, and HIGH QUALITY, libraries that are available for java...and this list just keeps growing and growing and growing...

    it is really amazing, and powerful, how big the third party community is getting...

    all of those who question the quality of open source java projects, can go circle jerk for awhile, and contemplate on the futility of ignorance...

    but never mind...

    if you don't believe in java, if you can't see the power of it, and understand the beauty of the language, and appreciate the simplicity of it all, then off you go, and the best of luck to you.


    Peter





    "oh well, too bad, so sad"

    "we use Java you see, and we can turn on a dime"





    sub $10,000 8x CPU servers...


  350. OK! I have some benchmarks as well, concerning J2EE and .NET. Here we go! You wanted to compare apples with apples, let's compare!

    I have 3 different enterprise environments.

    1) Sun Solaris box
    2) HP-UX box
    3) IBM z series running linux

    I've got nice results for my J2EE application servers. But... Wait a second! I have a blank page for .NET because I'm still waiting for my installation. I could develop, install, configure and run the whole application in J2EE world but I'm still at the installation point in .NET. So I'll call you back when my benchmarks' .NET parts are over and we'll discuss.

    ========================

    C'mon guys!
    OK! It's nice to see the real world facts! .NET runs faster than J2EE on Windows. We should take our lessons. But let's face it, if you have a Ferrari and you can only drive that in your back garden, it's worthless for you when you want to go shopping (I guess the trunk is not big enough for the shopping bags anyway :-).

    This .NET application may run faster than its J2EE counterpart however if the J2EE application runs FAST ENOUGH for the business I'll take it. What was the performance requirement imposed by the customer for "The PetStore"?

    My main point is that J2EE is fast enough for the businesses out there.

    I'm suprised with the result but I believe, we, the J2EE community, will work harder to find better ways, even on MS platform.
  351. Yagiz Erkan wrote on October 29, 2002 @ 09:17 AM:

    <quote>
    This .NET application may run faster than its J2EE counterpart however if the J2EE application runs FAST ENOUGH for the business I'll take it. What was the performance requirement imposed by the customer for "The PetStore"?
    </quote>

    So you are willing to pay more and work more for something that's "fast enough" instead of paying less and working less for something that's better.

    Not a very wise business decision, is it?
  352. <Q>
    So you are willing to pay more and work more for something that's "fast enough" instead of paying less and working less for something that's better.

    Not a very wise business decision, is it?
    </Q>

    Who said it costs more? It doesn't have to. With .Net you pay (and pay) MS's price. With Java you get choice (free to bunches of money).

    Why is .Net better based on this comparison? This comparison shows that given the criteria .Net performed 'better'. Far from scientific and far from realistic.

    It also is not a wise business decision to put yourself totally at the mercy of one vendor.
  353. Posted By: M Sarbu on October 29, 2002
    <quote>
    So you are willing to pay more and work more for something that's "fast enough" instead of paying less and working less for something that's better.
    Not a very wise business decision, is it?
    </quote>

    :-)
    I don't know if you like/are familier with a card game called Bridge. Sometimes you loose the turn in order to win the next ones:

    Even if we imagine that your project is creating a system from scratch without existing infrastructure (I don't know how many virgin-environment-projects there are out there?), I'd say "there are always solutions for a cheaper J2EE system, obviously more open solutions than .NET ones".

    What would be the life span of your new system? 1 month, 6 months, 3 years, 5 years?!? In 3 years, how many times you'll need to upgrade applications and OS with MSFT for extra money? To be able to manage a standard size MSFT production environment, you'd need a product like "Application Center" and it's far from being free. How many flavors of "Application Center" would you need to buy? Maintenance of the code? What's happening now to the DNA/DCOM applications developed 3 years ago? Can they just migrate and run the same code on a .NET server? etc. etc. (I still have books saying "DCOM/DNA is THE killer architecture, even Unix flavors are being developed..." - same things we hear about .NET).

    This list is long mate! But don't get me wrong, I'm the technical architect of my company. We manage MSFT projects and J2EE projects. I try to keep up with both worlds. But I still feel that MS solutions are like giving away free drugs in the beginning with a huge marketing effort. But once you get into that cycle you're spending the money that you thought you saved 3 months ago and maybe more.
  354. That was a cool one..
    :-)
  355. This is such a shame tha M$ cant beat the Java and now they are bribing companies to produce flase docs. Well this been M$ motto if you cant beat em then buy them . Linux and Java will bring MS down ......
  356. For once Middleware do something right!!! I would like this perf test done in Linux where we run the both the Java and .Net petstore app run. Why linux because thats were both apps will prove it can run on any platform and Linux is the OS for server & desktop (redhat 8.0) that will dominate for now on. If you all run this in windows I have to say you are bias to MS as .Net is optimized in Windows. I hope Middleware is not M$ troll. Second write the Java app with just servlets and use datasource database pooling. I challange Middleware to complete this test other wise please do post these kinda hocky results.
  357. Hi guys,

    in my opinion the discussion is far away from reality.

    Ok, .NET performacce maybe better than J2EE...so what? (Even if I agree that the J2EE Pet-Store could have been implemented with a much better performance)

    As I experienced at many customer side projects, the decision for the programming platform is driven by many other facts. Look at the environments at most of the insurances and banks (as it is here in Germany).

    - S/390-zSeries hardware
    - Many "old" IMS and CICS applications
    - Many OS/2 clients (believe it or not)
    - DB2 or Oracle
    - RS/6000 & SPARC Clusters for the Web
    - A big development team with a lot of COBOL and PL/I skills + a lot of investment in Java know-how and tools

    How would you integrate .NET in this environment? WebServices? You are joking...too slow (the network traffic alone would be too slow) and I don't believe that they are reliable for mission critical applications.

    Would you implement the old application from scratch? This is a even bigger joke. (millions of lines of code)
    And even if you imlement it from scratch. Would you use .NET with some hundreds of servers clustered? Is it manageable? And would a company which relies on security (banks, insurance, etc) switch on a system, where security bugs are reported daily?

    And by the way, the only really nearly 100% available hardware platforms I know are S/390-zSeries and Compaq NONSTOP Himalaya, both having no .NET support.

    The power of J2EE is its flexibility and the vendor support from big companies like IBM, Sun, HP, Compaq, Oracle, BEA...

    Anyway, nice report from The Middleware Company. I hope it was not only a marketing move...the article on MSDN seems to praise TMC and this report a little bit too much...
    MSDN TMC report

    Mirko Novakovic
    :wq
  358. TMC on Microsoft

    I don't believe that, really! If Middleware doesn't got anything (money, consulting, training...) from them then they are idiots.
  359. If only the real life situations were so easy to benchmark...

    TMC is a consulting company. They may be trying to get into .NET consulting as well. One benchmark isn't enough to choose one or another. I propose to run 5 benchmarks and compare their results: .NET on Windows, .NET on Linux, .NET on Solaris, .NET on HP-UX and .NET on AIX.

    "There are three kinds of lies: lies, damn lies, and statistics." - Mark Twain
  360. I agree with the post above...

    Having just read the report I can say that it is a more stinging indictment of the latest version of Websphere (B I beleive?) and an older version of BEA Weblogic (6.1 perhaps?) can't run the older J2EE spec (1.1) faster than the latest beta version of .Net on a tuned Windows box! Take that for what it is worth...

    Next they should run a test with the latest version from BEA (7.1) using the jdk 1.4.1 or even the beta version of Websphere (5), tuned to use the new NIO classes, on Windows against the older 1.0 version of .Net and see what the tests show.

    Or better yet, use OC4J (with new 9i Type 2 JDBC drivers), Orion, JBoss 3.x, Pramati (or any other App server that is better than the 2 they chose) etc on Linux, Solaris/Sparc, AS/400, HP-UX or z390 against anything MS has and publish the results (oh wait...no .Net for any of that).

    What I have learned from this "benchmark":

    1) Don't use Entity Beans for 90% of common development (including a "petstore" app). Other persistance models will do until scalability outwieghts performance.

    2) Don't use Websphere on Windows, if you can avoid it (but we already know that, right?)(Actually I wouldn't use Websphere period if I didn't have to, but I digress...)

    3)Make sure I compare apples to apples whenever I compare something, or I'll look like a dolt (Solaris 9 on a 64 bit Sparc runs better than Windows 95 on my PII....so what?)

    4)When getting "Architecture" advice from the MiddleWare Company and/or TSS, eat one large grain of salt (see #3)

    5)Really there is more to a platform than pan-ultimate performace....If I really wanted performance, I'd stick to COBOL or assembler on the mainframe, because say what you want, that code is FAST. If I need to consider maintainability, support (free community that is), lots of libraries, maturity and (most of all) CHOICE, .net does not come close.

    6) Don't trust TSS (see #3) for Architecture advice, since after reading the report, I know of more than a few ways to make the petstore better...oh, did I already say that?

    Again, thanks to TSS for shaking things up and causing a controversy.. I hope this results in better J2EE code out there. I 'm not sure if that is what they meant to do...to bad it has cost them any credibility they had.




  361. I completely agree with you

    >Having just read the report I can say that it is a more >stinging indictment of the latest version of Websphere (B >I beleive?) and an older version of BEA Weblogic (6.1 >perhaps?) can't run the older J2EE spec (1.1) faster than >the latest beta version of .Net on a tuned Windows box! >Take that for what it is worth...

    Not only that, but they used an old version of JRockit 3.1.5, they de-tuned server A (if it is WebLogic 6.1 (or 6.0), it comes with 15 threads per instance by default)

    I think TMC should come clean and detail exactly what versions of A and B they were using.

    I'm afraid that my opinion of TMC (and by inference TSS) has sunk today.

    Paul
  362. The developers of the J2EE benchmarks should read Flyods Design patterns book or better still attend the class conducted by the server side.

    Raghu
  363. Well, many congrads for a well written and seemingly unbiase report. Job well done!!!
    I don't understand why Java developers are getting defensive about this report. This report is not a comparison of languages and vendors but the lang and environment and the technologies behind them.

    This report is not aim to make java look bad and .net look good but try to get to the "truth" as close as possible. For java programmer feeling bad about this report, GET OVER IT and write good code as usual using the best practices known todate. If ppls start thinking .net rules and java stinks, grow up! A good working system is not just about performance and lines of code.

    If there's some truth to the report then, this just means that Sun/IBM/BEA/<JavaHeavyWeight> needs more work in optimizing compiler and JVM runtime. You can't be the best all the time, M$ has the luxury of learning from Java/JVM , both the good and bad, and Java world should do the same.
    This will only good for the Java community.
  364. Im surprised that so many ppl here are buying into this "unbiased" comparisons.

    As somebody has already pointed out, the following two things speak a lot:

    >1. Microsoft designed and coded their .NET PetStore
    >application and a Microsoft employee performed the
    >configuration / performance tuning. To my mind, whether or
    >not The Middleware Company were really trying to
    >beat .NET's performance figures (which given the use of
    >entity beans seems unlikely), it would have been much
    >fairer to have given the two J2EE vendors a chance to
    >provide their own PetStore apps and configure /
    >performance-tune them.

    >2. The Middleware Company provide performance figures
    >on .NET 1.1 RC1 and Windows .NET Server RC1. These are not
    >fully fledged production products. There were no signs of
    >any beta products from the J2EE app server vendors, nor
    >did they run the J2EE app servers on Windows .NET Server
    >RC1. This is obviously heavily biased in MS's favour.


    Dump these "we-want-your-job-and-will-come-right-to-your-front-door-to-get-it"-guys and look for your Java resources elsewhere ... The times are getting tough and we all work for money!



  365. This is not fair for J2EE[ Go to top ]

    First of all, I'd like to congratulate the Middleware Compagny for their astounding work. It's probably most of the less biased comparison done so far between .NET and J2EE.

    But I think that your analysis is not very fair for J2EE in several aspect.

    PERFORMANCE
    - You don't make use of Local Interfaces, all the components are Remote, which suppose that all method invocations goes over RMI.
    - Since .NET is more like JSP/Servlet/SessionBean/JDBC, you should have make the comparison with that kind of stack. Im 100% that J2EE will have perform MUCH more better.
    - If you do some testing on Windows, please use MSSQL because its probably the most optimized DB on Windows platform.

    LINE OF CODE
    - I when through the java PetStore code. Sorry guys, but the exception handling really sucks. There is a lot of:
      void foo() throws xxxException
      {
         try
         {
           ...
         }
         catch( e xxxException )
         {
            throw new xxxException( e.toString() );
         }
      }

    - You are user BMP/DAO instead of CMP2.0. This have a big impact on the total LOC.
    - Why not using xDoclet? I aggree that it doesn't reduce the number of line of code, but at least, it give to the develloper a smaller 'view' of its code.

    PRICE
    - Hey guys, J2ee is not just about WebLogic-WebSphere/Oracle. There are other players like JBoss-PostgreSQL, that in term of price, are unbeatable.
  366. This is not fair for J2EE[ Go to top ]

    This test is more accurate than the previous comparison but it still says VERY LITTLE and is embarassingly flawed !

    I will send a MUCH longer response but I have ONE QUESTION for the people at TSS as follows :

    **************************************
    What would we as a development community have to do to get you guys to RUN THESE TEST AGAIN WITHOUT ENTITY BEANS.

    You had someone write DAO objects for the MS code. Let's do the SAME for the JAVA CODE.

    I would like an ANSWER from the responsible people at TSS.
    **************************************

    Moreover, Run a baseline test to get some performances of just the databases. It would be nice if JPROBE or something was used to see where most of the time is being spent

    I would like a response from someone who was responsible for the test. Or an explaination why you won't run this test !!!!















  367. Server Side has done this hype just to gain more money.
    Just look at suggestion of SS screen while trying to download testing report:

    ?My company is trying to decide between J2EE and .NET, and might be interested in hiring a Middleware Company consultant to come to my site and help with that decision.?

    This was another unfair comparison. You understood this if have read this thread and not MS fan.

    It is not the first time SS is used by MS guys for marketing .NET

    My opinion is:
    MS.NET is not bad choice for middle size applications with no integration with non-MS legacy and with tight budgets.
    Also it is good if you have .NET experts and don?t have J2EE experts.
    Also it is good for ASP model when you don?t bother about platforms your customers use.

    You shouldn?t use .NET for really critical applications (stability, security, much lower scalability)
    You shouldn?t use .NET for governmental applications (security, single vendor dependency)

    Take open source for really low cost applications if you plan to distribute many installations
    Take J2EE for very complex and huge systems involving integration with multiplatform legacy applications
    Take J2EE for any other kind of application if you have qualified J2EE developers and don?t have .NET developers

    I?m sure .NET works faster on windows platform on similar Intel box
    I?m also sure that with given amount of money faster system will be developed on java (open source products + little bit more hardware)
    I?m also sure that faster in absolute figures and significantly more unbreakable and stable system will be developed on java (mainframes, unix)
  368. obviously you have no idea what .NET platform is. no point arguing with you
  369. Best practices?[ Go to top ]

    On page 5 of the report:

    >>2. Both applications had to be created according to >>best-practice coding standards such that serves as a valid >>design pattern that real customers can follow when building >>their own applications.

    Best practices? I guess they (The Middleware Company) never read Floyd Marinescu:s "EJB Design Patterns" book. As stateless sessionbeans provides performance compared to a servlets-only solution, why should Entity beans (used in the test) still be considered "best practice"?

    The J2EE vs. .NET graphs in the test reminds me of the differences in the "Performance scalability" test made at Rice University - when Entity bean solutions were compared to Stateless session bean solutions. Everyone knows that each muterator call on an Entity bean yields a network call. Everyone knows that using a stateless sessionbean together with JDO:s is the prefered way of doing it instead. Since everyone knows these things, why are they not considered a best practice?

    Now let's consider the patterns in Floyds book as best practices, and redo the tests - using XDoclet and Struts as well, since these tools should really be considered best practice these days (is there anyone NOT using them?)

    JOnAS on Linux together with PostgreSQL is still cheaper than anything MS (or Sun/IBM/Oracle, for that sake) will come up with, and for a middle-sized company of 0-500 employees, running your companys systems on a single standard noname 2GHz Athlon running Linux,JOnAS & PostgreSQL (costing only in hardware) is a reliable, cheap and customizable solution. No .NET Server licence, no SQL Server license, no what-so-ever licence (if you don't need support, that is). Cheap, versatile, and functional. What more do you need, as a company?
  370. A couple of questions here:

    How "loaded" was the database to begin with?
    1G? 2G? 3G? 20G? I can't find a reference to this in the paper.

    I ask this simply because SQLserver doesn't scale well. For small-medium apps, sure. But for "big", terra+ data, you have to go with the big guns, like Oracle.

    Why wasn't Oracle used as the DB in both cases?
    This may give a better apples and apples comparison for larger applications, although I understand that most .NET apps are going to go with sqlserver.
  371. Real Architects Real Issues[ Go to top ]

    It's been very amusing reading the various posts and flames written here on this. It always amazes me how polarized the views can get.

    Why do we get so entrenched whether this framework is so much better than that framework. As architects our job is to design and develop the best solution for the job we are hired to do. That means evaluating the existing base of experience and available resources as well as the best technologies.

    If J2EE is the best solution given the situation you are in and it can be easily supported when you are gone then use it.

    The same goes for .NET.

    No architect worth his/her salt is going to make a design/purchase descision based on a Tutorial Example. They are going to prototype what they need to do and examine all the options and then make an informed descision. Yes, we look at these types of benchmarks and analyze them but the final decisions come after we take the time to see for ourselves.

    Also, so what if TMC is starting .NET services. Is .NET not a middleware offering? Right now TMC has focused on J2EE beacuse when J2EE came out .NET wasn't even a twinkle in BG's eyes. It only makes since that .NET would evenutally become an option. But it will never be the ONLY option.

    Andrew

  372. Real Architects Real Issues[ Go to top ]

    Well said andrew.

    In real world, we do not necessary deliver a pure Java solution. Every enterprise has its legacy systems that require the achitects to propose a hybrid solutions.

    cheers.
  373. Real Architects Real Issues[ Go to top ]

    But the TMC PetStore is already 17 times better than Sun's >
    I'll tell why it is almost the original Petstore1.2
     > They used SAX & DOM :Faster xml parser should be used Castor for instance
    >WAF the Controller is almost the same:screen mapping is still managed by ScreenFlowManager.The main problem is in the controller more even than the inciminated EJBS. to invoke perform on the EJBAction u have to go through the webController,the EjbController and use the StateMachine.
    Why dont' we just store StateMachine in an InstancePool and use it straight after the HTMLAction.JDK collection is also slow Trove Collection will definitly make the controller faster ...and much more should be done
    THIS IS NOT A 4 MONTHS JOB ..4 DAYS may be
    Faisal
    UK,BIR
  374. A lot of talk here, not much action. I'm considering putting together a WAR that will implement the PetStore functionality using 'best practices for performance' instead of using things like Entity beans for J2EE's sake. I've done numerous web applications, and this seems like a simple order entry/order status/customer admin functionality (although I haven't gone through the website extensively). In fact, I'm willing to write it to use the same datastore as the .Net version (sql server). I don't think it will be too difficult, probably a few weeks of my spare time (and I'll take their web pages and images to save me time if the architecture is sound). My only concern is if I went through the work to produce the WAR, would TSS be willing to take their test environment (I don't have the resources to get the load testing equipment that they used in the test) and drop my WAR in it and run the tests. I know this is time out of their budget, but I'm willing to dedicate the time if they are.

    -Chris
  375. Being a messenger[ Go to top ]

    Floyd Marinescu writes:

    ..."TheServerSide.com is an independent unit at the Middleware Company and did not participate in the performance comparison (note also that the report was posted on TMC corporate site, not on TSS).

    The results of TMC's performance comparison caused us to debate internally about whether we should link to it from TheServerSide"...

    So it was the results that made you debate whether to link or not, that's amusing...if the results had been in favor of J2EE you would automatically publish them, no matter the quality of the work?!? It's also amusing that the comparison offers no conclusion or summary chapter, did they fear that this would be even "worse" than just offering the measurements?

    On the whole I believe tests like these are difficult to evaluate, and certainly in such an emotional environment as this. I am really fed up with the "Microsoft/Sun/.NET/Java is superior/crap" kind of sentiments, since they get us nowhere with understanding the technology. This is what sometimes makes me want to leave the business environment altogether and take refuge in academia. But alas, I fear the situation is often no better there :-)

    -Mikael
  376. Reading all the responses to this thread make me think there must be a hidden agenda behind all of this. The study was conducted by TMC for one and some things are really strange.

    If TMC's motives was to show that .NET is a superior architecture for developing these kinds of applications and then start to offer .NET consulting it would make at least some sense. If you look at their main page there is not a single trace of .NET based offerings.

    As it is now all they have accomplished is to downplay J2EE as a viable option for these kinds of applications and this would probably hurt their business. Or is this all they have accomplished?

    What other motives could TMC have for publishing these results? Use your imagination, I am using mine. There has been some really interesting responses (from Cameron for one) that makes me start to speculate...
  377. To answer your question on TSS motives to publish, just read the initial post:


    "The TMC team spent 4 months performing this benchmark. They worked 100 hours per week on this and worked every weekend, skipping holidays. Trust me, they were trying very hard to make J2EE win. But in the end, J2EE did not come out on top.

    TMC had a lot of heartache seeing the results of this benchmark. We internally debated about whether we should post this or not. In the end, we decided to go forward and publish this report.

    TSS decided to post this news item, since TSS' commitment to the members of this web site is to publish all the facts, even if those facts are sometimes not positive about J2EE, so that we as a community can improve."

  378. 100 hours a week... Who was paying for all those hours? Are we saying TMC just donated 100's of hours per week of the billable folks time to do this?
  379. "The TMC team spent 4 months performing this benchmark. They worked 100 hours per week on this and worked every weekend, skipping holidays."

    How many people were in the team? 4 months working 100 hours per week equals roughly 4*4.5*100=1800 manhours per member of the team (this is an enrmous workload for one person). "Team" implies at least two members which equals at least 3600 manhours . I know that not all of this time was spent optimizing PetStore but 3600 manhours is a lot of time to do lots of stuff.

    TMC states that they did not have time (or money) to "gut" the application and try either CMP or Stateless SessionBeans/JDBC. Cameron Purdy states that some simple modifications shows significant improvements. Something is not right here.

    TMC also states that CMP2.0 could not be used since this would not make an "apples" to "apples" comparison. Why was it necessary to conduct the benchmark with these two J2EE vendors (where one did not support J2EE1.3)? Why not select application servers that implement J2EE 1.3 and leave the other vendor(s) out of the comparison?

    As I stated before, something is not right here. If this really was TMC's best effort they have lost a lot of credibility in my book...
  380. <quote>
    The TMC team spent 4 months performing this benchmark. They worked 100 hours per week on this and worked every weekend, skipping holidays. Trust me, they were trying very hard to make J2EE win. But in the end, J2EE did not come out on top.

    TMC had a lot of heartache seeing the results of this benchmark. We internally debated about whether we should post this or not. In the end, we decided to go forward and publish this report.
    </quote>
    Talk about shooting yourself in the foot! As J2EE consultants and self proclaimed experts you would expect a better job in coding from the TMC.
    I don't care about the conspiracy theories about selling out to MSFT, etc. One thing's for sure I am not buying whatever the guys at TMC are selling.
  381. "The TMC team spent 4 months performing this benchmark.

    > .... But in the end, J2EE did not come out on top.
    > ...
    > TSS decided to post this news item, since TSS' commitment
    > to the members of this web site is to publish all the
    > facts, even if those facts are sometimes not positive about
    > J2EE, so that we as a community can improve."

    The key question is, what good are the facts if most people here seem to believe the facts were based on faulty assumptions?

    I used to work in the sciences and one thing that I've learned is that when the facts don't meet common experiences, humility is necessary. It's easy to make bold statements "I discovered this and I was brave enough to publish.". It's much harder to talk to others and find out what went wrong, and run tests on your feedback to see if you were wrong of the common experience. You might very well be right, but you could be right about the wrong things.

    Unfortunately, it's very human to do the former and not the later. Unfortunately, this human failing can also result in a loss of trust and respect. Once lost, both are hard to recover.

    IMO, what should have happened was that after the second month of frustration, you should have asked your peers (in this case, TSS's readers and a comparable .NET forum), for input on the design of the J2EE implementation *without publishing your results*. That last point is important since you don't want to bias your peers. If they mostly agreed that the design was sound and allowed you to accurately measure what you wanted to measure, then you should publish the result right there and there. There's no point in wasting 2 extra months. Your reputation would be spared because you were open to our input and we are as responsible for the result as you are.

    If however, most people give concrete evidence why the tests are unoptimized and they show that you're not measuring what you're measuring, you know you have a problem on your hands. You have three choices: publish your results as is with the qualifier that most people do not agree to the design, or you can quietly dump the tests since no-one can agree on what they really mean so the results would be pointless, or you could spend the next two months digesting and implementing the suggestions. If after those two months the results still disappointed you, then few people can claim that the your results are invalid because you've already taken their input into consideration.

    In either case, our trust in you would be greater and your heartache would be less. This consultation and humility is the key reason why Agile development, which is common in Java shops, is so effective.

    In the future, please adopt this policy. We'll all be better for it.

  382. Having read your benchmark report, I noticed you clearly state in several places you didn't have the time or financial backing to rework the database access layer using JDBC or JDO. You used entity beans, which are chronically slow for basic data access. This alone would account for much of the advantage .NET exhibits here. So I have to ask ... if you didn't have the money or time to do this right, why did you do it at all??? Now we have yet another erroneous benchmark supporting Microsoft marketing hype. Your credentials in the J2EE space just plummeted.
  383. Ted, would you have preferred MS showing off the old results, which were worse?

    And why is it erroneous? Have you found any specific settings that are incorrect?

    IMO, the credentials of this site just went up 100 fold. Instead of just rehashing sun marketing, they raise valid points about J2EE and EJB performance. Which site do you think I'll trust for future research & best practices, java.sun.com or here?

    I develop in j2ee and .net, depending on the project. I only want to see the technology get better as opposed to everyone sticking their head in the sand.

    Hopefully this spurs someone at Sun/IBM/Bea/Oracle/JBoss to analyze the results and find the bottlenecks!
  384. I think some of the JDO vendors should redo the petstore using JDO.

  385. Andy,

    I've worked in both technologies too. However, I've been designing and developing J2EE systems for a long time now. You asked if I see any settings that are wrong. Yes.

    Most of the benchmarks concerned data retrieval. Anytime you compare a system using a heavywieght data access layer against a system using direct access, direct access wins. By wide margins. If TheServerSide was serious about using "best practices" why wouldn't they have designed the data access layer using the "JDBC For Reading" pattern in Floyd Marinescu's book EJB Design Patterns from Wiley? Anybody designing performant J2EE read intensive solutions utilizes this or read-only entities.

    J2EE has three data access mechanisms to choose from (Entities, JDBC, JDO). A good designer is careful to choose the right mechanism for the right task. Choosing version 1.1 BMP entity beans in a performance test? That's just bad design. I work with bright Java / J2EE people. They don't use version 1.0/1.1 entities anywhere where performance is the key issue. They know better. Why didn't TheServerSide know better? This whole thing smells bad.
  386. That's what I was thinking too.

    Publishing this benchmark really doesn't seem to make a lot of sense, for a company that is supposed to be a J2EE supporter.

    If I were to do it, I would only have considered a J2EE 1.3 solution, using the latest version of the Sun Pet Store, the latest app. servers, and running on Sun's J2SDK 1.4.1 if possible.
    I think they should only compare tests run with the same database servers. (BTW, MS now has a native ADO.NET provider for Oracle, which should provide the same performance for .NET as when using SQL Server.)

    The above choices would have provided a more fair comparison, in regards to performance. Now, concerning code size, I believe the Sun Pet Store implementation is very poorly designed and coded (specially the version used in the benchmark). That, coupled with the fact that it doesn't use libraries such as JSTL or Struts, contributes to the huge code bloat we see.

    The point is, if TMC couldn't have made the above choices (and I believe they could), then they should not have done it. After all, how can such a benchmark, that compares the latest and greatest from MS with outdated (and not so great) J2EE 1.2 software, be beneficial to the J2EE community? For me, at least, it wasn't.
  387. From Ibatis.com:

    >> October 28, 2002: Microsoft has released version 2.0 of their .Net PetShop. I'll be looking at it over the next week or two and planning the next version of JPetStore. If you have any comments, suggestions or source code, please feel free to email me.

    It's time to rock, let's go.... ;-)<

    Cool!
  388. <quote>
    The original Sun petstore used BMP. There were numerous helper classes being called. It would be more time and money than we had to gut the application and redo it with CMP.
    </quote>

    Are you aware of SUNW Java PetStore 1.3.x? If not, download the source code and study it.

    <quote>
    If you take a look at the code we have made substantial performance optimizations with BMP. The fine grained control we received with BMP (compared to CMP) allowed us to take advantage of database hints, and indexes, and minimize database calls. CMP can sometimes be constraining from this flexibility perspective.
    </quote>

    This argument is flawed. Do you indicate that with CMP you cannot use hints and indices?

    <quote>
    As for the famous N+1 database calls problem: We tried to optimize around this problem by returning the key rather than doing the database query.
    </quote>

    I don't understand what you mean. To me N+1 means you first make one call to get the list of id's, and subsequently for each id you hit the database once for the attribute values. Are you aware of Gene Chuang's Fat Key pattern that solves the N+1 problem?
  389. <quote>
    Are you aware of Gene Chuang's Fat Key pattern that solves the N+1 problem?
    </quote>

    Fat Key Pattern depends on your container implementations. As far as I know, it doesn't work in WebLogic 6.1SP2 or above.
    I tried some performance comparison between SLSB+JDBC, BMP, CMP. If you can accept optimistic characteristics, CMP with optimistic caching and optimistic concurrency outperforms SLSB+JDBC (but of course, not in a every application models). BMP is mush mush slower because it is hard to optimize its load behavior (N+1 problems, etc).

    Please rty CMP with your application model in the most recent version of App Serveres that support EJB 2.0 spec.

  390. I appreciate your heartfelt sentiment and may be indebted to TMC for conveying difficult news. Could you say more about how you may feel this could impact your business at TMC - this certainly can't be a boon for EJB book sales or seminars. Prior to downloading the results, I was asked if I would like TMC's help in deciding between .NET and J2EE - I would be very interested to know what a TMC consultant's next-step would be if the answer is .NET. Would you ever consider a curriculum that would include .NET? Much may be at stake based on your published results and how you (TMC) respond to the questions of your past customers and partners.
  391. I don't think TMC did anything bad or wrong in publishing this. It seems to be leading to lots of valuable discussion. Perhaps the most interesting point is that even among professionals, there seems to be *considerable* disagreement on how to implement with EJB.

    If you wrapped J2EE in a single IDE that generated a single style of code, you'd have the Microsoft way ... and that is the wrong way. It is only natural that in a more open environment there is going to be more iteration on the spec and open discussion/disagreement...and more ways to do things right, and wrong.

    Instead of slapping TMC on the hand and saying "bad", let's have dialog on possible improvements.
  392. The problem is not the technical people reading it. The problem is that this document will be read by decision makers (with no tech knowledge) that cannot take the report for what it is. They will take it for fact and don't bother with the benefits you get (accordning to me) from using J2EE despite the perfomance penalty (if any).
    This is playing on M$ backyard. In my opinion J2EE is usually more amied at good maintainable code where M$ is taking the easy path of performance (much easier too prove).

    Bj'rn
  393. <Q>
    The problem is not the technical people reading it. The problem is that this document will be read by decision makers (with no tech knowledge) that cannot take the report for what it is
    </Q>
    It seems the 'techical' people are having problems too. And they influence the decision makers.
  394. I urge every J2EE developer to never visit theserverside and never use the middelware services.
    This is not acceptable and the behavior of this company is not ethical.
    Subscribe to few good blogs related to J2EE and you would get much better honest news regarding J2EE that are not influenced by money.
    It is a shame that TSS and TMC have done that.
  395. I'm looking at the network diagram in the report. I've got to ask this question though "why is the web and app server on the same box?". Isn't this supposed to be a test of a 3-tier implementation. Could the .NET implementation be a 2-tier application, the report doesn't make that clear. In fact, the report tries to give the impression that it is a 3-tier app like the J2EE implementation.

    If indeed it is a 3 tier app, is .NET server beta optimizing the local communication between the web and app servers? That is using shared memory or its equivalent? Could this fact alone account for the dramatic improvement between .NET 1.0 and .NET server? Did the J2EE versions tested perform similar optimizations? Why wasn't the webapp and app server place in seperate boxes, was it done this way to highlight the difference between .NET and .NET server?

    The whole thing seems rigged to me, however I'll wait for the answers from TMC. Hello, TMC are you listening?
  396. giggle.
  397. OK, I've stayed out of this till now.

    First, Floyd: You are an editor for this sight which is pretty close to "member of the press." This means to me: "editorialize." You should editorialize. Press people regularly make commentary about firms that are related to the publications they write for. Ironically the one that comes to mind is MSNBC. You should not fear making commentary. Your technical insights should be welcome to all.

    Hell making commentary it doesn't even mean you don't like .NET. It seems to me you'd be *more* neutral if you helped us understand the flaws in this spec because this one is going to be full of marketing hype.

    I have some real problems with this whole shebang. Not because I don't like .NET. I think .NET has done some really cool stuff that J2EE should emulate. However:

    First. I understand the hardware issue was part of the reason for Linux vs Windows. But why not use Windows vs. Windows. Also, the objective issue here is price:performance ration. In that case, who gives a **** what is used. Just find me the best price:performance. But I understand the technical need for some apples:apples.

    Second: The EJB issues raised here are sound. If indeed they used EJB 1.1 then this is certainly a no good thing. We're using an old version of J2EE against a brand new Beta technology. I wanna know what servers and versions we're used especially to know about local/remote issue.

    Third. Server Versions. Same thing. The tests use old versions of servers vs a brand new beta. WTF. That make sense to anyone else. Why not use the latest beta against the latest beta? I could show you that ejb 1.0 could probably outperform Cold Fusion 2.0. What does that mean?

    Fourth. JDK. Same thing. Why not use JDK 1.4.x. Or even the latest beta, since again we're beta:beta.

    Fifth: XA tx's. Like Cameron Purdy said, this one is not so good. Was there a technical reason?

    Sixth: Who did the optimizations, funding, tuning, harware. We really do need to see some full disclosure. This should be a trust issue for BOTH sides.

    Seventh: Why not use J2EE servers *known* to be faster than the ones used. This is J2EE vs .NET. MS chose their best stuff, why not the J2EE community. Choosing WLS/WAS just because of market share does not seem reasonable to me.

    Eitgh: Why not use *real* life best practices. It looks like the MS .NET system did. But we all know that the Petstore is a whole huge bunch of "best practices." but noone would ever really use them all in real life. Some of them aren't even that performant. The objectives of best practices, in fact, aren't always to be the most performant. Best practices include scale, architecture, security, robustness, whatever. SO WTF here as well?

    I'm by no means a believer that J2EE is perfect. In fact, was an MS developer for a long time. I switched from the MS world to J2EE world for a whole host of technical reasons. However, I'll switch back if .NET is cool, and so far I think it is pretty cool. HOWEVER, if you're going to make a marketing based comparison as a third party, it seems there should be greater due diligence in ensuring the fairness of the test.

    Jason McKerr
    Northwest Alliance for Computational Science and Engineering
  398. Any comments of Rickard review of the J2EE/NET comaprison.

    I think he makes some really good point
    http://dreambean.com/petstore.html
  399. Even if TSS can ignore most of the criticisms, this does deserve a reply, as it is from a recognized expert on Java serverside programming.
    Maybe the TSS 'experts' aren't as good as they thought they were.
  400. The sources of the J2EE petstore are not
    correct. The application doesn't compile de facto.
    You need to modify the references in customer_weblogic_ejb.xml file in order to compile.

    The cleanall ant task doesn't work also.

    I haven't managed to make the whole stuff work yet.
    But these errors in ant script and weblogic specific deployment descriptors make me think that people of MDC were so upset by the results that they didn't test their
    release. Or maybe they have totally lost confidence in J2EE
    technologies and they are converting to .Net power ;-))
     
    I see a new web site title for you guys:
    The serverside.com
    Your Microsoft .Net community

    running on Windows .Net instance or W2K .Net instance => wonderful ! :-)

    Ok i admit .Net is now taking a serious lead specially in code compression and cost ( compared to commercials J2EE WAS). Microsoft offers a complete, performant and homogenous plateform to build and host entreprise applications.

    But I don't think i'll switch to Microsoft .Net technologies.
    I think they recently made a great effort to comply to standards and improve security. But if they go again into monopolistic situation they'll behave how they used to ( i.e commercial arrogance).

    I seriously hope that SUN, IBM or Opensource Java Community have received this clear message: J2EE is or could be in danger in short term.


  401. --For an interesting critique--[ Go to top ]




    If this post ever gets read amongst all this....

    I suggest you read the following critique

    Sadly, I think TMC has shot themselves in the foot here. Despite their hard work, they have released a rather flawed report.

  402. The TMC team spent 4 months performing this benchmark. They worked 100 hours per week on this and worked every weekend, skipping holidays.


    and..

    >> The reason we didn't rewrite from scratch is just the amount of time involved to do that and the amount of time available for the project


    Huh?? Didn't Clinton (at iBatis) rewrite the original PetStore into a highperforming, low-LOC DAO version ALL BY HIMSELF in a couple of weeks? (check http://www.ibatis.com/jpetstore/jpetstore.html) And a whole team working 100 hours per week for 4 months couldn't achieve the same??

    Mysterious..did they REALLY want J2EE to come out on top...? I mean, REALLY? Then why the h-ll could a whole team working 100 hours a week for 4 months not achieve what one person was able to "while watching DVDs", as Clinton himself puts it??

    Fishy...or rather: TMC is one h-ll of a incompetent company. Or rather, they are not incompetent, they just have very low IQ (small and badly working amount of braincells, in plain english)

    Confess now, TMC - MS did bribe you, didn't they..?
  403. * Editors note - Oct 29 - afternoon

    >* Local interfaces: Did we use them? If not, why not?

    > App Server B did not support EJB 2.0 so we could not use
    > local interfaces to do an apples to apples comparison.

    > App Server B did not support EJB 2.0 and thus the new
    > CMP model. It was not possible for us to do an
    > apples-to-apples comparison if we used the new CMP 2.0
    > model.

    Is anybody is interested in a comparison of App Servers A&B?
    Since these App Servers are unidentified an A apple-to-B
    apple comparison is not very interesting.

    > As for the famous N+1 database calls problem: We tried
    > to optimize around this problem by returning the key
    > rather than doing the database query.
     

    Middlewares optimization of not doing the database query
    in the ejbFindByPrimaryKey(key) method does *Not* solve
    the BMP N+1 database calls problem.

    >The TMC team spent 4 months performing this benchmark.
    >They worked 100 hours per week

    All the more reason to do it right.

    Dave Hunter
  404. It looks like the J2EE community could not accept the result. I am supprised many developer are so
    arrogant. Why not those developers just put an optimized petstore and let TMC to redo it? Or comparing
    the performance of the code from TMC and yours so we can see how much it can be stretched.
  405. Wow!!

    Let's face it... the .NET results were orders of magnitude better than those of J2EE. Even with all the optimization and code re-writing people are talking about I for one don't believe that the underlying message will change... J2EE has serious competition.

    J2EE has 'owned' the enterprise application space and as a result the community has got complacent. We have failed to recognize the issues (price/performance is the biggest) and have lived comfortably with our monopoly of this space.

    Oh and by the way... don't shoot the messenger.


  406. Its clearly time for TMC's management team to come in here and provide us with a full disclosure of what brought TMC to do this benchmark, and who was involved. If there was really a team working for 100's of hours, someone paid for that time.

    And to claim TSS is not TMC is silly. They are one and the same. Who pays the salary of the people who work on this site?

    Congrats to MS. In the end, they're the only winners out of this.
  407. It would seem there have been some great points made regarding potential solutions that would help improve J2EE scores in this analysis. I truly hope TMC's agenda is as forthright as their claim - that they are indeed merely the messenger. I would think TMC would have chosen a different battle before taking on a competing technology: JDO vs. Entity Beans; Hibernate vs. Entity Beans; pick the winner who would then ultimately face off with .NET. This thread may have much to say about these findings - but the damage won't truly be evident until your boss, my boss, our customers read this report from an MS site. I don't believe this wakes up the J2EE community - it will inspire technology decision makers in a way that will influence my career. .NET is not an unpleasant environment, on the contrary - it's an exceptional technology. I would prefer to stay J2EE - I hope the fight is fair.
  408.  Hi everyone,
        Hot thread, hot thread indeed.

    I hope my voice will be heard and my proposition will be considered seriously by all TSS community. First no matter what, I think we all should THANK TMC for posting these results, at least for making us aware of the seriosness of this competition, and for making us think what we all AS COMMUNITY should do. Second, I as many of us agree that changes made to Pet Store were not adequate to consider this benchmark a fair technological competition. Moreover I totally agree with several postings there regarding seriousness of the impact on J2EE because of these results publication. Cause I have no doubt in my mind that these results will be used by MS guys to declare their superiority. And considering the fact that many companies are making their decisions based on such publications this will play a fatal role for J2EE. Because of all these facts I PROPOSE the following:
       
    1) To all TSS community to take part in voting for credibility of this benchmark results.

    2) To Floyd personaly: Floyd I understand and respect your position regarding this whole thing, but I think you should HELP the community to resolve this issue by organazing this vote for credibility of the results on TSS.

    3) To TMC: if your intentions were pure to help the community to understand where we stand in our platform wars, I propose to review this voting results and in case if community decides to find results invalid (for reasons of using not adequate technologies in this benchmarking) recognize them to be invalid.

    I understand that my last proposition is very bold, but I hope TMC understands seriousness of these results publication and if community decides there are strong reasons to recognize these results to be invalid TMC should consider the community&#8217;s opinion.

    That is all what I wanted to say about this issue. I hope my voice will be heard and I apologize if I made a few grammatical mistakes, English is foreign language to me.

    Best regards to all community.

    Alex.
  409. I have a question to TSS community.

    I have earlier commented on the behavior of the anti MS crowd. Nothing is deigned to low (exaggerations, innuendos, pure lies, personal bashing, etc etc) if it further their cause and campaigns against Microsoft (-which by the way will publish the next version off word with XML as file format-).

    How can anyone doubt that The Middleware Company didn't really make their best shot? They are earning their living by teaching J2EE for god?s sake - they own this site! It is so refreshing that between all this marketing hype and hypocrits there is at least some people that are honest and have integrity. Their credibility has raised 100 fold.

    And to all those that says that the tests were not done correctly, if 4 months of working 100 hours a week with 10 weeks of tuning by an expert team of J2EE developers doesn't do the trick what can then by done? Just assume for the sake of argument that it is possible to do it (not that I think so), could you not agree then that it is a little too difficult? Or are you going to say with the usual arrogance "complex solutions demands competent..."? The Middleware team is not good enough?

    And all this focus on performance? What about rows of code? 2000 against 14000? Lower cost?

    The question is,

    Yahoo is the world largest site with over 200 mill users and 1.5 billon hits by day. There are somewhat over 8 mill rows of code.

    Do you think that J2EE would be a proper technology?

    Regards
    Rolf Tollerud
  410. To Tollerud:
    Yep, the TMC guys clearly are not competent enough. I feel sorry for them, cos it sure'll be hard to sell your services after this kind of crap
  411. Rolf -

    Can you look at the code used for the J2EE portion and say honestly that it is highly optimized? It's not if you know J2EE. This will help you to understand some of the responses that you have seen.

    I don't have any particular views as to the credibility of TMC as I don't use their training or consulting services, but I do know optimized code when I see it in the J2EE/Java/OO realm.

    They made decisions to go with a component-to-component test vs a platform-to-platform test and were limited in the tuning they could do based on those decisions AND the limitations of one of their choices of app servers. To do the test properly, I believe they should have written a completely optimized J2EE app for an app server (or app servers) which supports the latest specs (and there are plenty out there) and tuned from that point. Then they could have compared based on loads, etc. However, they didn't call me up to help design the test so what can I say?

    I realize this test confirms everything you have heard or believe about J2EE and so you love the results and assume that the test shows the ultimate truth since it was designed and implemented by a reputable company in the J2EE world. It only says one thing - if someone designed a J2EE web app the way TMC did, .NET would probably outperform it. The results were based on design decisions only and do not accurately reflect the capabilities of the platform. Simple as that.

    In reality, if people were designing a J2EE web app for performance, they wouldn't design it like TMC( or Sun in the original Petstore) did. You would be doing yourself a disservice if you thought that this test were the end of the story.

    Cheers
    Ray
  412. I don't think its a question of competence, what bothers me about the report is that is was it a fair comparison. One vendor, Microsoft, developed and tuned their application with the motivation that it was faster and had less lines than the petstore app. Sun, developed the pet store app to show design techniques.

    It's like comparing a race car with fiber glass bodies and turbo charged engines to a luxury sedan with leather seats and gps navigation. Now TMC comes in and claims that they optimized, so it's like turbo charging the luxury sedan, it helps, but doesn't make it faster. Then the race, begins, unfortunately its an off road course, and the competition submits a finely tuned cross country rallye car, the luxury sedan is smoked and the competition publishes the result and declares victory.

    It's just simply not fair! Session Beans will beat performance Entity Beans any day (see Rice research, in fact by a factor of 2). Optimizing the channel between the webserver and the appserver gains tremendous benefits, just ask the JBOSS and the JOnAS guys.
  413. Ray,

    Which one of the following a,b or c, do you think was the reason was for the test results?

    a) The Middlewhere Company, which is a successful teacher of J2EE and has written some of the best books on J2EE, is incompetent.

    b) The Middlewhere Company, who is entirely dependent their J2EE credibility to survive, sold out to MS for the price of borrowing a test lab

    c) Microsoft, with an almost unlimited supply of money, resources and talented people, could build a better system on their own OS.

    Take your pick.

    Regards
    Rolf Tollerud
  414. Rolf -

    I choose choice

    d) That the reason for the test results had to do with how the test was designed and implemented. (Yes I know that this is true for almost any test - that's the point).

    Again, I'll ask you if you can honestly look at the code for the J2EE portion and tell me if it is optimized to its fullest potential. Is it?

    Cheers
    Ray
  415. d) of course,

    and one of the following:

    e) some incompetent TMC employees have designed the test and its J2EE implementation, and an ignorant supervisor has approved it;

    or, more probable in respect to TMC's reputation,

    f) the unfair circumstances have been chosen deliberately, due to strategic/marketing/financial reasons.

    Juergen
  416. YAB..yet another benchmark. This thread is humorous.
    It's only a matter of time before these benchmarks are redone and J2EE comes out on top. Then the other side will complain about unfair optimizations...pointless arguments.

    So I guess peformance is no longer important? Now it's soley an issue of maintainability. You can cry all you want about poorly written code but how can you explain 14000 vs. 2000? Would you rather maintain 14000 lines of code or 2000? Also, how does this not translate into faster development time and therefore cheaper development costs? Even if the licensing fees are outrageous, who cares if you can finish the job in half the time? You're still saving money.

    I agree that if peformance were the sole factor, then write everything in assembly. Obviously this isn't really feasible due to development time. From a cost perspective we're looking to get a reasonably performing app completed with a minimal cost and development time. .NET wins hands down.

    The major reason for choosing J2EE is platform independence and this is still a major factor in a lot of decisions. If you're tied to legacy hardware, then J2EE wins. For new projects, I really don't see how you can justify using J2EE when you factor in cost and maintainability.

    I'd mention the Mono Project, but I'm sure I'll get a few flames about how it's not complete yet and how they're doomed to fail.

    First it was the use of stored procedures, now it's poorly written java and ad hominem attacks.

    These migratory complaints have gotten out of hand.

    If your religion will allow you, check out the first episode of the .NET Rocks show:

    First Episode - August 30, 2002

    Pat Hynds - Critical Sites In this, the premiere episode, Carl interviews Pat Hynds from Critical Sites, a Sun Partner, IBM Partner, and Microsoft Partner. He shares his insights into .NET and provides real-world stories of development in J2EE and Visual Studio.NET, and compares the results.
    http://www.franklins.net/dotnetrocks.asp
  417. If your religion will allow you, check out the

    > first episode of the .NET Rocks show:

    Thanks! I'm vegetarian!
    And I guess it wouldn't feel like circus without a clown! So your comments are welcome...

    =====

    Anyway, where were we?
    Still absolute silence on the TMC side...
    Whoever worked on that test must be reading those comments! Aren't you going to defend YOUR best practices?
  418. I can understand the feelings of the Java community and I can agree that it is not really a fair fight.

    But the unfairness is in the resources which is at Microsoft disposal. It is like attacking with sticks and stones against a army of 500.000. They have tons of money, resources and thousands of talented people. In Redmond even the receptionist has an IQ of 150. And in the J2EE case, they could build upon the knowledge from Sun Java Technology. And in addition to all this they have their own operating system, which they know intimately.

    What irritate me are the morons that claim that MS is behaving criminal or immoral. In every crime there have to be a motive! And what possible motive could Ms have for criminal activity when they already have all advantages?

    It doesn't help the Java/Unix/Linux community to shout things that are not true even if it is in what they perceive a good cause, such things always backfires. Instead take up the real problem which is quite simply that that Microsoft is too big.

    It is no shame to loose against such an overwhelming force. Microsoft can always win any fight which has their attention.

    Regards
    Rolf Tollerud
  419. <Q>
    And what possible motive could Ms have for criminal activity when they already have all advantages?
    </Q>

    Greed? A very strong 'motive'. And current 'advantages' don't guarantee future ones.
  420. Rolf,

    To quote a line from the movie Billy Madison:

    Mr. Tollerud, what you've just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling, incoherent response were you even close to anything that could be considered a rational thought. Everyone in this room is now dumber for having listened to it. I award you no points, and may God have mercy on your soul.

    Peace
  421. Reading this post made the time it took to get here well worth it.
  422. *******************************************
    For those who are not monitoring the top of this thread, you might want to take a look. The Middleware Company has published an FAQ responding to questions from this thread.

    They've also published a document stating that they are considering doing a re-test, based on the feedback presented in this thread.

    And finally, for, the 4th time, I'd like to remind every one that TheServerSide is simply the news agency here. We did not participate in the test. The Middleware Company performed this test and published the results on their own website. We simply linked to it. Please stop attributing this benchmark to TheServerSide.
  423. guys,

    only shortly:
    1/ such benchmarks as done by TMC are not a work from professional company, ergo no competence in the area.
    they lost their credibility.

    2/ ;; they are considering doing a re-test ;;
    sorry, i'm considering to boycott TMC/TSS site.
    this is the only way to stop MS marketing machinery.

    yours
    janosh
  424. Looks like Soon TMC will start .NET classes. After looking at Petshop code,I think, Definately people of that club need some lessons on Design patterns.
  425. Ray,

    I agree that there are still much to be done to improve TMC version of Petshop. But.

    Once in the stream of life I was a green young programmer. I constructed an algorithm which made me so proud that I posted it in the cafeteria billboard and promised a bottle of whiskey to anybody that could improve the code. Imagine my chagrin when before the end of the day, the code had been improved 30 times!

    The point is that code is never finished. It's always possible to improve. That is by the way one of the main differences between "a product" and "contract programming.

    TMC spent an incredible amount of time and effort and they also reach a 17 fold increase in performance. That's pretty good I think (by the way that means that MS claim of 28 times more fast than the original Petshop was pretty close). But somewhere you have to draw a line against time and money.

    In .NET, you get this performance "out of the box"

    Regards
    Rolf Tollerud
  426. <In .NET, you get this performance "out of the box"/>

    Rolf,

    I think that the MS team did optmizations also, so it's not "out of the box".

    Regards,
    Alex
  427. Rolf -

    I think the point of many in this thread is that perhaps TMC spent all that time and money on the wrong approach. I personally think they spent time and money on the wrong application (petstore), but that's beside the point.

    I know of a goodly number of .NET apps that have performed horribly "out of the box", so I suspect that it takes good design and implementation strategies in .NET as well.



    Cheers
    Ray
  428. Does this indicate where TSS stand in the food chain within TMC?

    IF TMC were sincere in their motives why couldn't they let people like Rikard evaluate the J2EE petstore before running the benchmarks. After all M$ supplied their own code didn't they?





  429. M$, $un, Ora&cent;le,[ Go to top ]

    MS is no worse than Sun, and M$ is really not all that funny or creative...

  430. "out of the box" is because Microsoft engineered the application specifically for performance.

    Sun engineered the J2EE version to use as a case study in J2EE design patterns. Performance wasn't a consideration.
  431. Robert:

    You wrote:
    "out of the box" is because Microsoft engineered the application specifically for performance.

    Sun engineered the J2EE version to use as a case study in J2EE design patterns. Performance wasn't a consideration.

    But isn't the point of a "Case Study" is to STUDY the case to learn how to do something the right way?

    I'm missing the point...



     
  432. The comparison sucks.

    You have to use the identical database and the identical HARDWARE to compare the two applications.

    The use of local interfaces would make no difference, at least with server 'A'. App server 'B' seems to be not usable at all, so why do you use such crap (sorry, but it's sold for much money) ? But it makes a difference how you handle the transactions stuff.

    Just for those who believe that EJB's are slower than a tradtional servlet app: Today I visited the workshop 'The performance of the J2EE APIs' (BEA WLS) and the best performing application in their tests was with EJB's (CMP 2.0) and NOT with servlets. Think about that, especially those of you who think they can do it better themselves.
  433. In truth, it's not a good comparison...but it's got to do more with the inherent weaknesses of entity beans. You just shouldn't have to work as hard as TMC did to get effective persistence with adequate performance. Maybe they didn't walk the dog far enough, but how much is enough???

    Take a look at this report from Rice University that compares many different forms of CMP and BMP with a simple session facade in front of a raw database layer. In fact, that's what the Microsoft's solution did. And that's probably a more accurate comparison.

    http://www.cs.rice.edu/CS/Systems/DynaServer/perf_scalability_ejb.pdf

    Based on that report, I would expect a session based implementation to be much faster, probably by a factor of 2-1 or better...but then, that's no longer PetStore, is it? In the end, it's hard to fault TMC for making a mere 17 fold improvement in performance. In fact, I'd say that many of the CMP/BMP "best practices" are no more than band-aids over a technology that was never intended to be used in this way.

    The J2EE community needs to wake up. This could very easily be a production application. In fact, the production applications that I have seen more closely resemble Pet Store without the improvements.
  434. Don't panic !!!

    According to Rickard ?berg, "Several independent sources have now confirmed that The Middleware Company was indeed paid by Microsoft to conduct this report."

    Go to http://dreambean.com/petstore.html for more details.

    Tschuss
  435. "Again, I'll ask you if you can honestly look at the code for the J2EE portion and tell me if it is optimized to its fullest potential. Is it?"

    But the TMC PetStore is already 17 times better than Sun's original! If you think TMC people are idiotic, in what position does that leave Sun programmers (the writers of the Java API, the pushers of new Java specs)? This really makes me nervous.

    I know the original Pet Store was not meant for benchmarks but for best practices. Furthermore, I am a real fan of natural design in which you think in terms of elegancy, simplicity and flexibility and not in terms of speed. But if EJB's elegant and flexible design makes you soooo slow, may be something is terrible wrong with the EJB standard itself.
  436. But then what about Sun programmers?[ Go to top ]

    But the TMC PetStore is already 17 times better than Sun's

    > original! If you think TMC people are idiotic, in what
    > position does that leave Sun programmers

    Simple. The Sun example tries to use as much J2EE technology as possible in one example for the sole purpose of showcasing the technology and some of it's best practises.

    In order to have this shotcase application, they *purposely* violated the #1 best practise of any design -- the KISS principle. Anyone who's been on a project that uses technology just for the sake of using it knows how much of a performance and maintenance nightmare it can be. That's what happened in the Sun PetStore.

    Because Sun intentially ignored the KISS principle, it doesn't matter how good the Sun designers are, the performance would suffer. A platypus by any other name would still be butt ugly.

    If you want to see one example of how a KISS implementation of PetStore would look like, take a look at www.ibatis.com.
  437. Ya, but the results are so dramatic that this goes well beyond mere desings and kiss methodologies.
  438. OK, Lets demonstrate the advantages that Open Source is supposed to have: Who has done a functional equivalent to the MS Petstore done with J2EE "done right"? It's been two days, c'mon people, we're slacking off here... ;-)



  439. Kudos to TSS on the updated disclosure! I think the only thing "missing" is what involvement the "vendors" had in actually defining what the test would be (e.g. which vendor suggested requiring XA for J2EE?)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  440. Let me congratulate TheServerSide and MiddleWare Inc. on investing their time and money to develop a comparison!

    I imagine their goals was to gain credibility and market visibility, not to invest money and be criticized.

    Show me your tests, and code, I say to those that throw stones.
    I assume that decision makers see the courage of MiddleWare and that they call it like it is. Something you want from your consultant, to tell you the real story. Or do you just want to hear oh, you are doing it right, you do not need an expert for that.
    Maybe charge for the report next time.

    .V
  441. This report does not show anything except how incompetent The Middleware Company "experts" are.
  442. From Rickard's site:

    "Several independent sources have now confirmed that The Middleware Company was indeed paid by Microsoft to conduct this report."

    TSS + TMC, Floyd: Shame on you.
  443. Let's go straight to the point:

    If there are no resources (human or financial) to teste the things the right way, we simply don't do the job.
    Otherwise we are compromising all the effort made by J2EE sponsors (Sun, Oracle, IBM,...) in defining open standards.

    I do not see any explanation for this kind of report. It must have been something more in the background that explains this.
    And for those who argue that PetStore is not a good model for benchmarks (even with the improvments made for this test), you are just telling that the authors of the best books in J2EE technology are incompetent.

    That's all.
  444. Names, e-mails, URLs[ Go to top ]

    Now it's time for you to do a full disclosure, you can't just say:

    "Several independent sources have now confirmed that The Middleware Company was indeed paid by Microsoft to conduct this report."?

    Names, e-mails, URLs will do. If you don't have them, it's just FUD.
  445. Names, e-mails, URLs[ Go to top ]

    There is also a huge difference between
    "Microsoft helped offset some of the costs" which is accurate and openly stated by TMC

    and

    "Several independent sources have now confirmed that The Middleware Company was indeed paid by Microsoft to conduct this report." which implies MS paid for the whole thing and that TMC tried to hide MS's involvement.

    Amusing that the author criticism of FUD doesn't apply to himself...

  446. Names, e-mails, URLs[ Go to top ]

    Michael Geiser

    > Amusing that the author criticism of FUD doesn't apply to himself...

    Regardless, his article (http://dreambean.com/petstore.html) has points that I sincerely hope TMC will answer. With the respect that I have for TMC, I can't believe TMC's people did as what is stated in the article without valid reasons. With the respect that I have for Rickard, I believe him on the technical points that he puts forth.

    I hope the FAQ is updated.
  447. probably they should hand over all their projects to you :)
  448. I smell FUD. Marketing ploy by a J2EE consulting firm to spread fear, uncertaintty, and denial about the complexity of J2EE developments. Stay tuned to next month where a new "TMC lessions learned/now fully optimized/and only we know what we are doing" petstore will blow the .NET version away.
  449. The engineers who conducted the test should be fired because they could not beat .Net!
    Bill Gates should be fined or sent to jail because .Net exceeded Java speed!
    Leaving the jokes apart, I think the root of the problem comes from the inception of these technologies.
    Java was not designed for the Internet but rather it was patched to do this task.
    On the other hand .Net was designed from the beginning for this task.
    It is like trying to make a supersonic plane with piston engines.
  450. Expensive Apples[ Go to top ]

    There are really two main reasons for not running the J2EE app

    >servers on Solaris. The first is because to "do it right" you
    >really should run Solaris on a SPARC box. However, that means
    >that you run into trouble because you are using two dissimilar
    >hardware platforms...
    >The second reason is more mundane? setting up a lab with
    >Solaris-SPARC machines in addition to lab of Intel boxes would
    >have taken more time and money than we had.

    You could have run Solaris8 on the same hardware(Intel boxes)-
    I am running Solaris8 on my Dell laptop.
    As for money - I think Solaris8 on Intel is free for anyone using fewer than 8 CPUs.

    Dave Hunter
  451. Here's my take on it: The MS app had the advantage of starting from scratch and building an application to implement a 'Pet Store' site completely for performance, and didn't have any 'patterns' that it had to use for purposes of education....It just hit the database in the fastest, rawest way possible.

    The TMC app, on the other hand, had a big piece of crap that it took ownership from and attempted to weed out the inefficiencies (such as the origional BMP implementation) and managed to get a 17 fold performance increase, but it still was using the same underlying architecture to implement the solution (which, I don't think the origional architecture was built for performance, rather the design lended itself to portability and 'accepted' design patterns (for maintanance gains, rather than performance gains). TMC has admitted that removing Entity beans and such would have resulted in a complete 'gutting' of the application, but at least that would have put them on the same footing as MS for this application. And it's really a shame because if they just started with a blank slate using the latest technology (and .net IS a latest technology) instead of using out-dated patterns and processes, we would have seen something better. Is it just me, or has anyone else been on a project where they inherited code from another project team, and by the time the next release was done, you realize that it would have been cheaper and faster just to rewrite it from scratch? *cough cough* Mozilla *cough cough*.

    Anyways, this is going to be very damaging to the Java community, a lot of loyal developers are ready to lynch TSS over this report....And, good or bad, they did provide a forum for Java developers.

    -Chris
  452. An interesting "first cut" comparison. Some obvious things I'd like to see, if you're going to use entity beans, most ejb 2.x compliant servers handle CMP much better than BMP, so I think that would have to happen before results could be considered useful. As to using entity beans at all, I think they are most useful in a clustered situation, all the companies I've been to using J2EE have been much more likely to use 2 2 processor boxes rather than a single 4 processor box, so I think that would be a valid comparison. And I think that using Oracle for all the databases would be mandatory, we want to make sure we're really comparing the two application server platforms, not the database.
    So, it's interesting, but hardly a real comparison.
  453. Fodder for Microsoft....[ Go to top ]

    It even made their front page: http://www.microsoft.com/

    I would suggest that in the future, TMC refrain from posting benchmarks unless they do it right. "We didn't have enough time" or "We didn't have enough money" isn't good enough. The fact of the matter is that TMC gave Microsoft ammunition to say that .net is better than J2EE.

    I would have no problem if the J2EE solution were engineered from scratch for performance, like the .net solution. But the bottom line is that TMC was too lazy to re-engineer the application and released questionable results.
  454. M$ paid TMC for this...[ Go to top ]

    It's nice to see that TMC can be bought. (14K vs 2K lines of code, but getConnection() isn't in a utility class, TX_REQUIRED,no use of Local, no caching of VO's). Way to sell out, while trying to make people think it was an "optimized" version of J2EE PetStore. Floyd, if that code is optimized, maybe you should do us a favor and stop writing Java code.
  455. Oh my god...[ Go to top ]


    Due to these newly released benchmarks, I have no choice but to go ahead and kill myself.

    TheJavaZealot
  456. Oh my god...[ Go to top ]

    What for... keep the spirit man ! You should know that the winner here are the .Net's guys who coded Pestore's version
    and The LOSER are those who coded J2ee's version NOT J2EE . If u check the source code ...Using Remote E Entity Beans, The controller almost unchanged .It looks like MS is behind all this ...hope to see this confirmed!
    Faisal
  457. Oh my god...[ Go to top ]

    I think that CSM is behind it...Are you high? IT IS NOT A CONSPIRACY...IT IS A BENCHMARK for chistssake.
  458. Oh my god...[ Go to top ]

    No I am not ...dood but who did the benchmark ? who coded the J2EE vesion of Petstore ?
    Faisal
  459. Oh my god...[ Go to top ]

    I believe that Sun themselves coded PetStore
  460. Oh my god...[ Go to top ]

    After reading this Analysis, How many people want to switch to .NET?
  461. Oh my god...[ Go to top ]

    <k k
    I would say less than 3,000 lines of code is much easier to stablize than more 14,000 lines.
    >>>
    its easy to write these 3000 lines of .NET code, What about Migrating them to new OS that MS realises every other year, What about saving them from all kinds of Visus. What was called Best few years back(the DLL's) is now called as "DLL the Hell" by MS itself. When they talk about how Stable Win 2000 is, they compare it to Win 95 and not any version Of Unix. So MS knows very well that they can be better then what they were earlier but not better then what Open Standards world has.
    MS solutions are like Pain killer, that give immediate relief but not cure the disease.

    Java/J2EE is the best. .NET(just copy of Java/J2EE) is not even close to it.

  462. Oh my god...[ Go to top ]

    Anil, when was the last time you got a 1.2 Project more advanced than "Hello World" to run under 1.4 without mods?

    How many times have you said, "What the f**k do you mean that's been deprecated? I needed that!"
  463. Oh my god...[ Go to top ]

    Do you know what you are talking about? I have gotten lots of class libraries, projects and other tools written for Java 1.1 and above to work with the latest Java versions. A week ago I ran WebLogic 5.1 (I think this version even runs under MS jview) using J2SDK 1.4.1 in Eclipse without any problems whatsoever. This is hardly a "Hello World" application.

    As far as I know Sun has deprecated a lot of methods and classes in the Java SDK but they have not removed these methods and classes, they are still sitting there for backward compatability. The deprecation just tells you that there are better ways to do things and that the API might be removed, still there is nothing stopping you from using a deprecated API.
  464. Oh my god...[ Go to top ]

    Personal attacks, Mattias, are rather lame.

    While I'm happy for your that your ONE application works in the example that you provided, did you do a full regression test with a baseline performance test?

    For example:

    http://forte.sun.com/ffj/documentation/relnote40.html#limit

    BEA WebLogic 6.1, SP2 Application Server has been certified by Forte for Java with the J2SE SDK or JRE 1.3.1 only. If you use the BEA WebLogic 6.1 server, ensure that you are using the J2SE SDK or JRE 1.3.1; do not run the IDE with the J2SE SDK 1.4 or JRE 1.4.


    Some stuff doesn't work even if it "should". It's a fact of life.
  465. Oh my god...[ Go to top ]

    When moving application from JDK 1.3.x to JDK 1.4 or higher, I have observed some problems because of XML parsers added to JDK, most of them are fixed by changing the order of jars in classpath.
  466. Oh my god...[ Go to top ]

    I've run into similar problems with other applications where library loading order is vital. I'm exchnaging my COM DLL Hell for a lower level of Hades were I have to remember to order libraries in a config file in a certain way or I throw an Exception all the way up the darn call stack...

    I'm changing my name to Jebedia and turning Amish...
  467. Oh my god...[ Go to top ]

    I am sorry if you took my question as a personal attack that was not my intention at all. It just seemed like you did not understand the way deprecation works in Java.
  468. Oh my god...[ Go to top ]

    How many times have you said, "What the f**k do you mean that's been deprecated? I needed that!"


    Do you ever read JavaDocs properly? Every deprecated method explains what alternate method to be used.
  469. Oh my god...[ Go to top ]

    "MS solutions are like Pain killer, that give immediate relief but not cure the disease."

    And the disease is called EJBitis :-)

    ".NET(just copy of Java/J2EE) is not even close to it. "

    Luckily enough, if it were any closer I would have to write five times as much code, would need a bunch of gurus to figure out what options to use (Struts, Velocity, CMP, BMP, JDO, XA, ...), and I would still be twice or thrice slower :-)
  470. Oh my god...[ Go to top ]

    This reminds me of My Son saying "Don't give Options, just tell me what to do"
  471. MS is open source[ Go to top ]

    MS is open source:

    http://mavnet.sourceforge.net/

    http://www.go-mono.com/


    .V
  472. MS is open source[ Go to top ]

    Actually not completely true.

    The ECMA spec for .NET covers only the base libraries of .NET. It's API little more functional than POSIX. You can do a lot of things in POSIX C, but it's still pretty limited when it comes to Web and GUI applications. Important APIs like ADO.NET, ASP.NET, WinForms, and remoting are not part of the ECMA Spec.

    The Mono team has committed to implementing ADO.NET (using GNOME's Database API) and a minimal form of ASP.NET for Apache (it seems like performance isn't a major issue yet and I'm not sure if full compatibility is the goal either), but for remoting, Mono is forgoing the COM interface in favour of CORBA. Also, WinForms is something that the Mono leads have given up on since WinForms is tied to Win32. According to the SharpDevelop (see http://sourceforge.net/projects/sharpdevelop) developers, it's very hard to do anything nontrivial without adding Win32 calls to you app. According to these and other developers on the Mono list, even WinForm examples on Microsoft's website include these Win32 hacks. Instead of using WinForms, Mono is committed to implementing a language binding for the Gtk+ GUI toolkit and others are working on a Qt GUI language binding. So if you want to do portable GUI programming in .NET, you have to go to a non-Microsoft GUI widget set. Some Mono developers are planning on porting WinForms to WINE (a windows emulation library for Unix-like systems), but this is basically a hack. This implementation of WinForms will always be playing catchup to the Microsoft implementation.

    Let's not forget that Microsoft is planning on updating the .NET VM and the C# language to include capabilities for generic and functional programming (see http://research.microsoft.com/projects/clrgen/). These capabilities are not part of the ECMA spec and Microsoft has not yet said that it will send these specs to the ECMA. If it does, it's unknown when. In the mean time, it's free to include these capabilities in it's own .NET implementation and all other clones like Mono will either have to reverse engineer Microsoft's implementation or accept the incompatibility.

    So basically, portable nontrivial .NET programming is not a practical reality any time soon.

    Mono might succeed within the context of helping you develop .NET applications for GNOME (see http://developer.gnome.org/news/summary/2002_October20-October26.html#15 for an example of the latest progress), but it won't help build portable non-GNOME applications in the forseable future.
  473. Oh my god...[ Go to top ]

    I would switch if .Net was platform independent.
  474. Oh my god...[ Go to top ]

    But then it may not give same performance
  475. Oh my god...[ Go to top ]

    "I would switch if .Net was platform independent. "

    But guys, that's the whole point. To me, .net isn't portable, END of discussion. You're basiaclly bought and paid for by a single company. sure, assembly language is faster than C++, because it knows that particular hardware like nothing else. So what? Who codes in assembly language? and if performance is that crucial, use some JNI in certain areas.

    I do agree that EJB's are pretty slow. But again, machines are getting faster all the time.

    Let's take the .ear file in this comparison and put it on a mainframe interfacing with db2 and see how it compares then.

    OH, wait... the .net wouldn't run.
  476. Oh my god...Platform (in)dependance[ Go to top ]

    If you have a Enterprise app and you are running it in an App server such as WebLogic or WebSphere, there is sooooooo much server specific implementation in the server setup and deployment descriptors that cost of switching is prohibitive.

    If the EJBs are not performant throwing more hardware at the problem will produce deminishing returns until the system grinds to a halt from cluster or other overhead.

    I betcha $1 that .ear won't run on the mainframe w/o mods.
  477. Oh my god...Platform (in)dependance[ Go to top ]

    We have a nice article and example on How a application can be ported on two different App Server available on this site.
    Also its about the programing practice. If you want you can have same .ear run on diff appserver, I think PetStore is an example of that.
  478. Oh my god...Platform (in)dependance[ Go to top ]

    We have a nice article and example on How a application can be ported on two different App Server available on this site.
    Also its about the programing practice. If you want you can have same .ear run on diff appserver, I think PetStore is an example of that.
  479. Oh my god...Platform (in)dependance[ Go to top ]

    ...cost of switching is prohibitive.


    FWIW: Together is pretty good at allowing you to switch EJBs to different app server vendors. For example, take in a WAS and turn out a WLS set of deployment descriptors.
  480. Platform independence[ Go to top ]

    J2EE may be slower, require more code and be harder to maintain, but at least we have platform independence.
    Why not rally around our strong point?
    Performance, development time, maintenance and cost are stupid, anyway.
  481. Platform independence[ Go to top ]

    J2EE may be slower,

    EJBs are part of J2EE.

    >require more code
    Maybe 3rd party code.

    > and be harder to maintain,
    Maybe not. Try doing distibuted .Net with VS.Net.
  482. Oh my god...[ Go to top ]

    hey...java is "platform independent" because there is (or at least its assumed) a JVM on every platform to run the javabyte codes....WEll hear this, .Net is platform "independent" as well because it's planned that the runtime (.Net Framework)will be on every platform to run the IL codes.
    And let me remind you that there is .Net framework for Unix, Linux and the redmond boys (with others) are vogorously coding for everythingplatform.Net!!!! scary huh!!!

    Well you haven't heard anything yet... now there is everyprogramminglanguage.NET (fortran.NET,pascal.Net C#.NET, J# (really java).Net, perl.net, c++.net,) ...in short .Net means language independent as well my friend...Now that is one hell of a liberation for all!!...

    I hate to admit the brilliance of the redmond boys but I put the blame squarely on the heads on McNeally and his crew for their short sightedness. they should have listened more!!

    If history is our best teacher...Sun's java days are as numbered as wordperfect or lotus 123.
  483. Oh my god...[ Go to top ]

    **********************************************
    Well you haven't heard anything yet... now there is everyprogramminglanguage.NET (fortran.NET,pascal.Net C#.NET, J# (really java).Net, perl.net, c++.net,) ...in short .Net means language independent as well my friend...Now that is one hell of a liberation for all!!...
    *************************************************
    If Programing language means just the syntex then the above statement may be true.
    If language independence is libration then, May god save people fighing for such cause.
  484. Oh my god...[ Go to top ]

    ****************************************************
    I hate to admit the brilliance of the redmond boys but I put the blame squarely on the heads on McNeally and his crew for their short sightedness. they should have listened more!!
    If history is our best teacher...Sun's java days are as numbered as wordperfect or lotus 123.
    ****************************************************
    Those were the different days when MS killed wordperfect or lotus 123. Now people are more people know what they want. Situation is very similar to of British rule in middle of 20th century.
  485. Oh my god...[ Go to top ]

    k k


    Have you read the post by Robert Devi?

    hehe, your post is LOL. nice try.
  486. Oh my god...[ Go to top ]

    If this comparision is not just with the programming/design practice promoted by MS and Sun. Can TSS/TMC give estimates on Numbers if the Petsore was rewritten using their JADE or whatever framework.
    I think then we can get better numbers, also get to know how's TMC "The most advanced J2EE education service"
  487. Oh my god...[ Go to top ]

    "And let me remind you that there is .Net framework for Unix, Linux and the redmond boys (with others) are vogorously coding for everythingplatform.Net!!!! scary huh!!! "

    To bad that from the 2400 usefull classes in the .net framework only 180 will be ported. So yes we will have .Net on UNIX but it will have only one wheel.
  488. Oh my god...[ Go to top ]

    Bill may be buyong the bullets, but Scott is loading the gun, putting it in your hand, and pulling thr trigger.


    Was that over the top? I can't tell sometimes...
  489. Probably they should have put "Prepared for Gates, The Emperor" in the first page, like the one they did it for Sun http://www.sun.com/aboutsun/media/presskits/gov/J2EE-vs-DotNET.pdf
  490. Humility[ Go to top ]

    Can somebody please tell me how an application developed by TMC can be THE reference for J2EE performance? How humble is that?

    Can I have that answer for free or do I have to pay TMC for the consulting?
  491. Humility[ Go to top ]

    The application was developed by Sun as a design blueprint. You're right, Scott's not very humble, is he?
  492. Humility[ Go to top ]

    Michael Geiser wrote on October 30, 2002
    > The application was developed by Sun as a design blueprint.
    > You're right, Scott's not very humble, is he?

    You MSFT guys will never understand that, won't you? NOBODY owns J2EE. In this world your ownership is as important as the power of your code. You cannot fool the people so easily with the shiny teeth of your marketing/commercial team.
  493. Yagiz,

    <Q>You cannot fool the people so easily with the shiny teeth of your marketing/commercial team.</Q>

    What about the Oracle marketing/commercial team?

    "We at Oracle have redisigned the Petstore application so that it is now 10 times as fast as .NET."

    The Middleware Company has improved the Original Petshop by a factor of 17. That means that MS was correct in their original claim that .NET was 28 times faster. Do you still believe Oracle?

    A team of J2EE specialist from a company which have staked there whole business idea and survival on J2EE, is a successful teacher of J2EE, and has written some of the best books on J2EE made their best shot and come up with this poor results. The Java guys, realizing that it is to idiotic to claim that The Middlewhere Company is incompetent now have raised conspiracy theories.

    Don't they forget something? Tearing out your own heart?

    If The Middlewhere Company have decided to move away from J2EE to also cover .NET that speaks mileage in my IMO and is more disastrous to Java than any test results could possible be.

    In the Java body Jakarta is the brain, Middlewhere/TheServerside is the heart and Sun can be compared to the appendix.


    Regards
    Rolf Tollerud
  494. Rolf,

    <Q>You cannot fool the people so easily with the shiny teeth of your marketing/commercial team.</Q>

    This quote goes with the previous one. I meant that we are a communicating community. I, personnaly (and I don't think that I'm alone in that), don't believe the benchmarks; I believe experiences. And because you can never experience everything on your own, you communicate with the other in order to learn from their experience. I think, in our context, which make one experience better than 1000 benchmarks is its real-life aspect. If there were one of us in every commercial meeting/decision, asking intelligent questions, decision takers wouldn't be so easyly fooled.

    <quote>
    What about the Oracle marketing/commercial team?
    </quote>

    They are exactly the same. By definition, a company aims to increase its benefits. And I guess we all know that any commercial trick is good to get one more cent. That's exactly where an open standard like J2EE beats .NET. J2EE is above the companies. I know it may not be as ideal as it sounds however it's lot more open than .NET. If you want to escape your J2EE provider, you can with significantly lesser effort compared to .NET.

    <quote>
    A team of J2EE specialist from a company which have staked there whole business idea and survival on J2EE, is a successful teacher of J2EE, and has written some of the best books on J2EE made their best shot and come up with this poor results. The Java guys, realizing that it is to idiotic to claim that The Middlewhere Company is incompetent now have raised conspiracy theories.
    </quote>

    I don't agree on that. We all had teachers/professors at college that didn't have a clue about the real life situations, didn't we? And nobody knows everything. If they are as professional/knowledgeable as you claim they are, how can people find remarkable areas lacking optimization just few hours after they published the report? If they are as professional as you claim they are, how come we have so many question starting with "WHY?" in all these messages?

    <quote>
    In the Java body Jakarta is the brain, Middlewhere/TheServerside is the heart and Sun can be compared to the appendix.
    </quote>

    How can you believe that a consulting company and its web site form the heart in this body. The heart consists of the thousands of members of the J2EE community. TSS itself is just a mean of communication. If you move TSS members to a different place, TSS becomes just a plain web site. TSS is a community meeting point allowing me to send you this message and not more than that...

    Regards,
  495. Yagiz,

    I can not understand how you can take down TSS as easily as this. There are thousands of forums on the web but they are all more or less like Slashdot (sometimes I think that there is not possible that it exist a lower life form than Slasdot members).

    On the contrary, TSS members are for the most part consultants, a very different kind of animal. I have learned very much from this discussions and have been entertained too. When I have to work with Java I get all help I need from prople with more experience than myself.

    If there is any truth in the speculations that TMC should extend there middlewhere competence to .NET technology nobody can be more happy than me. Before I could perhaps think that they was not 100% objective but now I know that they publish the truth even if the hurts themselves.

    Integrity is the most rare thing on earth.

    Regards
    Rolf Tollerud
  496. Rolf: "Integrity is the most rare thing on earth."

    Followed closely by irony.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  497. Floyd, The Middleware Company (TMC) and TheServerSide (TSS) for the purpose of this discussion are the same.

    Everybody is going on and on about the results and metrics from the J2EE vs. .NET test, THE TEST IS INVALID - the reality is TSS has established an agenda for their own "selfish" promotion.

    It is not about publishing results from an evaluation, it is about taking part in deception to mislead the public.

    This is the problem I have with TSS now.

    Of course, damage control is taking place now, but too late - TSS has lost credibility.

    I don't think Floyd was personally involved; my guess is the Precise (TSS parent co.) people trying to exploit this for obvious reasons.
  498. <quotes>
    There are thousands of forums on the web but they are all more or less like Slashdot.
    </quotes>

    EXACTLY!!! What's the difference between TSS and one of those forums? The answer is: MEMBERS. The "developer community" is the key concept here. But, please don't get me wrong. I still think that TSS does a great job to KEEP us together. They prepare a great environment. But this doesn't make the 5-people-TSS-team neither the king of this realm nor the heart of Java.

    BTW, I certainly believe Floyd Marinescu when he says that TheServerSide and its team was not involved with the benchmark.
  499. Free Your Mind[ Go to top ]

    I'm really curious where all this is going? With all that hidden marketing moves of the industry, nothing is certain I guess :-)...

    :-) This reminds me of a guy called Morpheus who once said:
    - "Do you believe that my being stronger or faster has anything to do with my muscles in this place?" ... "Free your mind."
  500. Is there anyway Floyd and company could break away from TMC and be independent? How much would it take from our community to support this?
  501. Is there anyway Floyd and company could break

    > away from TMC and be independent?

    ROFL!

    Watch Rickard's space:

    "It seems (and I haven't confirmed this just yet) that this was a very well orchestrated "hit" by Microsoft, who was aware of the results days before it was released, and had plans to use it in communications with analyst groups as soon as the report was released (Gartner, Meta, etc.). There are also some interesting details with regard to who owns TMC currently... "

    He told also that he has an internal M$ mail.. Locking forward!
  502. Guy,
    lets say u r given with an amount (Consider minimal amount) of budget, what MS can offer ?

    I use the same application used in the benchmark, but i change the platform which i think best suit for j2ee as follow:

    - jboss latest version
    - multiprocessor Intel platform running Linux Redhat Latest version
    - Investment over time and IT physical environment (wireless, broadband) over time
  503. Humility[ Go to top ]

    ahhh, actualy, $un does own Java and J2EE. Remeber they sued MS over Licensing agreement violations. That says OWNER or they would have no grounds for the suit, now would they?

    BTW: I'm not a MSFT "troll". The team that I architect for does J2EE on App Server A...errr...WebLogic 6.1.


  504. I for one won't be using an MiddleWare Company training!

    They missed such obvious optimizations! They should be very emabarassed!
  505. I don't know James; TMC did a number of subtle and effective optimizations... feel free to add to the Sourceforge project at any time, OK? Thanks..


    "If you're not part of the solution, you're part of the precipitate."
  506. The tilted report produced by TSS was bound to happen. M$ is already in bed with BEA and it’s only in Microsoft’s best interest to fly over TSS for a threesome. Seems TSS was inspired to implement J2EE at a disadvantage. I expect to see headlines such as:


    The Enquiry

    TSS GETS A HICKEY FROM MICRSOFT!



    *** Read all about it ***
  507. Report copyrighted???[ Go to top ]

    How can M.S. make use of portions of the report?
    Is the report not copyrighted by the Muddleware Company?
    - or does M.S. own it?


    I see snippets at www.microsoft.com -
    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/psimp.asp
  508. I may be cynical, but why do I have a hard time believing that TSS went completely offine for maintenance at 2:45pm?

    I wonder what went bad?
  509. I may be cynical, but why do I have a hard time believing

    >that TSS went completely offine for maintenance at 2:45pm?

    It was also offline yesterday (10/29) around 5:30pm. It's because Java is so robust.
  510. ouch!!!

    we can reach you at msarbu at microsoft dot com....

    THAT'S A JOKE FOR THE HUMOR IMPARED...
  511. Posted By: Michael Geiser on October 30, 2002 @ 04:42 PM


    >THAT'S A JOKE FOR THE HUMOR IMPARED...

    Michael, I did not atack you personally, and you don't have to yell :-)

    Check their uptime for a clue.
  512. If I am not wrong MS Hotmail does not run on NT/2000/XP
  513. Posted By: Anil Patel on October 30, 2002 @ 04:47 PM in

    >response to Message #63357.
    >If I am not wrong MS Hotmail does not run on NT/2000/XP

    You are wrong. Check netcraft.
  514. As Envel Le Meur said
                   DON"T PANIC !!!

    Read what Rickard Oberg comments
    "Several independent sources have now confirmed that The Middleware Company was indeed paid by Microsoft to conduct this report."

    Go to http://dreambean.com/petstore.html
  515. I was just about to sign up for the TSS J2EE architect course, but will not pursue because of this invalid J2EE vs. .NET test.

    This test proves nothing and just creates FUD; I hope Microsoft doesn't exploit this misinformation.

    I have nothing against .NET; in fact, it is great to see Microsoft finally making good software for a change.

    TSS should just stick to technical facts and document real life J2EE case studies.

    I don't find TSS credible anymore, even when Floyd is saying TSS is just the messenger.
  516. Inertesting MS uses BSD webserver for marketing ....
  517. Were J2EE Threads FORKed?[ Go to top ]

    As of Redhat 7.1 with jdk 1.3.1, the Linux jvm defaulted to forking a process for each java thread. Obviously not a high performance solution. I'd be curious whether this was the situation with TMC's Linux test.

    If so, this makes it even more plain that an apples and apples test would have been to run both the J2EE and .NET petstores with identical OS. As it stands, there are too many variables for this test to be a useful technology comparison.
  518. Were J2EE Threads FORKed?[ Go to top ]

    [quote]an apples and apples test would have been to run both the J2EE and .NET petstores with identical OS.[/quote]

    Can everyone please read the pdf or the faq? I don't know why I keep seeing this misconception. They tried both Java App Servers on Windows 2000 AND Linux.

    For App Server A, they got better performance on Windows, so they used those results.

    For App Server B, they got the same performance on Windows and Linux, but they used Windows because of the built-in performance monitor.


    In terms of Solaris on x86 - is anyone serious? Solaris is not optimized for x86 *anywhere* close to where Linux is. Do you really want to see those results?


    And in terms of involvement from Bea and IBM, they asked both vendors for help but were rejected. Is that due to short supply of resources OR the fact that they knew they were going to get creamed?
  519. Were J2EE Threads FORKed?[ Go to top ]

    Another point about Entity beans (CMP in particular)...

    They are good for a variety of things, raw performance isn't often one of them. Good points are: Database portability, data/object modelling, reuse, maintenance, and SCALABILITY.
    If you can cache some of the data at least in the middle tier you win out when you start to scale up/out (by reducing hits on db).

    I've got not idea what sort of caching was used for .NET, and the commit options/caching weren't revealed for J2EE version either.

    Some J2EE caching/clustering vendors have done these sorts of tests before of course, one that springs to mind is this one which recoded Nile benchmark to use Entity beans and got it run 10 times faster than SLSB version:

    http://www.persistence.com/assets/pdf/pt-extreme_ejb.pdf

    By my loose calculations this would make it faster than the published .NET results at:

    http://www.gotdotnet.com/team/compare/nileperf.aspx

    Paul

    PS Regarding complexity of changing PetStore to use SLSBs only, looking at the code I see they've used DAOs anyway - so it should be trivial to change the actual persistence mechanism (all the SQL is in helper files anyway) - right?!
  520. Were J2EE Threads FORKed?[ Go to top ]

    linux makes no distinction between a process and a thread. there are heavyweight tasks (processes) which have their own memory space and data structures, and there are lightweight tasks (threads) which share another task's memory space and data structures. if you run top while a multi-threaded java app is running you will apparently see a lot of process forking, but that is mis-leading.
  521. Is it true that The Middleware Company has to sell an excessive amount of services before the end of the year in order for the primary stock holders to get their earn-out bonuses? http://austin.bizjournals.com/austin/stories/2002/08/05/story3.html

    Is it also true that consulting deal with Microsoft put The Middleware Company significantly closer to earning those dollars?

    Lastly, why is Floyd acting like there is no relationship between The Middleware Company and it's internal portal, TheServerSide?
  522. Wow, Middleware was sold in August at $10K:

    http://austin.bizjournals.com/austin/stories/2002/08/05/story3.html

    Including TheServerSide.com, of course. So this site is owned by Precise.

    > Lastly, why is Floyd acting like there is no
    > relationship between The Middleware Company and
    > it's internal portal, TheServerSide?

    I think they made a good deal with M$. They've already admitted to got all their expenses, travel costs and a full-featured lab from M$. So in fact, MICROSOFT BOUGHT THIS BENCHMARK!

    I guess this is only the tip of the iceberg, you know. There's a deal running in the background where this f*cking benchmark is only a part of.
  523. From EDGAR Online:

    <snip>
    NOTE 3: ACQUISITION

    In June 2002, the Company acquired all the outstanding shares of The Middleware Company, a United States based services company, in consideration of approximately $6.0 million. The company provides consulting, training courses and other related services. The purchase price consisted of less than $4.0 million paid in cash. In addition, the stockholders are entitled to up to an additional $2.0 million of the Company's ordinary shares to be issued in the fourth quarter of 2002. Additional consideration of $150,000 in estimated transaction costs is included in the total purchase price. The stockholders also have the right to receive additional contingent consideration including consideration of 20% of revenue for achieving a set revenue minimum and consideration equal to the amount by which revenues exceed another set minimum revenue target up to a set maximum relating to the one year period subsequent to the closing.
    </snip>
  524. Partner of the company that purchased TMC:

    http://www.precise.com/Partners/Strategic/index.cfm/FuseAction/Detail/ID/151/
  525. Lastly, why is Floyd acting like there is no relationship between The Middleware Company and it's internal portal, TheServerSide?


    Guys, of course there is a relationship between TMC and TSS. TMC owns TheServerSide. However, my and my team are pretty independent of TMC's training/consulting business.

    All I am saying is (for the 5th time), that TheServerSide and its team was not involved with the benchmark beyond simply announcing it.

    I am saddened that people on this site are blaming me personally and TheServerSide for something that we had nothing to do with. I think TheServerSide has conducted itself with a lot of integrity and we've always done a good job being 'your j2ee community'.

    Blaming TSS for this is equivalent to blaming the son for his fathers actions. Last i checked, we do not live in the Klingon empire - even Gowron and Duras would frown at this behaviour...
  526. Are all loadrunner settings are same for all the scripts used for testing???

    I downloaded loadrunner scripts and noticed that .NET webapplication benckmark scenario2_browse has different log and think time settings(standard log, ignore think time) where as other scripts have disable log and 50%-150% think time.
  527. I think we need more data before giving up on J2EE for .Net. I think a lot of people have questions that basically reduce to the following:

    a) Why was the J2EE so slow? I think someone should run the benchmark again using OptimizeIt or JProbe and determine where the problems spots are. (Maybe TSS can get Borland and Sitraka to fund it so they use the data to justify the need for their products.)

    b) Why does it take so much more code to build the J2EE solution? Again, I think someone should compare the code to see what .Net provides that J2EE doesn't.

    I think TSS should host a contest to see who can build a java pet store implementation that is as fast or faster than .Net's version. I say we put up $20 each and the winner gets the pot.

    Actions speak louder than words.
  528. Although i agree with most of members on TSS that comparing J2EE with .NET in terms performance was always going to give the results that have been achieved by the report produced by TMC.
    However looking at the readings one cannot ignore the huge difference in terms of performance between the two plateforms. Although it remains to be seen what will be the results if the vendors are themselves allowed to configure the servers (just as MS person configured theirs) and secondly if SPARC machines are used for J2EE.
    One also wonders, that how could the "gurus" of TMC, not predict the results before going through this exercise, and if they had predicted the results, it is very strange for a company which supports/consults/offer courses on J2EE to do such a comparison. (some thing fishy here..! )
  529. The TMC benchmark results are completelly diferent from the results i've found in a March 2002 Oracle white paper called Oracle9iAS vs. Microsoft .NET Benchmark Results:
     
    "Based upon these results, it is evident that Oracle9iAS without Web Cache delivers up to 18 times better performance than Microsoft .NET without caching. Oracle9iAS with Web Cache was able to beat Microsoft .NET with caching by a factor of 22. An interesting observation is that Oracle9iAS without the Web Cache was up to 7 times faster than .NET with caching. Oracle9iAS also scales better by utilizing CPU more efficiently than .NET. In summary, Oracle9iAS and J2EE demonstrate far superior performance when compared to Microsoft .NET."

    Any comments about this?
    Regards,
    Pedro Candido
  530. Oracle J2EE 22X faster than .NET ...[ Go to top ]

    The problem with that benchmark was that Oracle refused to release any of the stuff you would need to get it working (scripts, actual code they changed, etc). So, it was impossible for someone to actually independently verify the results. Plus, it just seemed to be unbelievable. Regardless of whether you believe Oracle or .NET would be faster, Oracle was essentially saying they made a few changes and were not only roughly 20 times faster than the .NET solution, but were now hundreds of times faster than their own prior solution (if .NET was, I think, 28 times faster, and they are 20 times faster than that, that's a pretty big performance jump).
  531. [Quote]Oracle white paper called Oracle9iAS vs. Microsoft .NET Benchmark Results: [/Quote]

    Simply not true as verified in this report from Veritest - in which Oracle declined to participate in the audit of their own test -

    http://www.gotdotnet.com/team/compare/veritest.aspx

    Benchmarking is a serious business - and to make any a success - regardless if it is an internal benchmark for intranet / internet applications or a technology shoot-out - which is the case with the TMC benchmark, a tremendous amount of effort, equipment, and resources have to be allocated. IMHO - TMC did their homework and implemented the proper methodology.

    1. Hardware was identical between both .NET and APP SRV A and B.

    2. The OS - was identical between the three flavors tested. While it may be argued that APP SRV A or B should have used brand X of an OS - it was documented that Linux was used and simply did not give the performance gains. So even on that ground - again consistency and fair playing field. Illustrates again that TMC evaluated options before committing to a specific platform.

    3. Database: There have been several comments in terms of the use of Oracle by APP SRV A & B vs. .NET using SQL. A majority of individuals stated that it was not fair because of course .NET would be faster on SQL than Oracle. Now, pose the same viewpoint and why wasn't APP SRV A & B used against a SQL Database? Could it be that the database connectors for both would have resulted in poorer performance numbers? Ideally it shouldn't matter which Database was actually used - however, for the specific Application Servers Used - Oracle was the "best case" to eliminate the database as being the bottleneck during the testing (better DB Connectors) - likewise SQL was picked for the .NET implementation. APP SRV A & B could have used SQL - and if those results had been published - foul ball would have defintitely been cried. Yet, with all the talk about cross platform and portability - but wait - the test was not designed to test PORTABILITY or CROSS PLATFORM capabilities - it was designed to actually test APP SRV A & B performance and load handling characteristics while scaling processors specifically. It would be nice to see both APP SRV A & B tested against SQL and .NET against Oracle as well...

    4. Funding. I have spent well over 7 years doing nothing but perf testing for a variety of organizations. Every Benchmark that is ever published comparing technologies is funded. From my gatherings of the lab diagram - take for instance just the network equipment alone - $50,000 to start (gigbit networking is not cheap), now factor in the client machines, the servers - $400K to get a lab set-up that can conduct a test of that scale. To have a lab designed that eliminates the concerns that the clients used for load gen or the network would infuse artificial bottlenecks in the test - was a great service to TMC for this benchmark. You simply could not achieve a load test of that scale with 5 or 25 client machines - and walk away at the end of the day knowing you had valid results. In terms of compensation - SUN paid TMC for the work done on the SUN Whitepaper - was foul cried there? I do not know of ANY business who would just give up tens of thousands of dollars for the mere sake of conducting a test. Heck, most corporations balk at spending monies for their own QA teams to have a reasonable lab to conduct testing.


    5. Petshop is not a valid Benchmark Application?

    What exactly makes it an invalid Benchmark? All the hype here is about how it was originally written to showcase "best practices" and not performance. That excuse may have been valid LAST YEAR when Microsoft and Oracle went head to head - but the J2EE version was RE-Coded - perhaps not in whole but a majority of it in parts (and a perf IMPROVEMENT made by TMC on the magnitude of 16X better performance). ANY APPLICATION BENCHMARK, including the TPC-W etc... all allow for the use of available technology to implement a solution whose end result meets a DEFINED SPECIFICATION. The Petshop application - was designed to meet specific criteria, such as being able to place an order and never lose a Transaction, specifically the use of distributed transactions, meet n-tier development requirements, and so forth. All appservers had to design the application to ADHERE to those TMC established benchmark criterias.

    The Petshop application is very much like any other application that has been developed and in use at 1,000's of businesses. How is it different than that of something your company has programmed? The only real difference is that in reality - the object of the test was to demonstrate reliability and performance of the solutions designed to meet the criteria agreed upon for the testing of a Petshop application on all