The Middleware Company creates a framework for case studies

Discussions

News: The Middleware Company creates a framework for case studies

  1. The Middleware Company, with the help of industry experts, has been working on a functional and behavioral specification of an application that can be used in technology case studies running in diverse application server environments, including J2EE and .NET. TMC would like to use the framework for another performance casestudy, as well as ones for interoperability and productivity.

    As a reminder to our readers, TheServerSide.com (a media business) is an independent business unit of The Middleware Company (a technology consulting company) and does not have any special knowledge of, involvement in, or influence over this effort.

    After reading the eWeek article we asked TMC for more information about this specification and learned that they would like to use this case study framework for at least 3 casestudies: a performance casestudy (as mentioned in the eWeek article), a productivity case study which documents and compares the experiences of multiple teams using different approaches to implementing this specification, and an interoperability case study, in which a .NET implementation of the specification interoperates with a J2EE implementation of the specification.

    Read:
    Benchmarks Wanted.
    The Middleware Company Case Studies page.

    Threaded Messages (50)

  2. Open source?[ Go to top ]

    Is this "framework" going to be open source? If not, you might as well forget about it.

    As a reminder to readers of this message, it is independent of the author and does not have any special meaning, other than to point out that TMC and it's media mouthpiece TSS have limited credibility in this regard and that they should maintain the highest transparency possible.
  3. Open source?[ Go to top ]

    Is this "framework" going to be open source? If not, you might as well forget about it.

    John, In short, yes. We have not taken the time to write up a "license" yet, but basically we have put everything we have on the case study website. Feel free to download it and use it for your own case studies, and post the results. Please send suggestions for improvement to the email address on the website.

    Regarding the media mouthpiece and credibility - you are right - and we are trying our best to do what you suggest.

    Salil Deshpande
    The Middleware Company

    ****P.S. I may not have time to read and answer all the questions that come up here this week, but we will catch up at some point and may answer them in a FAQ on the case study website.
  4. Case study?[ Go to top ]

    You are calling them case studies, not benchmarks?
  5. Case study?[ Go to top ]

    Yes, for a few reasons.

    The term "benchmark" evokes emotional reactions from people.

    It implies something more formal than what we have (with committees and voting etc.) and what we have with our expert group (and now public) process is less formal.

    Also, "benchmark" implies that the only things that are important are the results, specifically performance results. That's an important data point, but things such as issues debated by the expert group, what's in or not in the spec that's supposed to represent typical customer requirements, documentation of experiences and difficulties encountered, and other things like that can be more interesting and meaningful, IMO.

    Please also note that this is just raw material for case studies... a performance case study is just one of the many case studies possible.

    Salil Deshpande
    The Middleware Company
  6. Case study?[ Go to top ]

    Makes sense thanx. Keep up the good work.
  7. Open source?[ Go to top ]

    imho, i think you shoudn't lose your time writing a licence: GPL exists and is perfect for such piece of code.
  8. Interesting way to handle the whole situation. Looks like a solid effort.

    Toshi
  9. Re-inventing Wheel?[ Go to top ]

    Hi All,

    why didn't you join the effort of ObjectWeb (http://www.objectweb.org)? They already have JMOB - Java Middleware Open Benchmarking (http://jmob.objectweb.org) with RUBiS (http://rubis.objectweb.org). Several versions of RUBiS are implemented using three different technologies: PHP, Java servlets and EJB (Enterprise Java Bean). I think .NET version to add is just a short step. And everything is Open Source.

    Just a thought...

    Lofi Dewanto
    http://openuss.sourceforge.net
  10. Re-inventing Wheel?[ Go to top ]

    You very silly.

    ObjectWeb, of course, are the people who make the JOnAS application server, and they are showcasing RUBiS as one particular benchmark on which JOnAS happened to perform well.

    It doesn't inspire confidence in the objectivity of the benchmark to use software supplied by one of the vendors. Would people trust TSS if the benchmarking software for this analysis was written by BEA?

    Or what if it was written by Microsoft?

    Oh, wait, TSS already did that...
  11. ObjectWeb, of course, are the people who make the JOnAS application server, and they are showcasing RUBiS as one particular benchmark on which JOnAS happened to perform well. It doesn't inspire confidence in the objectivity of the benchmark to use software supplied by one of the vendors.

    Corby,

    Good point. So, the reason we took the approach of creating a 60 page functional spec is so that anyone (including BEA or Microsoft) could submit an implementation that conforms to the spec.

    Secondly, this spec is not just for performance benchmarking.

    Oh, wait, TSS already did that...

    :-) Your jab is well taken, but Floyd would want me to point out that it was not the good guys at TSS, but TMC (the evil parent company of TSS) that did that.

    So anyway, we might do "that" again - we just published our functional spec up front this time.

    The spec has been reviewed by a number of invited industry experts (see title page of the spec) who generously contributed their time to help get it to a stable state. Currently unincorporated feedback from the experts is also available in a separate document at the case study website.

    Salil Deshpande
    The Middleware Company
  12. Reuse[ Go to top ]

    Corby,

    <quote>
    ObjectWeb, of course, are the people who make the JOnAS application server, and they are showcasing RUBiS as one particular benchmark on which JOnAS happened to perform well.
    </quote>

    this is not about the result itself. This is all about the specification and at the end the "implementation" of a benchmark. What I meant is reusing the idea, specification and implementation (but not the optimization and result of the benchmark). You can check out the specification of Rubis at their site.

    Anyway, just a thought ;-)

    Lofi.
  13. Re-inventing Wheel?[ Go to top ]

    Corby,

    I am one of the author of the RUBiS benchmark. You are completely wrong when you say that RUBiS was designed by people who make the JOnAS application server.
    RUBiS was developed independently at Rice University. When we all left Rice, ObjectWeb kindly hosted this effort. Note that ObjectWeb is not JOnAS and that JOnAS is not ObjectWeb too.
    So, before being so arrogant and insulting people in threads, you'd better check twice what you are going to say if you don't want to be ridiculous again.
    I suggest that you apologize publicly on this thread.

    Emmanuel Cecchet
  14. Re-inventing Wheel?[ Go to top ]

    You are completely wrong when you say that RUBiS was designed by people who make the JOnAS application server.


    Emmanuel,

      I'm sorry that you are upset, but I never said any such thing. Please re-read my comment.

      I said that ObjectWeb is SHOWCASING the excellent work you did with the RUBiS benchmark. Or, as you put it, they are hosting the effort.

      RUBiS is an excellent piece of research that I followed when it was first publically released, but let's please be frank about this. ObjectWeb is eager to host it because it shows favorable numbers for the JONaS project and the Jeremie protocol.

      If BEA has said, "Here is a pre-existing benchmark where previous versions of our app server outperformed our competitors. I suggest that TMC bases their benchmark on our codebase." I would be skeptical of their motivations, and I would suggest that using this code would impugn the objectivity of the study. I see this as being more realistic than arrogant.
  15. Re-inventing Wheel?[ Go to top ]

    why didn't you join the effort of ObjectWeb?

    Hi Lofi,

    Yes, there's some reinventing here, and it would have been good to avoid producing yet another spec.

    Some discussion of why we did not go with SPECjAppserver and TPC-W is in our functional spec.

    There were other reasons to start with PetStore and formalize and tighten it up, one of them being that it is so familiar to J2EE developers. So there's more plentiful domain knowledge of that application, more sample implementations on various technologies (including .NET), and more implementors, so that increases our chances of getting third party submissions, which we all just have to make sure are conformant. Even for new implementors, the PetStore functionality is better known than CSIRO or RUBiS, so that should reduce the learning curve.

    Also, when we started we wanted to incorporate most of the feedback we received from the community to our last study, which we could not have done in an easily demonstrable fashion if we had completely switched application domains.

    I'm not an expert on RUBiS, so correct me if I'm wrong, but their approach seems to be different -- does a functional specification exist (I could not find one), or just the PHP, Servlet, and EJB implementations? (I downloaded RUBiS 1.4).

    Salil Deshpande
    The Middleware Company
  16. Here we go again[ Go to top ]

    Why is the serverside being used as an AstroTurf ground for satisfying TMC insatiable appetite for polishing bill gate's knob?


    Why would anyone want the same scheming dimwits to write a spec for benchmarking among other things? Shouldn't TMC be demoted to sh*tbird activities? Especially, when this very same company is owned by Precise with no J2EE partners and Microsoft as a strategic partner.http://www.precise.com/Partners/Strategic/

    I can readily see a conflict of interest. TMC, do yourself a favor and stick to dotnet consulting or weren't you able to find any open dotnet consulting positions?


    Floyd you wrote:
    "As a reminder to our readers, TheServerSide.com (a media business) is an independent business unit of The Middleware Company (a technology consulting company) and does not have any special knowledge of, involvement in, or influence over this effort" -floyd

    If this is so, why is dion almaer who is employed by TMC deleting posts on the serverside?
     

    The Server Side should stop distancing itself from The Midddleware Company, after all birds of a feather flock togther. Ed, this time around the "public beating" may be your last one.
  17. Spec authors and reviewers[ Go to top ]

    Thanks for the input. Check your facts. For the expert group authors and reviewers of the spec, please consult the title page of the spec. I submit that they are not dimwits.

    TMC is indeed owned by Precise, who is a Microsoft Strategic Partner. If that invalidates any case studies we do in your mind, that is your right, and I understand-- please disregard our effort.

    Note that Precise is about to be acquired by Veritas so you may have missed some other conflicts of interest to rant about.

    As far as TSS vs. TMC, let's see... General Electric (GE) owns NBC. A defect in, say, a turbine engine manufactured by GE is not the fault of NBC. Or is it? Please understand the distinction, and then decide for yourself.

    I believe TSS (Floyd) decided to report on this because eWEEK and JavaWorld already had. Very late, I might add-- this news is somewhat old.

    Salil Deshpande
    The Middleware Company
  18. Here we go again[ Go to top ]

    "n n", Dion Almaer doesn't work for TMC anymore, he is a full time employee of TheServerSide.com and he is the new Chief Architect, having implemented all of thew new forum features may have noticed that have been added over the last 6 months.

    As for deleting posts, the editors of TSS are pretty anti censorship but sometimes delete the most extreme of posts. A recent example is where someone has been creating new accounts and posting inflammatory emails using Rolf Tolleruds name - these types of messages are clearly grounds for deletion.

    Floyd
  19. Here we go again[ Go to top ]

    Why is the serverside being used as an AstroTurf ground for satisfying TMC insatiable appetite for polishing bill gate's knob?

    >
    >
    > Why would anyone want the same scheming dimwits to write a spec for benchmarking among other things? Shouldn't TMC be demoted to sh*tbird activities? Especially, when this very same company is owned by Precise with no J2EE partners and Microsoft as a strategic partner.http://www.precise.com/Partners/Strategic/
    >
    > I can readily see a conflict of interest. TMC, do yourself a favor and stick to dotnet consulting or weren't you able to find any open dotnet consulting positions?
    >
    >
    > Floyd you wrote:
    > "As a reminder to our readers, TheServerSide.com (a media business) is an independent business unit of The Middleware Company (a technology consulting company) and does not have any special knowledge of, involvement in, or influence over this effort" -floyd
    >
    > If this is so, why is dion almaer who is employed by TMC deleting posts on the serverside?

    Good questions indeed. TMC/TSS has no credibility and should not be involved in any kind of benchmarks.
  20. One Way said: Good questions indeed. TMC/TSS has no credibility and should not be involved in any kind of benchmarks.
    Cameron Purdy said: Then write your own implementation ... and run the tests yourself.

    "One Way", your questions are answered by Floyd in the post right above yours, and my messages earlier.

    Cameron may have said what he said to Chameleon in jest, but I wanted to say again that is real possibility. All the starting point codebases, and everything else we have so far, is on the case study website, ready to go.

    We are aware of our image problem, from the last case study. We are aware that there may be people out there that will claim that we are not (a) competent enough to conduct a case study, or (b) not trustworthy enough to conduct a case study, or (c) both. We disagree, but realize that we cannot change everyone's opinion at once.

    That is exactly why we are doing it this way. If you believe that the expert group has come up with a decent application spec that represents typical customer requirements (which people here generally seem to agree on), then we encourage you to download the stuff on our case study page, and do your own case study, and tell us what you did and what your experience was. I have personally called up CEOs and CTOs of other consulting companies and encouraged them to do so. Some will.

    And again I remind you that this raw material can be used for, but is not just for, performance case studies.

    Salil Deshpande
    The Middleware Company
  21. Windows Server 2003 & TSS[ Go to top ]

    This will quickly get marked as noisy, but I received a Windows 2003 Server disk in the mail the other day, and the text on the inside jacket reads:

    "The Middleware Company compared the .NET Pet Shop Web application on Windows Server 2003 to the performance and scalability of a comparable, revised, and fully optimized J2EE application. The .NET-connected application on Windows Server 2003 was 300% faster, 76% less expensive based on price per performance, and saved development time by requiring 11,000 fewer lines of code."

    Being a fan of TSS, I was ashamed that TSS/Middleware Co. sold out so easily and gave Microsoft such nice marketing material. So TSS/Middleware Co/Whoever you are really shouldn't be involved in anything to do with measuring/benchmarking/evaluating or whatever you wish to call it.

    Very sad indeed.....

    Tom
  22. Windows Server 2003 & TSS[ Go to top ]

    Tom,

    I don't think your post is Noise and hope it is not marked as such.

    Microsoft did not consult us or ask our permission before doing that. In fact, we had no idea they were doing that. We found out along with everyone else when the CDs went out. The timing is pretty sad, I must admit.

    Please note that so far in the current effort we have already involved a number of independent J2EE experts to help us and watch over us.

    Many things have changed. As just one example, if you read the spec, it says that the Lines Of Code count used in the last study is invalid and obsolete.

    Our choices, after some of the clumsiness of the last study, were (a) quit, (b) try to do it better. We may fail, but we're hoping that with people getting involved as they have been, we can succeed.

    Salil Deshpande
    The Middleware Company
  23. Windows Server 2003[ Go to top ]

    Salil,

    Thanks for your candid and sincere response to my post. I WILL keep an open mind and hope good things come forth from this specification.

    Just when I thought the whole PetStore/TSS/.NET soap opera had run it's course, I got that Win2003 Server disk in the mail and was surprised to see "Middleware Petstore stuff" as the main focus of the marketing material.......

    Hopefully these case studies and framework will bring some credibility back to what was once THE website for Java server side development.

    Best Regards,
    Tom
  24. Microsoft did not consult us or ask our permission before doing that. In fact, we had no idea they were doing that. We found out along with everyone else when the CDs went out. The timing is pretty sad, I must admit.


    Thats the whole point? Who cares if middleware calls this as Benchmark or Casestudy.. If the results fall in favour of .Net thats it..This will be a year supply of ammunition for Microsoft marketing team !?.

    I am not discouraging the effort of middleware/theserverside.. But i would appreciate if the focus is on "How to build good J2EE applications".

    A small quote from the spec..

    >> 1) J2EE-SERVLET-JSP, i.e., a constraint that Servlets and/or JSP be used, but that no EJBs are used, and
    >> 2) J2EE-EJB-CMP, i.e., a constraint that EJB 2.x with CMP2 and Local Interfaces be used
    >> 3) DOT-NET-CLASSIC, i.e., a constraint to use the classical .NET architecture as prescribed by Microsoft

    Rather than having a DOT-NET-CLASSIC i would expect J2EE-Servlet-JDO to be in that place.. so that it doesn't fuel the .Net - J2EE war !!!..
  25. Stored Procs[ Go to top ]

    Salil,

    I gave a curosory glance at the spec.

    Not clear to me whether you allow sprocs or not.
     
    SProcs are an established db best practice for sql logic.
    It is bad for business logic.

    Many .Net Enterprise applications use wrapped data logic
    in a Sproc data access layer.

    I accept that many java programmers are ignorant db programmers.
    Something you do not understand - do not accept.

    I hope uou allow plsql/tsql sprocs - otherwise this case study
    is not very intersting to me.

    Chameleon
  26. Stored Procs[ Go to top ]

    Chameleon: I accept that many java programmers are ignorant db programmers. Something you do not understand - do not accept.

    When you say "db", I assume you actually mean "Microsoft Access" or "Microsoft FoxPro"?

    Chameleon: I hope uou allow plsql/tsql sprocs - otherwise this case study is not very intersting to me.

    Then write your own implementation that uses stored procedures, and run the tests yourself.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  27. Stored Procs[ Go to top ]

    Chameleon,

    It's funny you ask. When we started working with the expert group, I thought that everything that could be said about stored procedures had already been said before, and that "of course" stored procedures should be disallowed.

    As it turns out it was debated all over again by the expert group. Rod Johnson (who is also on the Servlet 2.4 expert group, and is author of "Expert One-on-One: J2EE Design and Development") kicked off the debate by saying that stored procedures should be allowed, because they are a J2EE best practice. He also makes this point in his book. He went on to say that he was going to do a submission, but would not, if stored procedures were disallowed.

    So there was lots of debate - most experts chimed in with their two cents - and in the end the group decided that although using stored procedures is indeed a best practice for enterprise applications (whether J2EE/.NET/PHP/...) that for the purposes of these case studies, the specification should disallow them.

    The fact that Microsoft's original PetShop (not from our October 2002 case study, but from way back, over two years ago) put a lot of functionality in stored procedures, and the public perception of that, was also one of the factors considered. (The Microsoft PetShop in our October 2002 case study did not use stored procedures).

    If you visit the following link you can read some of this and other debates, and who said what:
    Link to pdf of unincorporated expert feedback

    Salil Deshpande
    The Middleware Company
  28. Spec tilted towards J2EE[ Go to top ]

    Salil,

    I think your expert group is more heavy on J2EE expertise - with a mandate
    not to lose to the best performance features of .NET. I do not see too
    much MS expertise on the panel (may be some one from MS) ?

    Your test approach is non-confrontational - with a way out for J2EE camp if J2EE loses - as it is not a "benchmark".

    I agree that business logic implemented in Sprocs is bad.

    But data access wrapped in sprocs is pretty much a standard best practice in .NET world & MS recommended blueprint.

    BTW - this is also a security feature as SQL injection bugs are minimized.

    When a database is involved in a benchmark, there should be database specialists in the review panel. Not a bunch of middle tier experts with
    no advance skills like qury tuning, round trip elimination skills, query plan optimization, etc. (my guess).

    Anyway this "no sprocs" is a sun party line.

    Most .net enterprise applications are not written that way.

    So the "benchmark" does not follow MS .NET best practices just like the java
    camp would like to follow sun party line (blueprints) religiously.

    Most java programmers I personally know - would rather have nothing to do
    with a database. Their line - just give me a xml file - the tool should
    generate a schema, class should map to a table, code gnerator is a serializer
    - do not bother me with database stuff. Understandable - because that is
    not their speciality.

    Anyway - the spec as it stands - I feel is written by a J2EE bunch - making
    sure J2EE will not lose towards .NET (by not allowing .net best features &
    MS blueprints - no sprocs, no page caching - btw - you realize that win2k
    uses http.sys kernel caching - so j2ee will lose here).

    Chameleon
  29. Spec tilted towards J2EE[ Go to top ]

    Chamelon said: I think your expert group is more heavy on J2EE expertise

    I would agree that the expert group is a little heavier on J2EE, but there were .NET people on there too.

    with a mandate not to lose to the best performance features of .NET.

    That was really not the mandate.

    (may be some one from MS)?

    We intentionally did not invite anyone from Microsoft or a J2EE platform vendor to the expert group.

    When a database is involved in a benchmark, there should be database specialists in the review panel. Not a bunch of middle tier experts with
    no advance skills like qury tuning, round trip elimination skills, query plan optimization, etc. (my guess).


    When actually conducting a performance case study based on this spec, if the goal is to compare the performance of two application server platforms, or two middle tier application architectures on the same platform, (or anything like that in the middle tier), then it is essential to have a database expert tune the database, and make sure that even under the heaviest workload, the database is NOT the bottleneck. There are some fairly simple litmus tests one can do to determine if the middle tier is waiting on the database layer.

    Salil Deshpande
    The Middleware Company
  30. tuning[ Go to top ]

    Salil,

    Looks like you have already made up your mind. :-)

    Tuning a database & tuning a database application (+architecture)
    are very different things.

    Chameleon
  31. Hi All, I rarely post here, but I am very interested in the benchmarks, case studies, and comparisons between J2EE and .NET, now before I state much more here, I should state who I am, My name is Scott Worley, and yes I write books on ASP.NET and .NET technology. AND NO I AM NOT STARTING A FLAME WAR... (since I have also Tech reviewed and edited various J2EE books also.)

    Now that is over, for those that are reading further, thank you :)

    Okay I am all for seeing the strengths and weakness of both J2EE and .NET, it is acknowledged by far more people, than most realize that J2EE is a very viable platform for enterprise development, in my opinion I find the J2EE community sometimes gets a little too accademic when it comes to talking about solutions..

    Both sides of the fence (MS/SUN = .NET/J2EE) are far to evangelistic for words, sorry, but it is true.

    Now I would love to know who is helping on the .NET side of the spec. since there seems to be some comments on a possible bias towards J2EE, I have not looked at the spec. yet, but if the community would like, I am more than willing to contribute in any way I can, time and resource permiting.

    So what do you all say, and to TSS, I appreciate the effort you are putting into, getting as close a comparison test of systems abilities that you can. You will never make all sides happy, but the effort should be applauded.

    As a final note, using or not using Sprocs in the spec. well as stated earlier it is a MS best practice, but having said that a lot of developers do not use stored procs in the MS world also. this is not such a big issue in the real world, but in bench marking it would be considered unfair, is it not possible to have the spec require a DAL that can use either Sprocs or Statements, and run benchmarks against both techniques?

    Just an idle thought...

    Scott Worley
      Author: Inside ASP.NET, Compact Framework .NET Developers Guide, Tech Reviewer: J2EE Unleashed 2nd Ed., and others...
    CTO - IT Resources...
  32. Scott,

    Just a brief comment, as I mentioned earlier we did experiment with SP's and found that there is very little performance difference overall. I believe that the .NET Petshop authors did that also and found the same thing. No exhaustive performance comparison of this has been done on our part because it takes so much time to do but we did do some testing for a couple of days on two different databases and didn't see a worthwhile difference.

    On a personal note: I frankly welcome any cross pollination between the products and think that in the end that .NET is going to make Java and J2EE faster, stronger and better.

    Cheers, Will
  33. On a personal note: I frankly welcome any cross pollination between the products and think that in the end that .NET is going to make Java and J2EE faster, stronger and better.

    ---> Good, I really want to seee good interop between Java/J2EE and .NET development tools, and even more importantly the community.

    its about time the infiighting stopped and people realised what the both the Java and .NET communities have brought to the software development industry.

    for example JAVA/J2EE (well small talk really) Patterns and also Unit Testing frameworks.

    on the MS Side, what can be done with resource and ability and great developer productivity tools.

    Both communities have extremely skilled and knowledgable people.

    thats it.

    Scott Worley.
      Aurthor: Inside ASP.NET
  34. Scott Worley: but if the community would like, I am more than willing to contribute in any way I can, time and resource permiting

    Hello Scott, Would love to have you. When you get a chance can you send mail to the address specified on the case study website. The invitation letter on the case study website talks about minimum time and resource committments we hope you can make.

    Now that the spec is stable-ish, and codebases have been submitted, perhaps you can spend time scrutinizing the codebases, making sure they're conformant, etc. If you'd like to make changes to them, there's a document on the case study website that asks you to follow some simple guidelines.

    Scott Worley: Now I would love to know who is helping on the .NET side of the spec. I have not looked at the spec yet

    Please do check the spec for the list. Among the .NET people is the knowledgeable and rabid fan of Microsoft technologies, Roger Sessions. He did not rock the boat much (perhaps because the group's work was very even-handed, we'd like to think) but his presence was felt.

    Scott Worley: Both sides of the fence (MS/SUN = .NET/J2EE) are far to evangelistic for words, sorry, but it is true. TSS, I appreciate the effort you are putting into, getting as close a comparison test of systems abilities that you can. You will never make all sides happy, but the effort should be applauded.

    Agreed... which makes work like this close to a no-win situation, so we're always impressed and grateful when people like you step up to help.

    Scott Worley: is it not possible to have the spec require a DAL that can use either Sprocs or Statements

    Yes, but probably not without more debates over the merits of even allowing stored procedures again... and we would have to specify the boundaries (interfaces etc.) of such a DAL layer. One side note though: the codebase Microsoft has submitted for the .NET category does contain a DAL, for a different purpose: to switch between multiple databases without code changes.

    Salil Deshpande
    The Middleware Company
  35. Evangelizing .NET and J2EE...[ Go to top ]

    Both sides of the fence (MS/SUN = .NET/J2EE) are far to evangelistic for words, sorry, but it is true.

    >

    I can't let this one pass by, probably because I am a .NET Evangelist for Microsoft (for those of you who have watched Doug Purdy's video here on TSS, I'm his replacement -- he's moved on to doing some other cool stuff in Microsoft). Soo... (taking deep breath)...

    I'd like to think that my presentations to the Java community are _not_ "far too evangelistic". I work very hard to be open and respectful of our competition (especially Sun), and you won't find me "bashing" Java (I'm more like Harry Truman... if I speak a fact and you don't like it, you'll call it "bashing" anyway :-). A little over a year ago, I was a J2EE Architect (and a sworn enemy of Microsoft :-). I, too, felt that there was a lot of "FUD" coming from Redmond. What I see on the inside is very different. Inside my division (.NET Developer Division), we genuinely care about the problems people face on our platform and the interoperability people _must_ have with J2EE systems. The worst "FUD" I see now comes from overly eager press trying to wildly speculate, or from overly-aggressive developers in our community. I think I get more upset than most J2EE folks when I see somebody saying bad things about Java just to make .NET look better, because everybody points the finger back to Redmond :-( Frankly, I always welcome criticism of my stuff if people think I'm "dissing" Java/J2EE unfairly. If I don't "play fair", I won't have any credibility with the community I originally came from.

    Of course, we also compete against J2EE on our platforms, and we believe we have compelling reasons to use our technology, and try to present them in fair ways. PetShop is a good example (no, really :-). If we merely considered PetShop a "Marketing Gimmick", we would have stopped at 1.0/1.5. Instead, here we are with the 3.0 version, with several architectural improvement (and I might add: Code that I'm _now_ proud to show to other people :-). The really interesting thing is that the rearchitecture resulted in some performance improvements. That being said, people will see that the J2EE versions will show some serious performance improvements too (unfortunately, last I heard, none of the J2EE vendors wanted to openly participate, in spite of the extreme openness and flexibility of the benchmark -- keep an eye on http://www.middleware-company.com/casestudy/ for more details on that). Of course, if you don't like the way they ran the tests, you can run them yourself, since all the source/test scripts will be made available.

    Regardless, the folks at the Middlware Company should be commended. They've worked very hard to try to make this as balanced as possible. This is one of the best efforts I've seen at creating a balanced comparison between two different enterprise application technologies.

    Thanks for reading my ramblings! :-)
  36. Evangelizing .NET and J2EE...[ Go to top ]

    Same David Weller from Context and Valtech?
  37. Evangelizing .NET and J2EE...[ Go to top ]

    Nevermind, I answered my own question
  38. Stored Procs[ Go to top ]

    SProcs are an established db best practice for sql logic.

    > It is bad for business logic.

    Business logic is business logic. You don't clutter it with persistence logic be it stored procedures or ODBC/JDBC whatever.

    > Many .Net Enterprise applications use wrapped data logic
    > in a Sproc data access layer.

    Good for them. (If flexibility is not important/needed)

    > I accept that many java programmers are ignorant db programmers.
    > Something you do not understand - do not accept.

    Nice line. **Java programmers are ignorant db programmers**. You have the authority to demean most of the java programmers in the world. You must be some guru, eh?

    How's this: You have a customer base which already has a 'in house database standard'. Lets say 4...6 different database solutions out there. You're saying that if I want to get rid of the ignorant db programmer status I need to write stored procedure logic for all those databases? 'Cos I sure ain't going to tell the customer to switch database just to get my app working.

    Maybe the **ignorant database programmer** ( I take it that you mean using JDBC for persistence logic) translates to **respects customer's needs**.

    Not all companies have to luxury to be able to dictate the database which is needed in order to run the app.

    Not all companies (in previous situation) can afford to write stored procedures for all possible databases.

    > I hope uou allow plsql/tsql sprocs - otherwise this case study
    > is not very intersting to me.
    >
    > Chameleon

    Personally **being an ignorant database programmer** I'm not interested in stored procedures, but they are also one solution and it would be interesting to see the effect on overall performance/complexity.
  39. Stored Procedures[ Go to top ]

    Hello,

    1. Stored Procedures can be part of
    "Persistance Layer" - to facilate and speed up
    access to data, to enforse some complex "DATA INTEGRITY"
    constrains.

    2. Stored Procedure can be used ALSO as
    real part of "Business Layer" - to provide
    relatively complex data manipulations.

    ((Yes, somebody can argue that FK-PK is also,
    kind of, Business Logic and any data manipulation
    is, kind of, about integrity... I am, KIND OF, agree ))

    3. there are tens factors to consider usage of
    SP in every given project. General words like
    "SP - evil" or "SP - G-d" are useless (to say it politely).

    4. For benchmarking ("frameworking"???) middle-tier
    servers / sw / platforms SP must be left out or
    be 100% fixed by API.

    Alex V.
  40. Stored Procs[ Go to top ]

    Guys,

    Regarding this comment:

    >it would be interesting to see the effect on overall performance/complexity.

    Stored Procedures were one of the things we looked at really hard when doing the spec and the base implementation:

    http://www.middleware-company.com/casestudy/msPetShop3.2.zip

    When you look at this you'll probably see that there is only one place that SP's would really come into play. When placing an order you have to get the next sequence number (or identity depending on the DB), insert into three tables and update another (which maybe on a different DB instance). This is easier with an SP but using a properly prepared statement you can get performance that is within a few percent of that obtainable with a SP.

    At a courser grained level that order insert is only one step in the overall app. We did do some simple testing with and without SP's for this step and found in the end that there was no real effect on performance overall, although it did reduce the complexity of that one class a bit. However, it made DB maintenance a pain:)

    Where SP's might be really interesting is if you combined them with triggers and callbacks. Then you would (theoretically) be able to avoid DB reads (for example account, signon, profile information) by reading once, caching the results and relying on callbacks to update you if that information changed. That'd be quite a hefty change though.

    Cheers, Will
  41. I would expect J2EE-Servlet-JDO to be in that place


    I don't understand why O/R mapping solutions aren't allowed in the first place, at least with the initial categories. Lightweight persistence via a JDO implementation, Hibernate, TopLink, or CocoBase is a best practice when it comes to typical J2EE web applications. According to the functional specification, your case study _is_ a typical J2EE web application. The only notable exception is the distributed transaction requirement, as far as I see.

    So what's the J2EE-SERVLETS-JSP category about, in terms of data access? Just plain JDBC? Are abstraction toolkits like the Spring Framework's JDBC templating allowed? Home grown half-baked O/R mapping? Or even basic mapping solutions like Clinton Begin's iBatis Database Layer? If the latter is, why shouldn't Hibernate be too? Or is Clinton forced to rewrite his JPetStore using plain JDBC for this case study?

    Applications that leverage generic lightweight persistence solutions typically shine in terms of LOC, flexibility, and ease of deployment - often with very good performance, and without generating code. A comparison of such an implementation with both an EJB CMP one and a plain JDBC one would be interesting in many respects. Just comparing the latter two enforces the old boring "plain JDBC vs EJB CMP" debate that noone wants to hear anymore.

    Note that it might make sense to use e.g. Hibernate for most data access, but Spring JDBC for special parts of the application. A respective category shouldn't disallow such a combination. BTW, that's somewhat similar to the stored procedures argument: Noone advocates using SPs for everything, just for isolated hotspots.

    Juergen
  42. Juergen said: I don't understand why O/R mapping solutions aren't allowed in the first place

    Simply to limit the scope for now. I also would like to see this opened up. We wanted to avoid inadvertently promoting case studies of:

    .NET appserver with O/R mapping/caching product W
    J2EE appserver A with O/R mapping/caching product X
    J2EE appserver B with O/R mapping/caching product Y
    J2EE appserver C with O/R mapping/caching product Z

    ...which would increase confusion.

    In fact, Vivek Singhal, VP of Engineering and CTO of Persistence Software, which makes a very fancy O/R mapping and caching solution also expressed disappointment that it is currently out of scope. Cameron Purdy of Tangosol, which also makes a clustered caching solution, understood the need to limit scope and suggested a NO-HOLDS-BARRED category for the future, which sounds like fun.

    So what's the J2EE-SERVLETS-JSP category about, in terms of data access? Just plain JDBC?

    Yes, just plain JDBC, for now.

    J2EE-Servlet-JDO and many other such variations are absent again to limit scope for now. If there is general acceptance of the categories idea, then we'll add these. As you can see we are biting off a lot and there is enough skepticism about all the other things, that we really do need to limit scope where we can. You all don't need to wait for us though. You can define your own category and do your own case study based on the application described in the spec. Starting point codebases are available on the case study website. If you send us a precise definition of a category we can probably paste it into the spec and cite you as the source.

    Salil Deshpande
    The Middleware Company
  43. <Salil>
    Yes, just plain JDBC, for now.
     
    J2EE-Servlet-JDO and many other such variations are absent again to limit scope for now. If there is general acceptance of the categories idea, then we'll add these. As you can see we are biting off a lot and there is enough skepticism about all the other things, that we really do need to limit scope where we can. You all don't need to wait for us though. You can define your own category and do your own case study based on the application described in the spec. Starting point codebases are available on the case study website. If you send us a precise definition of a category we can probably paste it into the spec and cite you as the source.
    </Salil>

    I understand the need to limit scope. But where to draw the line? Clinton Begin's iBatis database layer is a simple but still O/R mapping toolkit. Is he allowed to use it for a submission in this category? As I've said, if yes, then why not Hibernate or a JDO implementation too?

    Besides, one could always write a home-grown generic database layer and include it in the submitted source code. What if someone simply includes the whole Hibernate source code in a submission? I suppose it would add to the LOCs then...

    Does the same apply to all other libraries? May one use Struts, Log4J, or in fact any non-J2SE/J2EE library? If yes, then why no lightweight persistence toolkit? In the end, all of them could be included in the WAR, without any installation effort. And which libraries would add to the LOCs? What if the libraries are large, but only a small subset of it is actually used?

    Limiting the Servlets/JSP category to no non-standard libraries at all would result in very unrealistic implementations. Third-party and especially open source libraries are a matter of fact in modern web applications. Their availability can even be considered one of Java's biggest advantages.

    In fact, we consider a submission using the Spring Framework, Log4J, and possibly Hibernate - a typical combination of application framework, logging toolkit, persistence solution. But I guess we won't do it if there won't be an official category for it, or if any of the libraries would add to the LOCs.

    Juergen
    (Spring Framework developer)
  44. Juergen Hoeller said: But where to draw the line? Clinton Begin's iBatis database layer is a simple but still O/R mapping toolkit. Is he allowed to use it for a submission in this category?

    As the spec is written right now, Clinton would have remove the iBatis db layer and use straight JDBC, to be conformant to that category. We have been discussing this with Clinton.

    Drawing [one of] the lines at straight JDBC is really a judgement call. We found that one of the side debates of our last study was, how much overhead do EJBs really add. The thing most people want a comparison to, is straight JDBC, not JDBC + an arbitrary DB layer. That is really why that category is constrained that way. Several people felt this was important - Tom Murphy of The META Group was one of them.

    For this purpose we think that Struts is okay to include (submittor's option), but again that's a judgement call.

    In fact, we consider a submission using the Spring Framework, Log4J, and possibly Hibernate - a typical combination of application framework, logging toolkit, persistence solution. But I guess we won't do it if there won't be an official category for it, or if any of the libraries would add to the LOCs.

    That is great that you are considering a submission. At your leisure, can you send us mail at the address on the casestudy page so that I can get your email address and we can discuss this? We can work together to define a category that is specific enough so that it doesn't become NO-HOLDS-BARRED but general enough to allow alternate but equivalent implementations of the frameworks that you want to use. Then we'll run it by the expert group and add it to the spec.

    Categories are not carved in stone. The goal is to not make this a contest, but structured experimentation which we can all learn from.

    The current spec has made the LOC count obsolete (reasons are in the spec document).

    Salil Deshpande
    The Middleware Company
  45. Windows Server 2003 & TSS[ Go to top ]

    "In fact, we had no idea they were doing that. We found out along with everyone else when the CDs went out. The timing is pretty sad, I must admit. "
    >>OK, So... what is your company going to do?? Sorry mate, you did a pretty good job for MS and now is late for regrets. Congratulations!!!!
  46. Kudos![ Go to top ]

    Its great after the public beating you took that you are working with experts to do it better. We need more of this. More power to you all. Like the eweek article says, sail on.
  47. How about TSS creating and deploying an application comparable to ASP.NET forums? (download is here and discussion is here)

    That can be a nice and somewhat real-life excersise.
  48. Dimitri,

    We're toying with ideas similar to this, and asking the TSS folks to consider them. As you probably know, currently TSS consists of JSPs talking to Session Beans talking to Entity Beans talking to the db. It might be neat to implement a small [new] front-end feature or little section of the site with ASP.NET pages which invoke the existing Session Beans, using either vanilla web services or products like Janeva, JNBridge, or JuggerNET, and document the experiences and difficulties, as was done while adding new appserver products to the cluster.

    Regarding your comment that it would be real-life, some will argue that a forums site is not real-life. But with TSS now running on a cluster of 4 different J2EE appservers and using the Tangosol clustered cache, there's more engineering in it than might appear at first glance. Throwing ASP.NET into the mix would be interesting.

    Salil Deshpande
    The Middleware Company
  49. J2EE vs. .Net Benchmarks[ Go to top ]

    I just read Timothy Dyck's column in the May 12 issue of eWeek regarding the October 2002 benchmark and your efforts on a new platform-neutral benchmark.

    I want to thank you for taking a beating after the October benchmark and, rather than leave it alone, you decided to work with industry experts to create a superior platform-neutral benchmark. You are helping to correct a serious deficiency in the current server market where reliable benchmarks are rare.

    Kudos to your team,

    Matthew Zaleski
  50. M$ already?[ Go to top ]

    Have anyone seen that M$ has already created it's PetStore implementation for this benchmark? look here.

    How quick...
  51. M$ already?[ Go to top ]

    Hi Henrique,

    They've actually been working on that for a while, since early April or so.

    We solicited codebases conformant to our spec. They provided the one for the DOT-NET-CLASSIC category.

    We asked them to incorporate the feedback from the October 2002 study on their codebase into the new codebase. So versions of the codebase have been downloadable on the case study website home page.

    We asked them to document the changes they made. Those changes are listed on the case study website here (pdf).

    Any comments you have on the codebase (conformance, equivalence to the J2EE codebases, etc.) would be welcome, if you can send them to the email address of the expert group specified on the case study web site. We have urged a number of experts to look at these codebases.

    Salil Deshpande
    The Middleware Company