Sun and BEA have teamed up to produce some formidable new ECperf metrics. Running BEA Weblogic Server 7.0 on a Sun Fire V880 System yielded a performance figure of 16668.47 BBops/[email protected] and a Price/Performance rate of $26 BBops/ [email protected] using Oracle 9i Database with BEA Weblogic 7.0 jDriver for Oracle.


Check out New ECperf Results Here

Finally, a benchmark on Sparc Solaris (not counting the withdrawn Borland Appserver result).

However, in my quick scan through the report, I couldnt work out whether they had used the 32-bit or 64-bit JVM.

It would be interesting to know what perf difference there is between the two.

Has anyone done any real-world, performance comparisons between these two JVM's?

Cheers,
Nick

Nick,

  A 32-bit JVM was used for the benchmark. I am not
aware of any ECperf runs with a 64-bit JVM.

 regards,

  Fernando

Fernando,

Did you get that info from the report - or do you work for Sun/BEA?

Cheers,
Nick

Nick,

 I work for Sun. I wrote most of the FDA/FDR.

 regards,

  Fernando

Ta. I didnt mean to pry - just wanted to know if I'd overlooked it.

Can we expect a 64bit JVM result sometime soon?

-Nick

a) What is "advantage edition"?
b) Why are they running an unsupported configuration? (ie jdk1.4 with wl 7.0 on solaris 2.8). According to http://edocs.bea.com/wls/platforms/index.html#solaris8 this was unsupported last i knew.

Advantage is a non clustered version. Is you look closely then you'll see they use round robin DNS to load balance the incoming T3 requests over the WL instances on the box.

They aren't using any of WebLogics clustering capabilities. Plus, advantage is almost half the price of the Premium edition (which does have clustering). It's BEAs way of artifically lowering pricing for the benchmark as a response to an inability to have a packaged prices discount which other vendors use but it's arguably 'worse' in that it isn't a real world topology for most users but a novel one none the less.

Whats more interesting is why they bother with all the round robin stuff and don't just run a single WL instance on the Sun and hence avoid the need to do any load balancing plus they need significantly less RAM (they'd probably get by with 3-4GB of RAM). I guess performance wise they don't scale well to 8 CPUs and hence all the jumping through hoops to get a cheap clustered solution.

Billy (yes, I work for IBM but my views don't represent IBM).

Billy Boy:

You're back ... tooting that old blue horn of yours :-)

Let's not talk about all the dirty tricks that IBM has played with pricing, benchmark specials et al to get an artifically low price. Instead, let's talk about how this BEA/Sun result significantly outperforms your AIX result!

Andy - I thought Billy's response was very good. He was honest about (a) working for IBM and (b) that BEA couldn't do "bundles" so they resorted to other ways of putting a lower price package together.

BTW - Both BEA Weblogic and IBM Websphere absolutely _fly_ on these new Sun boxes. (IBM tries to hide it, but their Websphere product is not only available for Sun Sparc/Solaris but appears to be as popular on that platform as anywhere ;-) I am really impressed with the Sparc 3 platform, and it's hard to deal with these old Sparc 2 machines now!

Now if only IBM and HP would start sending me some hardware to play with ;-)

Peace,

Cameron Purdy
Tangosol, Inc.

"IBM tries to hide it, but their Websphere product is not only available for Sun Sparc/Solaris but appears to be as popular on that platform as anywhere ;-)"

IBM doesn't try to hide it. It performs and scales well on all the major platforms, and works with all the major databases. As you point out, it flies on Solaris and has a very large base of users in production on that platform.
Yes, I work for IBM.

IMHO, i guess the scalability per CPU depends also from the JDK implementation, not just in the AppServer vendor.

Also, the round robin trick is a way not to introduce session replication overhead, which, to me, make sense for situation where you need load balancing and not session replication.

BTW, in common BEA configurations, the ones with the "Web Server + AppServer (clustered) + DB" triad, the Web Server plugin (apache, IIS, Netscape...oops, ONE) ALWAYS use round robin to load balance between app server instances.

The expensive clustered version is meaningful only in situations where you do want session replication (do not interrupt the user experience, or, better, the transactions you are managing), not just to achieve high availability of systems.

Luca,
The round robin trick also loses you failover capabilities. The point I raised wasn't really to do with whether it's round robin or weighted, it was to do with what happens if an app server fails.

But, none the less, it's a legal configuration as far as ecperf goes. I guess we may see other vendors using the same trick thereby removing any temporary advantage in any case in the long run.

Thanks
Billy

Billy,
To add to the points Luca has made, the WebLogic web server plug-ins do support failover. However, if the WL app server instances are not clustered(http session replicated) you would not get seamless http session failover (a new http session will be created on the new server the request fails over to), so the failover would not exactly be transparent. There are a few producion sites who do not bother about seamless session failover and they have gone ahead with the standard Web server+WL plug-in+WL non-clustered server setup to save a few bucks and they are happy.

-Dan



 

Billy,

my point was that you DO have failover capability, just you do not have session replication.

If you do not use clustering (in WLS 5.1 and above), and your home banking appserver on the web goes down, you will start a new session whenever you interact.

The automatic failover of the clustered version is just for current user sessions on the appserver instance.


Thanks, Dan, for the clarification.

talking about load balancing, this works just when you proxy requests through another instance of WLS.

Interesting enough, about the whole performance issue, is the point that IBM runs WebSphere on its own VMs, and BEA will just do that (given the recent acquisition of...i forget the name).

Just my .02 €

Luca
luca dot botti at tiscali dot it

Even the $10,000 version of WLS CAN do JDBC based session replication. (although websphere clustering locks you into it -with the DB being the single point of failure.)

btw, ecPerf is focused on testing performance not failover.
Matt

Matt: "Even the $10,000 version of WLS CAN do JDBC based session replication. (although websphere clustering locks you into it -with the DB being the single point of failure.)"

It's also the single point of "doesn't-scale-well". Having profiled Websphere apps, it's amazing to see how much JDBC work is required by Websphere to maintain sessions.

Peace,

Cameron Purdy
Tangosol, Inc.

Billy:

Why must you always resort to gross lies to convey your point? The Advantage Edition of WebLogic isn't as ineffective as you make it out to be!

Oh wait ... you work for IBM! I should've known better (after all you folks invented the dubious passport pricing scheme and the benchmark special optimizing tool).

Cary
Why does it always get reduced with you guys to insults. You can't trade posts on facts so you end up name calling.

First, something interesting and not related to your note. I was just checking the V880 specs and it looks like with 8 CPUs, it comes with 16GB of RAM so you're basically paying for the RAM anyway. For this scenario, it makes sense therefore to use the RAM by running multiple instances and pinning instances to processors rather than run a single instance across all 8 CPUs and waste 13GB of RAM.

Now, the stuff I hate doing which is responding to posts such as Carys or Andys typically. I never said Advantage was ineffective just as I wouldn't say WAS AEs (also non clustered) was ineffective. If you don't need builtin IIOP failover or HTTP session failover then both will work nicely and cost you less money. Great.

So repeating my-self for those of hard of sight, I just said that this adds yet another twist to the whole thing making pricing almost irrevelant in these ECPerf numbers. BEA saved some money by using Advantage here. They can't discount WebLogic and Oracle together like IBM and Oracle can and do with their stacks so this is what they do to get the price down.

On the point of passport pricing, that has been around a long time before ECPerf came along so I don't get where your point comes from. Buy n items from n vendors or buy n items for one vendor. Of course, the one vendor has a pricing advantage over n vendors. It's an advantage of going with a single vendor, but of course, if you want to cherry pick and buy what you perceive to be the best product in each niche then of course, it costs more and you have more trouble support wise when some goes wrong because all the vendors blame each other. There is an advantage to having one guy to scream at when your stack doesn't work rather than 5. But, thats the choice, and if thats what you want to do then it's a legitimate and valid choice.

As for your benchmark special, well frankly, we've beaten that argument to death successfully on other threads and for more information, refer to the other threads.

Billy (yep, I DO work for IBM).

The only valuable info i get from all this is :
we've know an true alternative to CICS and TUXEDO;
in addition, comparing to this 'old' TP monitors that were somehow locking the customer to a very propriatary middleware, we (customer) are gaining some kind of freedom due to the common underlying techno on which this new breed of TPM are built.
Laurent.

Before people mis-understand BEA's product packaging, just let me point out the facts :
1. Weblogic Server Advantage Edition supports clustering.
2. Weblogic Server Premium Edition supports clustering including the In-Memory Replication of StateFull EBJ's.

Bernard. (not expressing any of my company's statement)

Bernard,
Can you point me to a page where exactly what capabilities are enabled in each product are described? I can't find any reference in the docs to Advantage or premium hence the confusion.

Thanks in advance
Billy

Does anyone know to what extent the posted results reflect performance improvements done by Oracle in the 9i version? and how does that factor in relative to IBM's benchmarks using DB2?

Doron.

<quote>
Oracle 9i Database with BEA Weblogic 7.0 jDriver for Oracle.
</quote>

Then why ecperfdomain-N/config.xml says :

<JDBCConnectionPool CapacityIncrement="1"
  DriverName="oracle.jdbc.driver.OracleDriver"
  InitialCapacity="35" MaxCapacity="35" Name="ECperfDB"
  PreparedStatementCacheSize="300"
  Properties="user=ecperf;password=ecperf;dll=ocijdbc9;protocol=thin"
        Targets="ecperfserver-1"
  URL="jdbc:oracle:thin:@ecdata:1521:ecperfdb"/>

?

--
Dimitri

remember the previous bea benchmarks, where identical configurations (except for oracle vs jDriver jdbc) had a 10% performance difference.. well, if you look closely at the configuration files, besides changing the jdbc driver, the also increased the number of max database connections by 12.5% .. so i guess that explains the 10% increase overall :)

Dimitri,

  If you look in the FDA you'll notice that some
wls instances used BEA's jdriver and some Oracle's
thin driver.

regards,

  Fernando

Ah, you are right (I assumed that all 4 instances were configured identically) - ecperfdomain-1, ecperfdomain-2 and ecperfdomain-3 use thin driver and only ecperfdomain-4 uses jDriver.

Now my question is - why did they do that? Is it some sort of optimization?

--
Dimitri

Looks like I misleaded you. It was a general note about ECPerf to be a race track without rules... I think instances would be configured identically.

The only thing I see, ECPerf is completely useless until it gives narrow hardware matrix for the test. For now it allows too much freedom for playing with with hardware and only shows how wise could be a vendor choosing particular configuration, nothing more...

<quote>
ECPerf is completely useless until it gives narrow hardware matrix for the test
</quote>

Well. This is easier said than done. Too many complexities. Like, for starters, what wld be the basis to choose thehardware? Which vendor's? Will IBM agree to run on SUN hardware? If SUN were to publish a number, will SUN agree to run on IBM hardware? Since this is an appServer benchmark, why stop at just hardware. Why not include the DBMS also in the matrix? Then, will Oracle (for their submission) or IBM agree to work on the other's DB? Unlikely?

In any case, I think the benchmark is serving its purpose. Apart from providing a platform to compare app-server performance (different matter that there is no 'orders of magnitude' difference between vendors), it is also serving to get the vendors to significantly improve performance. And customers can use Ecperf results along with CTS (J2EE compliance) to make their app-servers decisions.

Cheers,
Ramesh
- Pramati

It's interesting to compare two of the published hardware configurations with the same cost/bbops (about $27):

1)
App Server: p640 4x375 POWER3-II
DB Server: p640 2x375 POWER3-II --> 10,300 bbops/min

2)
App Server: V880 8x900 Ultra-III
DB Server: V880 6x750 Ultra-III --> 16,696 bbops/min

The difference in throughput is about %60, but look at the difference in the number, clock, and generation of the CPUs used. I wonder a) if the difference is due to WebSphere being more efficient or b) if IBM didn't bother to run a benchmark on their latest chips (Power4?) - maybe the cost/bbops would have come out too high?

Guglielmo,
My point exactly, you should compare against Intel also for more insights but if a customer wants Sun or AIX/pSeries then such a comparison doesn't really matter, it's merely a what if scenario.

Billy