BEA has made a new TORPEDO submission claiming the best results. The Non-Verified submission was made using BEA WebLogic Server EJB CMP 8.1 SP2. This submission eclipses the previous best results held by Oracle TopLink.

BEA's submission totals 20 hits. It is a single server submission that includes source code authored to the CMP standard with configuration files that enable batched database operations, caching between transactions and relationship caching..

To see all of the results: http://MiddlewareRESEARCH.com/TORPEDO/

The TORPEDO Launch Discussion.
The Oracle TopLink Discussion.

As I understand it, the previous Oracle submission that took the top spot took advantage of Oracle-exclusive features. Does this BEA submission do the same? Were the Kodo and Hibernate submissions at 22 hits truly portable across all/most databases?

The BEA submission takes advantage of WebLogic specific features such as -
batched database operations, relationship caching and cache-between-transactions.
Mihir Kulkarni
BEA Systems

BEA submissions at 20 hits is truly portable across all databases.

Michael Chen
BEA Systems

Note the use of CMP in a record breaking benchmark result.

so CMP has similar speed or it is even little bit faster than Hibernate ...
so this flaming for last couple of years was just flaming ...

any comments from Hibernate team???

This "benchmark" has nothing to do with "faster". It is just about who can reduce the number of SQL statements as much as possible, which has nothing to say about speed.

I don't think most of the people who criticise entity EJBs, complain about the data access performance of CMP. It is more about what you can't do with entity EJBs. To list a few ...

- They don't support inheritance relations
- They are not re-entrant
- They are no good for modelling polymorphic relations
- They create tight coupling between the domain and the data

To summarize, entity EJBs don't support most of the object oriented principles and techniques which are inevitable for properly modelling a rich domain.

There are loads of other shotcomings as well, in terms of productivity, testability ......

Ta
Meeraj

so CMP has similar speed or it is even little bit faster than Hibernate ...so this flaming for last couple of years was just flaming ...any comments from Hibernate team???

Do you think the number of SQL queries is the be-all end-all of ORM tools? How about polymorphism, inheritance, ability to use and pass your entity objects and re-connect them later, ability to test mappings outside a container... The list goes on.

I don't remember anyone being particularly derisive of CMPs ability to produce efficient SQL, the issue is, and has always been, the SEVERE limitations the entity bean component model places on your domain modeling expressiveness and testability.

I'm not that surprised. We've been using CMP's for a more than 2 years and it's doing its job perfectly well. Fairly easy to develop, isolates business logic from database implementation and is performing well. Despite that I like CMPs I still think they could be made simpler, aka EJB 3.0.

Erik

Yeah, it would be interesting to compare and contrast the productivity or ease of development between these products.

It's worth pointing out _how_ the number of hits was reduced. Notice this information:

It is a single server submission

That can allow db-is-shared to be set to false, which caches EJBs by their PKs, which reduces hits for ejbFindByPrimaryKey and ejbLoad.

enable batched database operations

That allows all modifications to be done with a single database execution.

caching between transactions

That suppresses cache flushes at transactional boundaries, allowing for potentially dirty reads (although if db-is-shared is set, that is the case anyway.)

relationship caching..

My understanding is that this option will allow foreign-key lookups to be accomplished from a PK cache (see above.)

Net result: Very effective use of Out-Of-The-Box caching features with the CMP container in Weblogic, but the trade-off includes the potential for dirty reads, which will increase the number of failed transactions in a typical application depending on the degree of natural isolation of the per-user data sets.

I think each submission should include a section written in English ;-) that explains what the trade-offs are (both subjectively and objectively,) so that developers can really learn from these. There's actually a lot of useful real-world tuning being done in each of these submissions, and it's being done by the vendors that know their own products pretty well. It would be a shame to miss the opportunity to mine that information.

Peace,

Cameron Purdy
Tangosol, Inc.
Coherence: Shared Memories for J2EE Clusters

Cameron,

caching between transactions

That suppresses cache flushes at transactional boundaries, allowing for potentially dirty reads (although if db-is-shared is set, that is the case anyway.)

Caching between transactions does not suppress cache flushes. If a bean has been modified, the cache will be flushed. The possibility of dirty-reads does exist in a cluster, which is why caching between transactions is not recommended in a cluster, unless optimistic concurrency can be used or the EJB is read-only. However, this being a single server submission, the use of exclusive concurrency prevents any dirty reads. Exclusive concurrency can result in performance bottlenecks because there is only one instance of an EJB per PK and access by concurrent transactions is serialized through it, but for beans that are not heavily used, it is a viable option.

relationship caching..

My understanding is that this option will allow foreign-key lookups to be accomplished from a PK cache (see above.)Net result: Very effective use of Out-Of-The-Box caching features with the CMP container in Weblogic, but the trade-off includes the potential for dirty reads, which will increase the number of failed transactions in a typical application depending on the degree of natural isolation of the per-user data sets.

This feature allows for the container to load related beans using a single SQL statement and does not imply cache lookups. The term "caching" in the name of the feature implies pre-loading related beans from the database when a finder is invoked.


Arunabh Hazarika
BEA Systems

Hi Arunabh,

Thanks for the clarifications :-)

Peace,

Cameron Purdy
Tangosol, Inc.
Coherence: Shared Memories for J2EE Clusters

Hi Arunabh,Thanks for the clarifications :-)Peace,Cameron PurdyTangosol, Inc.Coherence: Shared Memories for J2EE Clusters

You're welcome. :-). Another source for tuning parameters for WebLogic Server are the SPECjAppServer submissions our partners and we have made over the years. What is lacking in those submissions though, is an explanation of how the values for those parameters were arrived at. Mihir's post should explain to some extent the major ones that were used.

Arunabh Hazarika
BEA Systems

It's worth pointing out _how_ the number of hits was reduced.

When we analyzed the TORPEDO application, we found that the submission made by TMC was on WebLogic Server 8.1 Service Pack 2, but it used the deployment descriptors (weblogic-ejb-jar.xml and weblogic-cmp-rdbms-jar.xml) from WebLogic Server 7.0. With the 70 descriptors, batched database operations are not enabled by default. They have to be enabled by explicitly setting the values to true in the weblogic-cmp-rdbms-jar.xml descriptor.
We changed the deployment descriptors to point to the 810 DTD which enable batched database operations and ordered database operations by default. With batching turned on, multiple entries in the database table are updated in one batch of SQL statements. Also, the operations are ordered to take care of database constraints. This reduced the Database hits for the placeBid and place2Bids TORPEDO operations to 3 and 5 respectively.

Further, the TORPEDO application was tuned to use a WebLogic specific feature called Relationship Caching. This feature improves the performance of entity beans by loading related beans into the cache. Multiple queries to load related beans are prevented by the use of a join query.
 
A good use-case for using this pre-fetching strategy is when you have a set of beans that are accessed as part of the same unit of work i.e., in the same transaction. In such a scenario, if the related beans are loaded into the cache at the same time, it prevents further database hits to load the related beans.
 
The TORPEDO application fits this use-case. The TMC submission made use of the relationship-caching feature for findAllAuctions, findByPrimaryKey operations on the AuctionBean.
On analyzing the application, we felt that we could reduce the Database hits by using this eager caching feature for some more operations. We tuned the application to use relationship-caching for some additional finders.
For example, consider the TORPEDO operation – placeBid.
The placeBid operation is as follows:
User user = persistentAuctions.getUser(userID);
Auction auction = persistentAuctions.getAuction(auctionID);
Bid newBid = persistentAuctions.createBid(bidID,auction,user,amount,maxAmount);

The getUser(userId) operation translates into getUserHome().findByPrimaryKey(ID)
Similarly, the getAuction(auctionId) operation translates into getAuctionHome().findByPrimaryKey(auctionID)
 
With no relationship caching, a SQL query is issued by the first line of code to load the UserBean and another SQL query is issued by the second line of code to load the AuctionBean thus resulting in two hits to the database.
With relationship caching configured for the UserBean’s findByPrimaryKey method, a single query is issued to load both UserBean and the related AuctionBean by the first line of code. As the related AuctionBean was cached in the same transaction, the getAuction call fetches the AuctionBean from the cache. This resulted in one less Database hit for the placeBid operation.

A similar optimization was done by configuring relationship-caching for the findHighBidsForAuction operation on the BidBean to reduce the DB hits for the operation to 2.
Basically, if you know that a related bean will be accessed after executing a particular finder method, it is a good idea to let the finder method know this via the relationship-caching feature.

Mihir Kulkarni
BEA Systems

Hi Mihir - That's exactly the kind of detail I was hoping for! Some of it was posted with the submission:

http://www.middlewareresearch.com/torpedo/results/weblogic-3/report.jsp

I wish we'd see more detail like this though on all the submissions ..

Peace,

Cameron Purdy
Tangosol, Inc.
Coherence: Shared Memories for J2EE Clusters

Hi Mihir - That's exactly the kind of detail I was hoping for!

thanks. I agree with you that each submission should accompany details such as these. The vendors provide the tuning knobs but what tuning is necessary can only be determined after analysing the application and the criteria you are trying to meet (reduced DB hits/higher throughput/reduced memory usage etc)

Mihir Kulkarni
BEA Systems

It is a single server submission

That can allow db-is-shared to be set to false, which caches EJBs by their PKs, which reduces hits for ejbFindByPrimaryKey and ejbLoad

caching between transactions

That suppresses cache flushes at transactional boundaries, allowing for potentially dirty reads (although if db-is-shared is set, that is the case anyway.)

This is an interesting benchmark. This benchmark requires user to restart the server between each testcase, and each testcase is a single transaction. Although we can set db-is-shared to false, and turn on cache between transactions, it is won't have any impact on the result. You will also get the same result if you run this benchmark in the cluster or single server.

Michael

I claim top spot too (2 hits):
1. Load database to memory on startup.
2. Save memory database on shutdown.