One might ask what is SPECjAppServer 2004 and why is it different from the preceding benchmarks. Both are good questions and here is a little background so the benchmark can be put into context.
SPECjAppServer 2004 is a 4th generation of the original ECperf benchmark which was designed to performance test EJBs. SPECjAppServer 2001 was a SPECified version of ECperf and tested EJB 1.1 Beans. SPECjAppServer 2002 quickly followed to provide a performance characterization of EJB 2.0 Beans using new features such as CMRs and EJB 2.0 Local interfaces. These benchmarks were a good testing ground for EJBs but what about Web Based Applications?
SPECjAppServer 2004 took the existing manufacturing domain from 2002 and rewrote the Customer domain as an HTTP interface rather than RMI. This provided a broader set of coverage of J2EE and also provides a more realistic balance of applications developed by users. It now includes JMS for inter-domain communication. I think this benchmark has greater relvance to customers than its predecessors.
Please take some time to review the following documents to better understand the benchmark and its applicability to your situation:
FAQ - http://www.spec.org/jAppServer2004/docs/FAQ.html
Design Document - http://www.spec.org/jAppServer2004/docs/DesignDocument.html
Run Rules - http://www.spec.org/jAppServer2004/docs/RunRules.html