This is an interesting question. I can't answer re the constraints of the ECPerf benchmark itself; it's so long since I've looked at it that I can't even recall it's technical nature.
You may be interested in the benchmark results published in Chapter 15 of J2EE without EJB
, which includes a comparison of local EJBs in two application servers vs Spring declarative tx management, hitting the same database. (Spring was configured to use the app servers connection pool and JTA implementation, so this was essentially a Spring vs EJB performance test.) This code is available in the Spring CVS.
We'd been happy to see more benchmarks involving Spring, as we believe that its performance is very good, and that people have the right to see comparisons. However, of course it's very difficult to set up a benchmark to be fair and representative of the real world.
Of course Spring does not aim to replace the whole J2EE stack. Say you replace EJB with Spring; this will still typically leave the app server responsible for connection pooling and JTA implementation, which are pretty critical for performance. For example, we found in our benchmarks that there was a big difference between the performance of the app servers we used to run our Spring code (although it was less than the difference when we ran the EJBs within the app servers as well).