Every client/server application will have different remoting requirements, but most criteria will include performance. At the very least, you will want to know how much (if any) performance you are sacrificing for the sake of your other requirements. In "Java Remoting: Protocol Benchmarks," Daniel Gredler examined Java's RMI/JRMP, Oracle's ORMI (with and without HTTP tunneling enabled), Spring’s HttpInvoker, Caucho's Hessian, Hessian 2 and Burlap, and three flavors of Apache XML-RPC (Sun-based, HttpClient-based and Lite-based).

The test consisted of remote invocations to a single method which takes an integer size parameter and returns a list of the specified size. The objects in the returned list each contained 10 instance variables (4 strings of about 40 characters each, 2 dates, 1 long, 1 integer, 1 double and 1 float, none of which were null). The lists returned were static, so there was no list construction overhead. The results of the first 10 invocations were thrown away, in order to allow the VMs to "warm up." The next 100 invocations were tallied and averaged. The tests were performed on a Windows XP SP2 machine with 2GB of RAM and a 2Ghz Core 2 Duo CPU, using Sun's JVM 1.5.0_09. The client and server were run in two separate VMs on the same host, in order to avoid spurious networking artifacts. The following library versions were used: Spring 2.0.6, Jetty 6.1.5, OC4J 10.1.3.2.0, slf4j 1.4.3, log4j 1.2.14, Apache XML-RPC 3.1, Commons HttpClient 3.1, and Caucho Hessian 3.1.3. Spring was used on both the clientside and on the serverside in order to hide the remoting details from the interface consumer and implementor. All serverside components were run inside a single Jetty servlet container, except for the ORMI serverside testing components which were run in OC4J.

Two graphs are included: a "small list" call (250 items or less) and a "large list" call(from 500 to 5000 items). The two graphs are consistent (a good sign for the libraries involved) - and (surprise!) the binary protocols did far better on average than the XML-based protocols. His conclusion:

It's important to note that no general conclusions can be derived from the absolute numbers ... Rather, the numbers must be examined relative to each other. That said, the following observations may be made:

1. The binary protocols (RMI, ORMI, HttpInvoker and Hessian) are always faster than the XML-based protocols (Burlap and the Apache XML-RPC variants) — except for ORMI with HTTP tunneling enabled.
2. Performance is pretty even amongst the binary protocols — except for Hessian, which performs well only when compared to the XML-based protocols, and ORMI with HTTP tunneling enabled, which performs on a par with the XML-RPC variants
3. Burlap has much better performance than the XML-RPC variants.
4. Native RMI and Hessian 2 have the best performance until the remote method invocations start returning larger lists, at which point vanilla ORMI takes a slight lead.
5. Changing Apache XML-RPC's transport factory does not seem to have a very large effect on performance.
6. It's amazing how fast standard ORMI is, compared to ORMI with HTTP tunneling enabled.
7. Hessian 2 bears watching!

Great stuff! That said, it would have been interesting to see SOAP thrown in there, just to see a comparison between XML-RPC and SOAP - one imagines that XML-RPC will dance around SOAP, being a lighter protocol, but it'd be good to have the numbers in hand nonetheless.

Great stuff! That said, it would have been interesting to see SOAP thrown in there, just to see a comparison between XML-RPC and SOAP - one imagines that XML-RPC will dance around SOAP, being a lighter protocol, but it'd be good to have the numbers in hand nonetheless.

It would also have been interesting to see Jini's JERI protocols performance being measured alongside JRMP

Great stuff! That said, it would have been interesting to see SOAP thrown in there, just to see a comparison between XML-RPC and SOAP - one imagines that XML-RPC will dance around SOAP, being a lighter protocol, but it'd be good to have the numbers in hand nonetheless.

It would also have been interesting to see Jini's JERI protocols performance being measured alongside JRMP

Hmm, very good point - but then again, JINI sort of has a different call/response mechanism than RPC does, IMO, so it wouldn't quite be an apples-to-apples comparison.

Great stuff! That said, it would have been interesting to see SOAP thrown in there, just to see a comparison between XML-RPC and SOAP - one imagines that XML-RPC will dance around SOAP, being a lighter protocol, but it'd be good to have the numbers in hand nonetheless.

It would also have been interesting to see Jini's JERI protocols performance being measured alongside JRMP

Hmm, very good point - but then again, JINI sort of has a different call/response mechanism than RPC does, IMO, so it wouldn't quite be an apples-to-apples comparison.

IMHO that's not quite right: Jini Services have a different call/response mechanism if you choose to exploit the architectural feature that permits this arrangement. However JERI itself is a plugin replacement for RMI and has the same semantics whilst being more flexible in terms of transport protocols, security etc.

It would be great to see the same benchmark with object graphs. I think that the XML based protocol have slightly different ways to handle that.

If you return static lists containing the same objects (but only different amounts per call) that RMI will send an identifier instead of the entire object? So basically the RMI version would be doing most of the work in the first 10 calls and then only passes lists with ID's instead of entire objects. It's been a while, but I recall something like that. This could also explain why another referred benchmark results in worse RMI performance.

Hi Barry, I haven't been able to find any references to that sort of object caching in RMI/JRMP. However, the variance in response times (for a single list size) suggests that no such caching occurs -- otherwise you would have numbers like max: 5000 ms, min: 100 ms, average: 150 ms. That said, if you do find some documentation that such a mechanism does exist, please let us know!

Hi Barry,

I haven't been able to find any references to that sort of object caching in RMI/JRMP.

However, the variance in response times (for a single list size) suggests that no such caching occurs -- otherwise you would have numbers like max: 5000 ms, min: 100 ms, average: 150 ms.

That said, if you do find some documentation that such a mechanism does exist, please let us know!

Having searched the docs I found that it serializes the first time and then writes a "handle" for the object on subsequent references to this object WITHIN the current writeObject call. So it would appear that these handles are thrown out at a later moment. I guess keeping the objects in sync would be a pain, but I do think that they could have given this more throught in RMI. So in short, you and the other poster are right, the benchmark is valid in the sense that the whole objects are shared on every invocation

So in short, you and the other poster are right, the benchmark is valid in the sense that the whole objects are shared on every invocation.

Cool, thanks for the update!

RMI sends identifiers for Remote objects. Ordinary Serializable objects are sent each time, in full.

As an aside (I realize we are talking Java here), in the days where Flex/Flash front ends are becoming a more popular alternative to front J2EE back-end systems, it would be interesting to see how Adobe's AMF protocol measures up.

Maybe also JBossRemoting is missing to the list ?