Java Development News:

The Mac as a Java Development and Execution Platform

By Eugene Ciurana

01 Jul 2001 | TheServerSide.com

Many of my friends and I use some kind of Apple Mac for our work, whether they're portables, desktop or server systems. When we're asked why, our response tends to be something like "the machines are way cool, and they run Java better."

Nobody can deny that Apple manufactures very handsome machines but there are endless debates about whether they perform better than equivalent machines running Linux or Windows. The introduction of Intel-based systems exacerbated the debates. This, coupled with a work-related project in which I get to test the performance of various servers running Java, moved me to try some of my benchmarks on several Mac models, and to compare them against other Intel- and AMD-based systems.

Are you ready to find out if Macs are really the best machines running Java when it comes to performance? Read on!

The Tests

Benchmarks are meaningless when comparing two or more systems' performance unless the same benchmark is applied to all. The tester must define meaningful metrics as well. The tests for this article were designed to compare execution speed in three categories:

  • Java involving only primitive types and array manipulation
  • Java involving only complex objects manipulation (autoboxing and collections)
  • Native manipulation of primitives and arrays

This test has two parts: a program that generates a sequence of pseudo-random integers, and a program that sorts them using a non-recursive quicksort algorithm. This quicksort was chosen because it performs better than Java's built-in sort in the collection classes, and it's known to put a microprocessor through its paces. All the programs implement the same algorithm. Listing 1 shows that the C and Java primitive versions are identical except for the method signature vs. function prototype.

Listing 1:

The pseudo-random program generates identical sequences based on the same seed; every system tested operated on the same data as the others.

All Java code was executed under the Java 5 VM, version 1.5.0_06. Other than the Xms and Xmx switches, no other optimization was forced at execution time on any system.

An additional test was added: Rená Grothmann's SieveBits program. SieveBits finds all the prime numbers between 0 and 99,999,999 and clocks the process. As in the quicksort test, the faster the program completes, the better the system's performance.

To execute the tests in your own hardware and compare results, access this zip file (7KB).

The Systems

The tests were carried out on eight different systems. The guiding principles for selecting them were:

  1. Provide "roughly equivalent" configurations; the machines had at least 1 GB RAM and run a 2.0 GHz or faster processor
  2. The systems should be available for more than one pass with the test suite
  3. Where possible, the machines should run the test alongside other work they'd normally execute

The results from an early test motivated me to add one more system, a PowerPC G4, as a point of comparison. After all, PowerBooks have as much a reputation for sleekness and performance as any other Mac.

Table 1 shows the system configurations used for the test.

Table 1:

ModelProcessorRAMOperating system
MacBook blackCore Duo 2.0 GHz2 GBOS X
XservePowerPC G5 2.3 GHz2 GBOS X Server
iMacCore Duo 2.0 GHz2 GBOS X
MacBook ProCore Duo 2.0 GHz
Core Duo 2.13 GHz
2 GBOS X
PowerBook 12"PowerPC G4 1.33 GHz1.25 GB RAMOS X
AMD Server (no model specified)AMD Athlon 2 GHz2 GB RAMLinux
Dell Inspiron 9300Core Duo 2.13 GHz2 GB RAMWindows XP
MacBook ProCore Duo 2.0 GHz2 GBWindows XP

When reading the specs, the only differences between the MacBook and the MacBook Pro 15" appear to be the screen size, video adapter and some aesthetics. If those are the only differences...do both machines show equivalent performance during the tests? Let's find out next.

The Results

There were many surprises during the test. An image speaks a thousand words...feel free to glance at the charts, assimilate the surprises, and then return to the article for explanation and context.

The Spike

The Java primitives benchmark running on a MacBook showed a weird spike during the first pass. This anomaly in performance seriously handicapped the MacBook when compared against the other systems. The first assumption was that the spike was caused by unusual activity during that portion of the test (Screen server kicking in? The launchd process running in the background?), but it was easy to recreate, as shown in chart 1.

Chart 1

Click for screen shot

The original benchmark was executed under OS X 10.4.6; there is no explanation of why the spike occurred every time that the Java primitives memory usage was at around 50 MB. Apple released the 10.4.7 update around the time of the test. Chart 2 shows the dramatic performance increase after upgrading the same MacBook.

Chart 2

Click for screen shot

Something was obviously wrong with Java primitive handling when the JVM was using about 50 MB on a MacBook in OS X 10.4.6! Apple declined to comment about exactly what (of relevance for this test) changed in the upgrade. This spike only showed on MacBooks, with the MacBook Pros showing a flatter line all along. The test for chart no. 2 was executed several times with a flatter line on the MacBook. Because of this, all the Apple machines were upgraded to OS X 10.4.7 for the final tests under the assumption that other relevant tweaks and patches were available for all Mac models.

The Java Primitives Manipulation Results

Chart 3 compares all eight configurations running the primitives manipulation test. The most surprising result was how slow the MacBook still was, even after installing the OS X upgrade, than all the other systems. In Java terms, the MacBook is slower than a PowerBook G4 with half of the memory!

Chart 3

Click for screen shot

The fastest systems for running this test were the MacBook Pro and the Xserve G5. Their curves track one another very closely. To the chagrin of Apple fans everywhere, though, the MacBook Pro hardware performs best in this test when the machine is running Windows XP, not OS X. Xserve seems to have many nifty optimizations in place, allowing it to outperform all the other machines by a nice margin.

The Java Boxed Objects Manipulation Results

The non-recursive quicksort benchmark program was written originally for comparing the speed of Arrays.sort(int[] a) against it (the quicksort is faster than Java's). Later on, while preparing some Java 5 professional training materials, I rewrote it using only boxed Integers and collection objects to show the performance hit between using boxed vs. primitives; the same exact operations take 10 to 13 times longer when working exclusively with boxed objects.

The Athlon-based server in chart 4 performed the worst for intensive object manipulations, followed by the MacBook and the Xserve, which have almost identical performance curves. The lines for the latter are so close that I had to double- and triple-check the data to ensure that they were from the two distinct machines. The MacBook Pro and iMac have more or less the same performance, with the MacBook Pro on WinXP edging everyone else out of the competition, including the Dell Inspiron.

Chart 4

Click for screen shot

The Sieve Results

An article about performance benchmarking isn't complete unless a Sieve test is included. It's a useful clichá, much like the ever-present cloud in network diagrams. Rená Grothmann's SieveBits test was chosen mostly for expediency; the source is available for download from Dr. Grothmann's site (http://mathsrv.ku-eichstaett.de/MGF/homes/grothmann/java/bench/).

As chart 5 shows, there weren't abysmal differences in performance between all the 2.0 GHz+ Intel or AMD systems; they all took between 8 and 13 seconds to complete. The Xserve didn't perform as well in this test, when compared to its dazzling performance in manipulating primitives and arrays. The operations performed by the sieve are equivalent to those in the quicksort...except that, in this case, the methods operating on the bit set are all synchronized. Perhaps the G5 JVM has less-efficient synchronization mechanisms?

Chart 5

Click for screen shot

The C Code Results

Somehow the results for Java performance weren't dazzling on any of the Macs. From my own experience, it seemed that the MacBook does some things blazingly fast while others are very sluggish. After conferring with the other people who donated system time for these tests, we agreed that a closer-to-the-metal test might be a good idea. Since the test involved the same algorithms as the Java test, it would provide a good baseline for interpreting the results between systems, and for comparing Java performance against native code.

The native quicksort benchmark was compiled using gcc in all the environments (Cygwin for the Windows machines). The Dell Inspiron 9300 was not included in this test due to a time conflict and lack of a gcc compiler, but Windows XP was represented in a MacBook Pro. The results were as unbiased as possible by compiling the code without any optimization options. The only handicap would be the quality of the code generated by each implementation.

All the Intel-based Macs performed at about the same level, with Windows XP on Mac giving the best performance (see chart 6). The WinXP line shows it as being not only the fastest system but also the one with the most "linear" execution. All the systems running OS X were more or less linear but present small peaks and valleys not present in Windows XP.

Chart 6

Click for screen shot

Are MacBook Configurations Equivalent?

Apple offers the MacBook and MacBook Pro 15" with a 2 GHz processor option. Earlier results running under different revisions of the operating system showed that there was something different between both machines' performance and Java. The differences could be attributed to different bus architecture along with operating system bugs. Both systems, running under the same revision of OS X and native code, perform head-to-head until the number of operations increases. In the long run, a MacBook Pro running OS X appears to be between 5% and 9% faster than a MacBook with the same amount of memory (i.e. the "equivalent" configuration, see chart 7).

Chart 7

Click for screen shot

A more interesting question is posed by the results in chart 8. If raw performance is the goal, what's the best operating system to use on a MacBook? The overall results for primitive data manipulation, whether native or Java, indicate that Windows XP might be the best operating system paring. This isn't the kind of news that we Mac folks like to hear, but that's what the results show so far. With history as a guide, and considering the 10.4.6 to 10.4.7 improvements, perhaps it's only a matter of time before OS X optimizations leave Windows XP on Intel processors in the dust. For now, however, it's important to consider that Microsoft has had over 20 years of experience with Intel processors, and that Sun's JIT optimizations for Windows are possibly some of the best there are.

Chart 8

Click for screen shot

Conclusion

There is no definitive answer when it comes to system performance. The answer depends more on what the tester is looking for than on some definitive metric. The goal of this article is to show that, considering equivalent metrics and similar system configurations, different systems can perform better than others due to a variety of factors. Garbage collection, a common cause of tuning headaches, was not optimized for these tests and might yield different results for the Java boxed objects sort. Windows XP wasn't optimized for thread performance. These tests were aimed at processor and memory performance, discarding other factors. Choosing a given configuration is an exercise in testing and choosing the appropriate metrics, not one of choosing the best brand, fastest processor, or the coolest machine.

Thanks to Geert Bevin, Jason Essington, Mable Flickinger, Andrew Lombardi, Joseph Ottinger, and Chris Woods for their help in the production of this article.

About the Author

Eugene Ciurana is the Director of Platform Technologies for Walmart.com Global and member of the Wal-Mart Stores Technology Council. He's been working on Java technologies for a bit over ten years, and has programmed most models of Apple computers since the Apple ][+. Eugene is the author of the action thriller "The Tesla Testament," available worldwide in October 2006. He can be found in various Java IRC channels under the /nick pr3d4t0r.

Related Resources