Can JIT'ed Code be Faster than Hardware Accelleration


News: Can JIT'ed Code be Faster than Hardware Accelleration

  1. Ever wonder what your code looks like once it has been digested by the javac compiler or the JIT? In his blog Mark Lam shows you all of this in the process of explaining how adaptive code can be faster than Java-specific processors. The article starts with a description of hardware acceleration and how it has been helpful. For example, co-processors, processors for sound and graphics have proven to be quite useful in accelerating applications. Mark asserts that the success experienced in these specific cases has lead to the misconception that hardware acceleration can match the performance provided by a JIT. To demonstrate just how a JIT can out perform hardware acceleration, Mark does an analysis of what happens when one executes a = a + b;. First, the byte code and then the assembler is listed just to show what exactly takes place. With these listings in place, Mark simply counts the number of instructions that both the accelerator and the JIT will execute. The assumption is that fewer instructions will execute in less time. There are mitigating factors.
    Based on the StrongARM 1110 processor, under optimal conditions where all accessed memory is already in cache, each of the instructions will take at least one machine cycle (if I remember correctly). If there is a cache miss, memory accesses can take up to 4 machine cycles assuming no virtual memory paging stalls. Otherwise, it'll take a lot more time.
    Using a number of assumptions Mark proceeds to calculate the number of machine cycles needed to complete the addition. The results, summarized in a table, clearly demonstrate that the general purpose CPU with a JIT consistently out performed hardware acceleration. Even though Mark recognizes that one may never realize the the speedups one as high as 83x), there is a recognizable performance boost. Mark moves on to demonstrate how the situation is even better for JITs when one considers complex bytecodes, inlining and other tricks employed to optimize code. With all of these advantages one would begin to wonder if it is worth it to consider hardware accelerators for Java.
    Regardless, the JPU does have value. While it cannot compete with a JIT in terms of performance, it will perform much better than a software interpreter. Another reason to use the JPU is for extremely memory constraint environments where you have absolutely no memory budget to spare. The JIT adds 10s to 100s of kilobytes of memory consumption. If this is not a cost that can be tolerated, a JPU may be the best solution for performance.
    There are reasons to be hopeful that Java processors may become useful outside of a few special cases. Mark ends by postulating that with an advanced Java processor there are some possibilities of the situation changing. While it may not speak for all cases, it will help in a number of common cases.

    Threaded Messages (5)

  2. Today's JIT compiler from Sun is incredibly fast, once it kicks in it flys, especially in JSE 6. C24 have done a lot of benchmarks, recently for Intel we were quoted in the release of their new 4 core CPU. We've done a lot of bench marking for Sun on AMD (although I have to say Intel's currently in the lead there). I'm in the process of working with Azul and getting some amazing figures on their new Vega2 CPUs (all 192 cores!). When it come to hardware, I can't go into too much detail but let's just say we're speaking to Intel and DataPower and we could perhaps add Azul to the group too. I've seen what hardware can do and believe me it's pretty damned impressive. One of the big areas being investigated is serialization, that's still something that the fastest JIT still can't do much about. My point? JIT and hardware are two different things, we need both to get the most out of a JVM. We're going to see some impressive things from both JIT and hardware in the coming months. -John- CTO, C24 London, UK
  3. Absolutely true...[ Go to top ]

    Anything that can be done in software can be done in hardware. There's nothing in his article proving that a hardware java cpu has to be slower than one in software. Having said that, the cost and time of development issues tend to always favor general purpose CPUs for peak performance.
  4. Re: Absolutely true...[ Go to top ]

    exactly. in fact, the limits of the software are the underlying hardware. with a well designed cpu, at the software layer, the best you can do is to leverage it. most of the time, unfortunately, bottlenecks are proposed...
  5. Hear here![ Go to top ]

    I could not agree more with pretty much every comment that has been posted in this thread. Since John mentioned the performance success he has experienced with Azul (thanks!) I thought it might be interesting to illuminate where the software/hardware boundary of our product is...
  6. Symbolics of Cambridge -- then Burlington MA, used to make a LISP machine including an operating system etc. on top of a CPU specifically designed to run LISP. Just got a sense of Deja Vu while reading about a Java CPU.