Discussions

Performance and scalability: Are modern JVMs obsolete?

  1. Are modern JVMs obsolete? (11 messages)

    Hardware world is steadily moving towards 64bit architectures, one can even say - we are almost there.

    Among other things, one important characteristic of 64bit architecture is that it can process way more RAM, several orders of magnitude more (roughly 1,000GB vs 4GB, theoretically).

    Are Java and its JVM implementations ready for this change? How much memory can you realistically use in a Java application? Not much!

    There are several reasons to this, two, probably, most important:
    1) When you begin consuming a lot of RAM, garbage collector needs to do much more work and eventually can kill the CPU.
    2) Usually, especially in web/enterprise applications, you may want to use RAM for caching and usually you run your application in a cluster. Now, imagine caching several dozens or hundreds of Gigabytes in a cluster. "In cluster" means - you have to sync cache, which means - you have to be serializing enormous amounts of data and sending over the network. As we know, Java serialization is not that efficient. Chances are high - the performance of such application will be terrible, due to the serialization+synchronization overhead and instead of cached, faster app you will get an essentially useless application - so slow it will be.

    So, what's the sense in being able to address 1,000GB of RAM if you can not realistically get an efficient use of it?

    Does it mean - Java/JVM needs further fine-tuning specifically in this regard? Or, while the rest of the world will enjoy 1,000GB of RAM, Java applications will have to suffice with 2-3GB?

    What are your thoughts?

    Threaded Messages (11)

  2. Are modern JVMs obsolete?[ Go to top ]

    I'm using the JVM with 7GB (-Xmx7000M) on Linux.
  3. Now, imagine caching several dozens or hundreds of Gigabytes in a cluster. "In cluster" means - you have to sync cache.

    Cameron? Where's the Enterprise Hashmap gone?
  4. Being able to start JVM with -Xmx7000M does not solve the problem. The question was performance and how practical it is, under the conditions mentioned. Do you have cluster? Are you synchronizing that data in the cluster? Is your application read-most or read-write? What is the load it gets? How is the performance?

    Besides, 7GB is of the same order of magnitude as 2-3GB, while the available theoretical RAM is several orders of magnitude more.

    Cameron's (Tangasol's) approach is exactly to avoid the shortcomings of Java/JVM. This gives a lot of credit to Tangasol, but none - to Java.

    In Coherence (at least - in its main implementation), they try not to use a lot of RAM on any single node and physically cluster the actual cached data.

    So, using Coherence, the RAM consumption on any single server is, still, of the order of magnitude of several gigs.
  5. Do you have cluster? Are you synchronizing that data in the cluster? Is your application read-most or read-write? What is the load it gets? How is the performance?

    I can speak to the clustering aspect. While our Coherence software does support synchronizing (what we call "replication"), it also supports partitioning, which means that in a cluster of N servers with a cache of M gigabytes, each server only caches M/N gigabytes (via automatic data load-balancing using a fairness algorithm).

    It is explicitly intended to support read/write data, not just read-only and read-mostly data. In other words, using concurrency control (locking) and/or transactional caching, the application can use the caches as a read/write mechanism.

    In terms of load and performance, we are used for some of the heaviest-loaded sites on the Internet, including some with over 100,000 concurrent users. (Before you think that I am taking credit for those sites, we are simply one piece in what is often a very complicated architecture; in some cases we are the back-bone and in other cases we are the finger nail polish .. ;-)
    Cameron's (Tangasol's) approach is exactly to avoid the shortcomings of Java/JVM. This gives a lot of credit to Tangasol, but none - to Java. In Coherence (at least - in its main implementation), they try not to use a lot of RAM on any single node and physically cluster the actual cached data.

    Right. We can also offload data from the Java heap into NIO buffers, such as 2GB NIO direct buffers. Unfortunately, they are LIMITED TO 2GB :-( .. a terrible design decision IMHO, at least for how we use them.
    So, using Coherence, the RAM consumption on any single server is, still, of the order of magnitude of several gigs.

    A common deployment strategy is to run many "cache server" JVMs, each with .5GB or so of heap, meaning you can get 400MB of cache data on the heap (and maybe another 400MB of redundancy managed in an NIO buffer for failover purposes) without using over 1GB total. The GC pauses are tiny, particularly with any amount of tuning, and the aggregate cache size can be in the tens of gigabytes (because the cache size is the total of what each cluster member manages).

    Regarding the specific issues with long GC pauses, Sun continues to invest a lot of work in it, including with 64-bit JVMs. Also, there are very innovative solutions such as the Azul box, which has no-pause full GCs.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  6. Cameron,

    that's all good and nice and that's why Coherence is one of the rare products, I, personally, truly respect, but - how about the JVM itself? :)

    On the hardware side, amount of addressable RAM increases by orders of magnitude and RAM is getting cheaper. Yet, we try to use as little RAM as possible, not to kill performance. And that when - we really want to cache a lot, because the DB overhead is ever-increasing, due to the increasing complexity of the applications.

    Unless I am missing something big, we have a problem here, don't we?

    You mentioned Sun's investment in GC. That's interesting but seems pretty new and not ready, right? Anything on the serialization/de-serialization front?
  7. I've talked to Sun engineers who claimed that they can get decent GC performance on 50GB heaps now.

    I think that we're going to see diminishing returns, though, b/c Java is designed conceptually around a global heap (conceptually .. since there's actually no "heap" concept in Java, just a concept of "new" and the fact that unreferenced objects will tend to be gc'd over time).

    There may be improvements by using "escape analysis" and thus thread local heaps, but ultimately I think the platform has long-term scaling limitations w.r.t. memory management, at least with the technologies that we have available today.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  8. Are modern JVMs obsolete?[ Go to top ]

    It seems to me that there have been an awful lot of assumptions made here. The argument is being made about using 64-bit servers that are just now hitting the market, 1 Tb of RAM which is far far beyond what most any server can handle and upcoming JVMs that haven't been released yet. Let's see what comes with future releases.
  9. Sorry for being a little late in writing in this thread.

    Whereas, you may be partially right (if we assume GC is never going to improve at all), but I can't agree to the second point of yours,

    1. What is need to serialize the data across the clusters?
    I would rather have an invalidation mechanism which will be implemented on messaging and only thing the cluster needs to take care of publishing the invalidation messages so that individual servers can flush of the cache and repopulate it same way the cache was built at the first place.

    2. One more thing is, have you faced a practical situation so far (or even visualised) where we would be needing that much of RAM to run a Java app? Only situation would be to replicate a huge RDBMS strutcure entirely in the object layer. I doubt by doing that, we'll really be gaining that much, we can have LRU based cache mechanism which would suffice for most practical purposes (at least in today's world)
  10. 1. What is need to serialize the data across the clusters? I would rather have an invalidation mechanism which will be implemented on messaging and only thing the cluster needs to take care of publishing the invalidation messages so that individual servers can flush of the cache and repopulate it same way the cache was built at the first place.

    I can only say this, that the "I would rather" should be "I would like the option to", since there are cases in which each is a better fit to the problem at hand. (We do both in our software for that reason.)
    2. One more thing is, have you faced a practical situation so far (or even visualised) where we would be needing that much of RAM to run a Java app?

    Absolutely. There are a number of problem domains that work with very large data sets, and if Java supported a terabyte heap, you'd have those apps running on Java in a heartbeat.
    Only situation would be to replicate a huge RDBMS strutcure entirely in the object layer.

    No, there are many types of applications that gobble memory. Database data for business apps is relatively small compared to the working sets of various modeling programs, for example. Some risk analysis apps can easily gobble a few hundred gigabytes.
    I doubt by doing that, we'll really be gaining that much, we can have LRU based cache mechanism which would suffice for most practical purposes (at least in today's world)

    Sure, of course! Nobody has revoked the 80/20 rule ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  11. But although in the near future we are going to have pervasive machines with 64-bit processors and a lot of RAM (>100 gigabytes) it doesn't imply that we must launch a single JVM in that machines that uses all that memory.

    For example, in our installation, we have several 8-way machines with 12 gigabytes of RAM, and we have a lot of JVM in this machines with 256-512 megabytes each. I don't see realistic to start a single 12 gigabyte JVM in this kind of hardware, because it just doesn't make sense in our particular scenario.


    Jose R. Huerga
    http://www.terra.es/personal/jrhuerga
  12. Jose,

    does not that sound like a workaround? I mean - would you enjoy doing the same implementation if there was no problem using dozens of GB of memory in JVM?