Scalability: Doubling Threads vs. Adding Another JVM

Discussions

Performance and scalability: Scalability: Doubling Threads vs. Adding Another JVM

  1. On Weblogic 6.1-Unix-jre1.3.1 - I'm trying to out whether it makes more sense to simply double the Thread count/Memory or instead start a new weblogic instance on the same physical server to scale.

    The current performance characteristics are that the CPU has low utilization (< 50%) -- threads seemed to be blocked by a Tuxedo Jolt connection pool processing. (Not really a pool.. there is a jolt connection for each thread in both scenarios).

    Whether I double the server execute thread count/memory or add another JVM on the same physical machine, the throughput increase is relatively the same (+- 5% (attributing to environment rather then factual differences) and same CPU utilization).

    The machine has 2 CPU's, so I see an advantage to having the ability for "concurrent" garbage collection with 2 JVM's - as the application is memory intensive (garbage collection every 5-10 seconds).

    The production env. is currently 2 physical machines (2 servers). By having the additional 2 JVM's.. that would make 4 - therefore if a server jvm fails (crashes), that would leave 75% capacity, rather then in the 2 JVM scenario - at 50% capacity.

    In summary, the reasons I can identify for adding the 2 additional JVM's (2 JVM's per physical machine) is that I'll have higher capacity in case of a JVM crash and that garbage collection can occur faster, since each JVM can garbage collect on individual processors.

    Anyone have any thoughts?

    Thanks!

    Threaded Messages (2)

  2. Mario,

    You should try posting your question to BEA's newsgroups as well (http://newsgroups.bea.com), since they have a very active community that often includes the WebLogic developers themselves. You can get insights from people who actually try these things out for a living. This may be especially helpful if it turns out that there's a bottleneck in the Jolt or Tuxedo tiers.

    That said, though, my recommendation would generally be to go with more JVM's. The reason is primarily to alleviate GC. You've indicated that you've already got some issues with GC frequency. Increasing the number of threads would require a simultaneous increase in heap size in order to avoid an increase in GC frequency. The heap increase may hold the frequency steady, but you will likely incur a greater duration because of it. As an example, with a 2GB heap for one of the systems I've worked on, even minor collections took 500-750ms, though they occurred only every 20s or so.

    Also, make sure you check out the GC tuning whitepaper at http://java.sun.com/docs/hotspot/gc. If you're encountering frequent or expensive GC, following these guidelines can sometimes have dramatic effects on performance.

    I typically try to avoid allocating heap sizes greater than 512MB (NewSize/MaxNewSize at 128-170MB, MaxPermSize=128MB) in order to minimize the effects of GC. If I need greater processing power to maximize utilization and throughput, I add instances rather than threads, until everything purrs.

    One other thought, on the reliability aspect you mentioned: Since you'll have multiple nodes on a single box (clustered, I assume), you'll want to make sure that you configure the replication groups appropriately, to avoid a situation where an entire box fails and you lose entire sessions because they were replicated between the two nodes on that box.

    Hope that helps,
    cramer
  3. Thanks[ Go to top ]

    Yes - it is clustered... app is stateless, so no session replication.. running on Tru64 (jdk1.3.1 - 64 bit) - 512 mb ram with (currently) 65 threads.

    Thanks for the feedback!