News: Reducing Garbage Collection Times and Sizing Memory With JDK 1.4
Understanding the behavior of the garbage collector is important for enterprise applications where delays of seconds or hundreds of milliseconds are outside of acceptable levels of service. A new article from Sun describes the new low pause and throughput garbage collectors available in JDK 1.4.1, makes performance suggestions and describes a new tool, GC Analyzer.
- Posted by: Nagendra Nagarajayya
- Posted on: December 05 2002 23:41 EST
Read Improving Java Application Performance and Scalability by Reducing Garbage Collection Times and Sizing Memory Using JDK 1.4.1.
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Manish Sharan on December 06 2002 13:09 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Krishnan Subramanian on December 09 2002 08:11 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Nagendra Nagarajayya on December 09 2002 13:35 EST
Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Nagendra Nagarajayya on December 09 2002 01:46 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Manish Sharan on December 09 2002 11:43 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Nagendra Nagarajayya on December 09 2002 01:46 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Ted Sfikas on December 09 2002 12:30 EST
- Reducing Garbage Collection Times and Sizing Memory With JDK 1.4 by Nagendra Nagarajayya on December 10 2002 00:02 EST
- setting xmx size has low upper limit on x86 (linux, windows) by cal racey on May 20 2003 07:25 EDT
Good Stuff!! But more information is needed. Some sort of guidelines to calculate the JVM -X values for a sample J2EE application would be great !
I am struggling with an MDB that is processes an XML Blob -- which can range in size from 1k to 200mb. And I am going nuts trying to figure out how to calculate my -x params and how much RAM is required.
This document is a great start!!
Usually, the easiest way to check up on GC activity is to run your AppServer with the verbose:gc flag turned on (JVM param). The above flag would give you output from the garbage collector for both minor and major garbage collector runs. Major garbage runs (atleast in Sun's 1.3.x VM) results in the VM suspending all threads while the GC runs though the heap. Minor GC runs usually run in parallel (background thread). So, that should make it quite apparent that major GC runs should be avoided at all costs. Or rather, major GC runs if they do occur - should occur infrequently and for short durations only.
The heap sizes you specify have an impact on how often the GC performs a minor or major run. So, the goal of tuning the heap size should be to set a heap size that minimizes the frequency & duration of GC runs while maximizing the throughput & response times of your application.
This is availalble from the analyzer but not in a direct way. I assume from your query, -x means -Xmx. -Xmx controls the maximum size allocated to heap. This includes the young generation and old generation.
The way you would calculate the -Xmx would be based on how much physical RAM you have on the server, and how much can you spare for the application. Your application should be sized to be less than the physical RAM. (see sections 15 - 16.3 for sizing your memory)
Here is a way to calculate your RAM requirements.
1. Find out the total size of objects created / active duration. This is given under active duration calcs. This is made up of short term and long term data. short term data is data that dies in the young generation, while long term data is data that dies in the older generation. Active duration in the script and paper is defined as the time a call-setup is active. This can also be viewed as the duration a transaction is active. (In your case, how long will the XML data be active ?) (try changing this from 32000 ms to 100000 ms or more)
2. "Number# of promotions in active duration" line gives you the number of promotion GCs. (this gives you the long term data)
3. Percent% long lived, and short lived, give you the ratio of your short-term/long term data. (Long term data is that gets promoted to the older, see 1).
4. Ratio of live to freed data, gives you the data that is freed when a old generation collection takes place.
This can be used to size the young and old generation. Suppose you start with a small young generation, and a small old generation, let us say young generation of about 32 MB, and old generation of about 400-500 MB. Take a look at the long term%, and see if it is high. High here would be anything more than 10-15%. See if you can bring this down to less than 5%. This can be done by increasing the young generation. Increasing the size of young generation increases the size of the short term data, while decreasing long term data.
With the low pause collectors, young generation needs to be small while old generation needs to be bigger (64MB young generation, 500-700 MB old generation). With the throughput
collectors, this can be bigger young generation, smaller older generation. Since yours is an XML processing module, you could use either of the collectors.
Most important, make sure that GC serial percentage is very low < 10%, ideally less than 5%.
Also, watch for Application Execution Efficiency. This has a value between 1-100, and should be as high as possible. This translates direclty to the CPU utilization you should see.
Thank you all for excellent information!
Thank you so much for posting this Nagendra, it is much appreciated.
Would you happen to know of any more articles like it? I am in the middle of load testing a Java application, and so I value this information very much.
The reference section has links to some good papers on GC.
Here are some additional articles :
in "4.3.2 Using the Throughput Collectors " you say
"For 32 bit usage: java -server -Xms3072m -Xmx3072m -XX:NewSize=2560m \
-XX:MaxNewSize=2560m XX:SurvivorRatio=2 \
I assume this is on a sparc system? I keep running up against a problem that max heap (Xmx) can't exceed 1960 meg. I have tested this with suns jdk1.4.1_02, blackdown 1.4.1, jrockit 8 and IBMs 1.4.0.
For an explanation see http://developer.java.sun.com/developer/bugParade/bugs/4435069.html
Has anyone found a way round this?
Most documents relate to jdk1.3 but the problem appears to be the same in jdk1.4+