The developers are all aware that they can limit the heap consumption by specifying the -Xmx switch for a JVM. But besides heap there are other regions in memory which your application is using under the hood – namely permgen and stack sizes. So in order to limit those you should also specify the -XX:MaxPermSize and -Xss options respectively. In a short, you can predict your application memory usage with the following formula 

Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss] 

But besides the memory consumed by your application, the JVM itself also needs some elbow room. The need for it derives from several different reasons:

• Garbage collection. As you might recall, Java is a garbage collected language. In order for the garbage collector to know which objects are eligible for collection, it needs to keep track of the object graphs. So this is one part of the memory lost for this internal bookkeeping. Especially G1 is known for its excessive appetite for additional memory, so be aware of this. 
• JIT optimization. Java Virtual Machine optimizes the code during the runtime. Again, to know which parts to optimize it needs to keep track of the execution of certain code parts. So again, you are going to lose memory. 
• Off-heap allocations. If you happen to use off-heap memory, for example while using direct or mapped ByteBuffers yourself or via some clever 3rd party API then voila – you are extending your heap to something you actually cannot control via JVM configuration. 
• JNI code. When you are using native code for example in the format of Type 2 database drivers then again, you are loading code in the native memory. 
• Metaspace. If you are an early adopter of Java 8, you are using metaspace instead of the good old permgen. This is unlimited and in a native part of the JVM. 

You can end up using memory for other reasons than listed above as well, but I hope I managed to convince you that there is a significant amount of memory eaten up by the JVM internals. If you are interested in how to measure the different memory allocations, read the original post.

Keep in mind that the Hotspot JVM is a C program. When you start a program (such as one written in C, like "ls" or "cat" or "grep" or "java"), the OS is going to allocate memory for that program. Then, if the program needs more memory to store its data, the program will ask the operating system to allocate more memory for it. Some of that memory is used for "stack" data structures for each thread, some is used by the operating system to store the program itself in memory, and most of the memory is used by the program to store its data, i.e. the data structures that it uses.

Hotspot, as a program, uses all of the above. One of the data structures that it allocates memory for is referred to as "the Java heap". That particular data structure is the one (and the only one) that is limited in size by the "-Xmx" setting.

Peace,

Cameron Purdy | Oracle

The opinions and views expressed in this post are my own, and do not necessarily reflect the opinions or views of my employer.