Application ClassLoader’s Footprint Memory Leak in Shared Containers

Discussions

News: Application ClassLoader’s Footprint Memory Leak in Shared Containers

  1. Basic Reason for the leak:

    Every time an application is deployed and un-deployed in a container, the application is given its own class loader (application class loader) by the container’s class loader. This application class loader might end up creating a cyclic reference to the container’s class loader, which makes it un-garbage collectible. From my observations getting rid of the application’s class loader from memory (1 to 2 MBs) could be a very difficult and tedious task because of the amount of third party tools we use, but can be contained, although not ideal.


    Avoiding the leak from Crossing Threshold:

    a)      One Main Culprit : Spring Class Based Proxies

    Many Enterprise applications use Spring, AOP and Hibernate as tools for their projects. With Hibernate comes the Session Factory beast in memory. The Hibernate Session Factories can become huge based on the project’s DB tables count and Hibernate Mapping Structure (150MB for the application that I work on). So when Session Factory gets attached to the un-garbage collectible Class Loader which has been left behind, we have huge memory leak in containers causing us losing valuable resources.

    So how does this Session Factory get attached to the un-garbage collectible class loader?

    Spring uses proxies to weave in Advices. Transactions are most commonly “AOP Advised” in Enterprise Applications using Declarative Transaction Management.

    As you might know, there are 2 types of Proxies that Spring creates, based on the implementation of the class that needs to be advised

    1)      Interface Based Proxies – not a problem

    1. Spring uses JDK proxies if there is an interface available for all the advised methods; this is the default type of proxy that is created. A new class is created with methods that needs to be advised, through which advises will be weaved in. A call is then delegated from the new class to the actual class that had to be advised.

    2)      Class Based Proxies – A Problem

    1. Spring uses CGLIB to create proxies if an interface is not available for the existing methods that needs to be advised. This type of Proxy should be an exception in general, and it is only provided for backward compatibility for third party libraries that cannot be modified.
    2. The problem here is CGLIB’s implementation leads it to tie a hard link between the Application Class Loader and the Session Factory when it is trying to advise the method of particular transaction choice.
    3. Hence trying to avoid Class based proxies for Spring Transaction Management, in an application with an existing Class Loader leak, may avoid it from crossing the “Threshold” (2 MB to 154MB in a project that I was working on).

    b)      Some Third Party Libraries:

    There are some third party tools that do the same thing of attaching itself to the class loader and creating a hard link between the runtime classes that they create and the class loader itself. Some examples that I know of right away are Dozer and Apache Common’s Logging.

    So it looks like there is no good way to solve the problem of Application Class Loader’s Footprint leak in a Shared Container turning out of control, hence the intent of this article is to make one aware of the issues and point them in the right direction.


    PermGen Space will be affected as well

    Now this leak may also cause your PermGen Space run out of memory since the meta-inf about the loaded classes may not be garbage collected.

    Krank Keviet has done a pretty good job explaining the issue in one of his blogs with respect to Perm Gen Space, a reference to his blog is below.


    References

    http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java
    J2EE Classloaders Demystified

  2. Another interesting post is this Spring one from 2005.

    It is indeed very difficult to get rid of ClassLoader leaks. A simple (and effective) solution is to host only 1 application per container, and restart the container when deploying a new version, but this is not always possible.

    Note that I am taking care of some known ClassLoader leaks offenders (notably commons-io, MySQL Connector/J, Log4J and more) in my monitoring utility, MessAdmin. While a by-product of its intended usage, it comes quite handy!

  3. You might want to check out Tomcat's new memory leak prevention and detection

     

  4. This problem isn't really specific to servlet containers or ejb containers. Any application that creates classloaders to load components dynamically and reload them "potentially" will see this type of memory leak.

    Ultimately, the developer needs to understand the finer details of classloaders and how to use it properly. Some people think it's trivial, create a classloader and it's done. In practice, making sure there's no leak takes discipline and a solid understanding of how the JVM handles it.

  5. Everyone, Thanks for responses.

    Yes, The problem can be applied to any situation where there are multiple class loaders and a cyclic reference between them makes the "child class loader" un-garbage collectible.

    I have seen it in other places where Web-Start thick clients (container equivalent) have "application-plugins" (web application equivalent), but these are places where you can afford a entire restart of the thick client (container equivalent) when compared to Server Side Application containers (where the leak is way too expensive and would cause multiple application outages).

    Also unfortunately for some enterprises, moving to a single container instance per application could be a resource constraint (licenses, as well as cluster setups, are expensive for containers like Weblogic)  

    Tomcat’s new memory leak prevention seems promising, and I would expect other Application Containers to follow suit as well which could be the ultimate fix, but until then, the verdict would be to Profile your application and know about your memory allocations and garbage collection, especially when you are sharing memory with other applications.

  6. Have a look at OSGI[ Go to top ]

    Actually, this is not an issue for OSGI-based solutions. Every OSGI bundle gets its own classloader that creates class(loading) space which enables real classloading isolation as well as eliminates memory leak problems.

    Have a look at Equinox, Knoppflerfish or Felix.

  7. Use compile-time weaving...[ Go to top ]

    To avoid Spring class proxies, use aspectj compile-time or runtime weaving. No more Spring class proxies and much imporoved performance will be your reward.