Discussions

News: How we reduced the memory consumption by 20x

  1. The original solution we discovered from the application used XMLBeans to store quite a simplistic data structure. The overhead posed by the model ballooned the size of the 1.3 million instance structure to a way-too-large 1.5GB. After going through several optimization phases, such as converting first to Java Collections, then to Trove Collections, cleaning the redundant data and switching to compressed ordinary object pointers we were able to squeeze the same data into 73MB. 

    Lessons learned:

    • Do not let integration details cross system boundaries 
    • Redundant data will be costly. Remove the redundancy whenever you can. 
    • Primitives are your friends. Know thy tools and learn Trove if you already haven’t 
    • Be aware of the optimization techniques provided by your JVM

    Read more at: http://plumbr.eu/blog/reducing-memory-consumption-by-20x

  2. For some yet-to-understand reason, the URL referencing to the original is broken. The correct URL is http://plumbr.eu/blog/reducing-memory-consumption-by-20x

  3. When writing code, especially code others will be reviewing, take some time to write clean code that's readable. Horrible formatting, poor organization, terrible naming conventions, references to generic types aren't parameterized, no consistency anywhere, the list goes on and on. Regardless of how this code performs or what message you're trying to convey...this code is just plain ugly! 

  4. Can you give a bit more precise samples on what you count to be wrong with the sample code. The only point I tend to agree are the use of generics in the MemoryCounterAgent which we have kept to have the snippet backwards compatible with the 1.4 source we still encounter every once in a blue moon. But could you be more precise of what else is not according to the good coding standards?

  5. Looks like you have introduces a few bugs and than called it an optimization. You have changed the data structures and call it an optimization.

    Following your pattern I can optimize even more:

    public class Person {

        public static final int id = 0;

    }

    If you have fixed badly written code, do not call it an optimization ... please !

  6. I am sorry, could you pinpoint the bugs I might have introduced? And I am afraid your joke about the Person class also goes too deep, I just fail to understand the point here. If I have indeed made a mistake at some point I am all eager to learn about it and correct what has been wrong.