Discussions

News: SAP Memory Analyzer 1.1 Released!

  1. SAP Memory Analyzer 1.1 Released! (22 messages)

    SAP has released version 1.1 of the Memory Analyzer, a heap dump analyzer. The tool is free of charge and available for various platforms including Linux, Mac and even Windows. JavaPosse has rated it Developer tool of the week in June. It's in no way bound to the SAP NetWeaver Java stack – it's a general Java developer tool everybody can use to track down memory leaks or too high memory consumption. The developers have shown the tool at JavaOne and many other conferences this year and got great feedback. The next public demo is this week at the Eclipse Summit Europe 2007 from October 10 - 11 in Ludwigsburg, Germany. New & Noteworthy
    • Retained Size Approximation - Approximate the retained size on arbitrary objects. The calculation is about 10.000 times faster than computing the precise retained size and presents a lower bound for the retained size. The margin of error is less than 3 to 5 percent. That's good enough to spot the big memory chunks and it is cheap enough to use it on every heap view to assess its impact on the heap.
    • Queries - predefined inspections on the heap dump to analyze collections, hash maps, arrays or group object sets by arbitrary attributes.
    • Group By Value - group objects by their value of a primitive field or any object reference. For example, this way you can quickly find hash maps which have never been modified (group by modificationCount).
    • Local Variables - list variables which are kept alive by a currently running thread.
    • Finalizer - analyze objects on the finalizer queue.
    • Free Download as Eclipse Plug-in or as Binaries for SWT supported platforms (Linux, Mac, Windows, etc.)

    Threaded Messages (22)

  2. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    It's nice to see such tools around here.. But, I think one important thing you should mention in the first page is missing... JDK's which can be monitored... I think IBM JDK is widely used especially in corporate solutions... And such restriction puts us in the IBM's tender hands...
  3. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    It's nice to see such tools around here.. But, I think one important thing you should mention in the first page is missing... JDK's which can be monitored...
    This is not a monitoring tool, but a heap dump analyis tool. But you are right, the tool "only" supports heap dumps from SUN and SAP VM's. That is because those heap dump formats are well documented and completely open. This is as far as I know not the case with the IBM heap dump format. In addition the IBM Heap dumps contain not the same information as the SUN Heap dumps. So it's not that simple to get the tool running with IBM Heap dumps (this has been discussed at server side before).

    I think IBM JDK is widely used especially in corporate solutions... And such restriction puts us in the IBM's tender hands...
    Regards, Markus
  4. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    This is not a monitoring tool, but a heap dump analyis tool.
    But you are right, the tool "only" supports heap dumps from SUN and SAP VM's.
    That is because those heap dump formats are well documented and completely open. This is as far as I know not the case with the IBM heap dump format. In addition the IBM Heap dumps contain not the same information as the SUN Heap dumps. So it's not that simple to get the tool running with IBM Heap dumps (this has been discussed at server side before).
    Sorry, my mistake, I may seem to define the tool as a monitoring tool.. But be sure I know this is a heap dump analysis tool :) What I mean is about the restriction of environments we use, and about not feeling free to use such tools :))
  5. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    Well, the great thing about heap dumps is actually that they give you a lot of freedom: You don't have to fiddle around with a Profiler and plug it in. A luxury you can't afford on productive environments anyway. Just configure the right parameter and you get a heap dump when your system dies with OOM - or when you actively request one. This is great freedom. That IBM is not supported is not the fault of the tool. IBM has its own strategies with their Diagnostic Tooling Framework for Java (DTFJ) which is kind of competing with JVMTI or the old HPROF. So it's them who walk on their proprietary road and a tool can't possibly support everything on the market. However, if we would some day see a converter for IBM system dumps to HRPOF binary heap dumps, this would be a great relief, as you are right and many productive installations run on IBM platforms. Are you writing one? ;-) Vedran
  6. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    couldn't help myself here, there are tools from the IBM site that do the same, but otherwise, I would not say that their hands are so tender ;-)
  7. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    Does anybody know if this tool can analyze binary heap dumps (with switch XX:+HeapDumpOnOutOfMemory)
  8. Does anybody know if this tool can analyze binary heap dumps (with switch XX:+HeapDumpOnOutOfMemory)
    Sure it does. Regards, Markus(https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/u/6389)
  9. Binary Heap Dumps[ Go to top ]

    Yeah, the tool operates exactly on those HPROF binary heap dumps you get with this switch or with JConsole and some other means. The WIKI page lists all the means to get heap dumps!
  10. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    couldn't help myself here, there are tools from the IBM site that do the same, but otherwise, I would not say that their hands are so tender ;-)
    I'm not aware that IBM has a comparable tool, in terms of usability, functionality and performance. Could you tell us which tool you mean ? In fact, I'm not aware of any tool even for any other programming language that offers the same functionality with the same performance. Disclaimer: I might be biased, because was involved in the development of the tool. Regards, Markus (https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/u/6389)
  11. Re: SAP Memory Analyzer 1.1 Released![ Go to top ]

    Hi Jin, I don't think so. IBM hasn't invested a lot in this. Take a look who wrote the tools and you'll see that sometimes the tools are the work of students. That doesn't mean the tools are poor, in the contrary, the IBM guys have been really smart and the tools have some great features, but it seems that these kind of diagnosis tools were not in IBM's focus so the tools did not evolve, e.g. by exploiting the ideas further, incorporating new ones or improving the convenience or performance. Some years ago their tools were much better than anything else on the market, but time has changed, especially with Sun's JVMTI. YourKit is a great profiler and the SAP Memory Analyzer is in my oppinion the by far best Heap Dump analysis tool, as it is extremly fast and capable to isolate memory problems. It's certainly not a dump heap walker. But, if you know of a comparable good tool for the IBM platforms, please let us know, as we urgently would need one and believe me, support is in great pain as they are currently pretty blind. Vedran
  12. The plugin only works Eclipse 3.3[ Go to top ]

    Hi all, Just a hint. The new plugin only works with Eclipse 3.3. Regards, markus
  13. Is this related to YourKit ?[ Go to top ]

    This tool looks great - it is very similar to YourKit with concepts like Shallow and Retained Size or Path From GC Root. Is it a source of inspiration ?
  14. You may also have a look at Objectreferenceanalyser: http://sourceforge.net/projects/refanalyse/ It provides a different approach to analyse your application/module using reflection. It visualizes the object references to help understanding the application and to determinate memory leaks in an other way.
  15. SAP and Java Tool[ Go to top ]

    Not only this tool, SAP also have a number of JAVA/J2EE tools to cover entire SOA framwork. Their Portal, ESB, Registry, Middleware application server, Eclipse based IDE etc are very cool and one important thing I have noticed, they do not seems like lie about their products and capacity like other middleware vendor. I had a situation where I need to get some clarification on their ESB (PI ) product capacity for a large organization about the capacity to handle 1.5 millions transaction, they clearly and sincerly provide the answer that is the good side of SAP's JAVA approach.
  16. As I understand the memory monitory tool should have some more features than SAP Memory Analyzer has. For example to detect a memory leak you should have somethink like a object reference tree. Imagine you detect a memory leak - some hang up objects in memory, how can you find out why they are in heap ? Other very important thing is an object allocation to detect where your programm produce a lot of specific objects. Because you use memory dump which is prodiuced by a SUN JVM you never implement features listed above. SUN JVM heap dump does not have information about object references and allocations.
  17. Hi Victor, You wrote:
    For example to detect a memory leak you should have somethink like a object reference tree. Imagine you detect a memory leak - some hang up objects in memory, how can you find out why they are in heap ?
    Well, this is not really true - the hprof formatted heap dump which is written by the SUN VM contains information about the references from each object. And not only the references, but also the values of the primitive fields (also pretty handy sometimes). So, the full object graph is available. But even then sometimes it is pretty difficult to analyze (e.g. in a heap dump with 20.000.000 or more objects). Therefore we have provided many different features which enable problem analysis and not simply heap browsing. Have a look at the tool, you may find it helpful. About the second point:
    Other very important thing is an object allocation to detect where your programm produce a lot of specific objects.
    Because you use memory dump which is prodiuced by a SUN JVM you never implement features listed above. SUN JVM heap dump does not have information about object references and allocations.
    It is true that there is no allocation information built-in. Allocation data is very helpful, but it also costs performance. The things I like about heap dumps - you get them with no runtime overhead, and you can get them on an OutOfMemoryError automatically. This means, you don't have to be connected with a profiler to the VM and wait for the problem to re-occur. At the end, both "live" profiling and heap dump analysis have their pros and cons. Finally, if you have time - look at my blog: https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/6856 I tried to describe a how one can find a memory leak out of a heap dump. For me the procedure works pretty good. Give it a try :-) Thanks for the feedback. Regards, Krum
  18. Hi Krum,
    Well, this is not really true - the hprof formatted heap dump which is written by the SUN VM contains information about the references from each object.

    Yes hprof makes heap dump with all information, but hprof has one little problem :) it creates heap dump on exit, but to detect a memory leak you should have as minimum 2 heap dumps to compare (before and after test)
    It is true that there is no allocation information built-in. Allocation data is very helpful, but it also costs performance. The things I like about heap dumps - you get them with no runtime overhead, and you can get them on an OutOfMemoryError automatically.

    It would be better to detect memory leaks on test farm, not on production deployment. I understand overhead when you trace object allocation and of course I meant activating object allocation tracing during test.
  19. Hi Victor, you can take heap dumps at any time you want. You don't have to wait for your system to die from OOM. ;-) And in fact we use them in testing exactly like this. Hava a look at the WIKI page. There you'l' see the various means to get heap dumps. Regards, Vedran
  20. As I understand the memory monitory tool should have some more features than SAP Memory Analyzer has.
    For example to detect a memory leak you should have somethink like a object reference tree. Imagine you detect a memory leak - some hang up objects in memory, how can you find out why they are in heap ?
    Other very important thing is an object allocation to detect where your programm produce a lot of specific objects.
    Because you use memory dump which is prodiuced by a SUN JVM you never implement features listed above. SUN JVM heap dump does not have information about object references and allocations.
    Hi Victor, We found that using the retained size measurement is the key to be able to find memory leaks without allocation tracing. (Full) Allocation tracing is usually very expensive at runtime whereas heap dumps don't have a runtime overhead at all. See also Krum's reply. You may want to check my blog at https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/6493 for a description of how the retained size calculation works. The "retained size" was as far as I know, introduced by the (great) Yourkit Profiler. Because we found that this crucial for the memory consumption analysis of applications we have built our tool around this concept. Regards, Markus(https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/u/6389)
  21. Hi Markus.
    We found that using the retained size measurement is the key to be able to find memory leaks without allocation tracing.

    I do not understand how? For example you detect a memory leeak - somewhere in you code you put instance in map for example and forgot to remove it. Without allocation trace you have to check all allocation of this object in any part of your program and it can be not a trivial task if somewhere you use something like Class.forName("BigAndHiddenMemoryLeak.class").newInstance() especially if class name is taken form configuration file :)
  22. Hi Victor, with leak was meant "accumulation point" and if you have such an "accumulation point", you'll see it easily in the dominator tree, for example. Have a look at the tool! It is quite good and was also rated pretty well, e.g. JavaPosse Developer Tool Of The Week Vedran
  23. Hi Markus.

    We found that using the retained size measurement is the key to be able to find memory leaks without allocation tracing.


    I do not understand how? For example you detect a memory leeak - somewhere in you code you put instance in map for example and forgot to remove it. Without allocation trace you have to check all allocation of this object in any part of your program and it can be not a trivial task if somewhere you use something like
    Class.forName("BigAndHiddenMemoryLeak.class").newInstance()
    especially if class name is taken form configuration file :)
    Sure there are corner cases where thedeveloper would not know where something was allocated, but usually in well designed code this will not happen that frequently. It's also less often not a problem, because you can inspect all the contents of the objects using our tool. Usually then it's relatively easy to find out where the object comes from. Knowing where objects were allocated doesn't always help, because the question to answer is "Why is it still there ?" We think we can answer this question usually very quickly. I also strongly believe the allocation tracing is pretty much useless for investigation problems with high memory consumption(not necessary leaks). Regards, Markus