Article: Using AOP for Application Performance Management

Discussions

News: Article: Using AOP for Application Performance Management

  1. There are many ways to manage and track performance data with Java, which include the use of profiler tools, explicit calls to manage timestamps, or bytecode instrumentation. This article by Srinivas Narayanan, Kamal Govindraj, and Binil Thomas discusses the use of AOP as an addition method to track performance data.
    Compared to other approaches, using the full power of an AOP tool allows for a much higher level of customization and provides greater flexibility. A few of these benefits are illustrated below:
    • The ability to monitor only selected parts of the application instead of limiting them by package or API names. The power of the pointcut definition language can be leveraged to do things such as monitor APIs only in certain execution contexts (using AspectJ features such as cflow), monitor APIs based on parameters and their types, etc.
    • The ability to capture statistics at a higher level of granularity. For instance, it is easy to gather data about how much time was spent in the servlet layer, Struts layer, persistence layer, JDBC layer etc. AOP makes it easy to define these layers through a declarative pointcut definition language.
    • Flexibility in capturing more detailed statistics. With tools that do not expose the power of AOP, users are stuck with the feature set that the tool provides. This makes gathering new statistics almost impossible. For example, some statistics that are very useful to have for APIs are frequency distribution, first and last execution time, avg. time taken excluding the first execution (this is useful because some APIs do a lot of work on the first request such as reading a lot of data and caching it, and including the first execution time in the average frequently does not show a true picture of the time taken for an incremental request). InfraRED provides some of these statistics, but not all. Even if these features are not available in the core product, the use of an AOP tool gives users the power to add such custom advices on their own.
    • The ability to construct call trees linking the various API executions within the application is easily achieved by using AspectJ constructs like cflow.
  2. Spring users should check out its JAMon adapter. JAMon sports nice graphs, can tell you how your business services, DAO methods, queries and view layer invocations are doing... You're limited only to how creative you get with its interceptor (and optional programmatic) implementations.
  3. Good - if short - piece on AOP and performance monitoring. I wish it talked about where AOP is inappropriate. I've been looking lately at Glassfish.org. Glassfish is an open-source project (GPL) of what seems to be an implementation of this kind of AOP technique. I would like to add something like Glassfish to my PushToTest TestMaker open-source test tool to do root cause anlysis. While a performance test is generating concurrent load on an application TestMaker would correlate the Glassfish observations with test cases being run. You could then say "What was going on in my web app while testcase #13 was running?" I'm curous what Server Side readers think of Glassfish? And, you recommend other's I should be looking at? -Frank Cohen http://www.pushtotest.com
  4. Glassfish -> Glassbox[ Go to top ]

    Oops. I meant to type http://www.glassbox.com. Sorry. -Frank
  5. We've been using Glassbox on and off for quite some time. It's a great solution for tracking down specific issues. It knows if your code is using Spring's JdbcTemplate for example and as such, is capable of pinpointing exactly where problems are (data access operations, et cetera). We usually include a demo of Glassbox in our AOP training where people are usually pretty impressed with it.
  6. You may want to take a look at InfraRed, which is referenced in the article (http://infrared.sf.net). It is also an open-source performance management tool. -Sandeep
  7. You may want to take a look at InfraRed, which is referenced in the article (http://infrared.sf.net).

    It is also an open-source performance management tool.
    -Sandeep
    Infrared looked very promising when I first read about it. However, when I tried to get it running on OC4J for an application that makes use of EJBs (MDBs only) and AspectJ the results were very discouraging. 1. Infrared supports only certain application servers were OC4J is not part of it 2. Only in the web container I was more or less able to integrate infrared. Integration in the ejb container was impossible. 3. Because our app already used AspectJ, I got lots of conflicts with classloaders and the like when using infrared's AspectJ instrumentation facilities. Using aspectwerkz worked better except for the ejb part. 4. In order to get the gui running I had to unpack the infrared war and replace a JSP... My conclusion about infrared was that it 'might' nicely work when you have a non-ejb, non-aspectJ app running on an open source server. For more sophisticated apps running on commercial application servers it's not ready yet.
  8. AOP != APM[ Go to top ]

    AOP is an useful instrumentation technology for a performance management vendor especially when the vendor ships with their product an aspect library that can extended by customers who are comfortable with the technology. But no sane APM vendor would based their complete product instrumentation design on AOP unless they were basically an AOP advocate desperately looking for a problem to solve and ignorant of the problem domain and competition. AOP + System.currentTimeMillis() does ensure effective application performance management. Likewise AOP + "a house of cards" does not create a performance management product. Instrumentation is just one of the many areas considered in designing and developing a comprehensive APM solution. Underneath all the AOP there must exist runtime services (accessible via open API's) that do actually work and do it with minimal overhead whilst collecting a large amount of useful contextual information. regards, William
  9. Re: AOP != APM[ Go to top ]

    Before anyone accuses me of bashing AOP please check out the following blog entries which show how AOP can be used effectively and responsibly within a performance management solution whilst addressing the real-world deployment concerns of operations strangely enough not mentioned in the article or available in any the solutions mentioned above. JXInsight 5.5 and AspectJ - Sample Application http://blog.jinspired.com/?p=132 Speeding up AspectJ Load-time Weaving http://blog.jinspired.com/?p=133 Runtime Inspection of AspectJ Concrete Aspects http://blog.jinspired.com/?p=134 Adding AspectJ into our product was relatively easy as we had something underneath that actually worked, performed well, and allowed for a diverse set of instrumentation extensions to be built. Making AspectJ useful and usable in a distributed and dynamic operational context is what we are busy working on. regards, William Louth JXInsight Product Architect JINSPIRED "Performance monitoring and problem management for Java EE, SOA, and Grid Computing Platforms" http://www.jinspired.com
  10. You don't have to use AOP to instrument an application! It only makes it easier. There are other parts to AOP than AspectJ and co. A Servlet filter is AOP too (albeit quite crude). For reference, here are some interesting projects to look at: * ClickStream, although very old and unmaintained, the one that started it all, and superseded by: * MessAdmin, my very own monitoring solution :-) * Lambda Probe, my main competitor (Tomcat only) * Glassbox Inspector * InfraRED * JAMon and its associated Spring adaptor * JManage ________ MessAdmin, Java EE monitoring made easy
  11. Sorry but all of these so called benefits have been available in other mature and production ready performance management solutions that have instrumentation kits which can be used to perform integration with customer environments without the necessity of an AOP runtime. There is nothing new apart from ease of extension assuming you have tools that can help customers understand what exactly they are instrumenting at runtime. - Performance management tools already ship with a large number of extensions for popular technologies and frameworks. These tools also contain Open APIs (much better than a large code source with no clear public interface) that allow selected blocks of code within methods to be profiled which cannot be done by AOP. These tools are much more contextual and powerful than cflow constructs (i.e. filter profiling of objects based on JNDI name). Have you heard of ARM? Open APIs: http://www.jinspired.com/products/jxinsight/api/ - These tools (including JXInsight) also support on-line and off-line classification of call stacks (code) and trace (contextual) stacks. These tools can also gather statistics at practically all layers and can correlate and trace calls across clusters of Java servers. This cannot be achieved easily by AOP today. - In the real-world and not in someone's distorted, narrow, and wishful vision vendors provide open API's that allow customers to instrument code, and add context to monitored work flows in the most efficient and safe manner. Customers are free to use AOP, CORBA Interceptors, Events listeners, Filters,... to push and notify the performance management runtime of execution flows. - Sorry but linking has nothing to do with cflow. A call stack and trace stack can link (associate, correlate) across requests, components, layers, threads, processes and machines. Finally instrumentation is only one activity in software performance engineering process. Software Performance Engineering Activities http://blog.jinspired.com/?p=38 regards, William
  12. AOP DOES make it easy to achieve the benefits mentioned by providing a nice declarative language features to do the mentioned in the article. I've personally done and seen other people do these things by hand (for various reasons, including that many of them cannot afford commercial tools), or even use several different highly priced commercial tools many of which were limited in their extensions mechanisms. The article is meant to illustrate an alternative technique for achieving the same goals in a simpler way for many developers and AOP DOES help in that regard. I have not worked with JXInsight and if that product does several of these things, thats great. Choice is good for the community :-) Open sourcing will help too :-)
  13. I've personally done and seen other people do these things by hand (for various reasons, including that many of them cannot afford commercial tools), or even use several different highly priced commercial tools many of which were limited in their extensions mechanisms.
    How much time and expense is required to achieve production level quality of a performance management solution? It is certainly greater than 5,000 USD no matter which country or continent the development takes place in (assuming you are fortunate enough to have an experienced software product development team that understands performance management, information visualization, statistical analysis,.... and not just instrumentation). AOP should be used as a glue between architectural layers, components, and services and not a license to re-invent a service (transaction, trace, problem diagnostics, resource management,...) over and over again with little added value and/or poorly implemented. If you disagree with my point of view then take a look at the UI capabilities of all the products mentioned above. The difference is evident. regards, William
  14. How much time and expense is required to achieve production level quality of a performance management solution? It is certainly greater than 5,000 USD no matter which country or continent the development takes place in (assuming you are fortunate enough to have an experienced software product development team that understands performance management, information visualization, statistical analysis,.... and not just instrumentation)
    Well, what pevents tools that use AOP from creating a production quality solution?
    AOP should be used as a glue between architectural layers, components, and services and not a license to re-invent a service (transaction, trace, problem diagnostics, resource management,...) over and over again with little added value and/or poorly implemented.
    The value add of AOP is clear as articulated in our article and in several other articles/tools by other people as well. It seems like you have a fundamental disagreement to the use of AOP for performance management, and the technical reasons for that are not coming out clearly in your posts - for e.g., a phrase like "...license to re-invent a service ... over and over again..." sounds philosophical and hardly qualifies as a technical argument. I encourage you to write a full length technical article that clearly illustrates the disadvantages of AOP. If there are enough technical arguments, the issue is worth a larger debate rather instead of having it get lost deep in the discussion thread of this article.
  15. Well, what pevents tools that use AOP from creating a production quality solution?
    Nothing. But having knowledge and experience with AOP does not imply you know what performance management is all about. This is what is clearly evident in each of the tools above which by the way I have looked at previously. By the I find it amazing you do not know the competitor landscape yourself but yet you are able to make such board and sweeping statements in your article. (Again) AOP is great for instrumentation but it is only one of the aspects of a performance management solution. Unfortunately this is not understood by most developers who have no experience working in a production environment. What we see at various sites is home grown custom tracing solutions that actually create scalability and reliability problems for applications on their own with little added value other than filling up the hard disk and learning the developer AOP. Instrumentation for the purpose of tracing, diagnostics, testing,... is an extremely important activity but one must be sure the underlying engine is reliable. The best way to solve this is to introduce open standard API's that can be called by used by the instrumentation but support a service provider interface. I tried pushing this with some of the open source (AOP based) performance developers but the basic response was "Huh,....this would allow users to switch from our solution to yours with very little work...huh.....that will not serve our purpose of growing a community (so that I can one day sell out to a bigger fish)." I do not need to write a full length article. I am already showing the difference between playing with AOP and actually using in a production environment. AspectJ LTW: Which Methods Changed? http://blog.jinspired.com/?p=135 You have talked about how AOP makes everything so easy and yet none of the tools list above have actually provided any assistance to users in identifying and managing the resulting instrumentation. In a few weeks of development I have done more with AOP in the context of performance monitoring and problem diagnostics than others listed above have done over the course of two years which proves my point AOP != APM. regards, William
  16. So, you agree that AOP makes the instrumentation process of performance management easy. Thats a big part of performance management. Of course, there are more things to APM. Like managing the performance data for example. One can build the rest of the tools that manage the data coming out of the instrumentation in a variety of ways - some use JMX as part of that toolkit, and there are plenty other ways as well. While these are non-trivial parts of a performance management system, the point of the article was not to delve into this. We did not make the claim that AOP is all there is to APM. We highlighted important and difficult areas of APM that are made easier by AOP. Like I said in my earlier post, nothing prevents production quality APM systems from being built using AOP. There are already examples. You yourself have changed your words in the course of this discussion from "Benefits have nothing to do with AOP" to "AOP is great for instrumentation...". That was the main point of the article. On my knowledge of the competitive landscape, I know enough about the overall landscape to still stand by the statements I made in the paper. You have said nothing deeply technical in the course of this discussion that changes my position - a more useful debate would have been to list top 10 technical reasons why other approaches (such as ARM) is better than AOP for instrumentation. I saw none of that. All I saw is AOP != APM. We didnt claim that in the first place and as I said, there are tools that build a full fledged APM system using AOP.