Profiling EJBs in the real world

Discussions

News: Profiling EJBs in the real world

  1. Profiling EJBs in the real world (13 messages)

    Fred Grott, in "A Question About profiling EJB applications," says that it "seems a better proposition to start at the coding steps to use junit frameworks to assist in profiling EJB apps than waiting until the application is completely developed to profile." It raises a question: how do people profile EJBs in the real world? It's very easy to pretend one is in an Ivory Tower, where there are two monitors for every programmer (or two programmers for every task), exhaustive unit tests are written before any actual implementation code is written, coders know Spring inside and out, and documentation is written (and referred to) constantly, but few people actually have that kind of situation. Instead, computing resources are scarce, programmers may be paired - but that's in name only, tests tend to be written only after a failure has been found, documentation is used to belittle coders' failure to read or write it, and Spring may be deployed as part of the project but most coders just write code around it anyway. If you're using EJBs or other resources, how do people in the "real world" use them? How do you determine if they're running slowly or acceptably? How do you determine sizings? (Are most people even aware of how to configure their containers?) Is a unit testing framework the right way to profile something like this?

    Threaded Messages (13)

  2. Re: Profiling EJBs in the real world[ Go to top ]

    If you're using EJBs or other resources, how do people in the "real world" use them? How do you determine if they're running slowly or acceptably? How do you determine sizings? (Are most people even aware of how to configure their containers?) Is a unit testing framework the right way to profile something like this?
    I've been lucky enough on my last few projects to have enough sway as an architect to decide how EJB's are used. So in short: they're not. However, to elaborate, there are actually two instances where I've actually used EJB's lately: - Stateless Session Beans: basically as a tightly coupled java based transport/remote facade for remote applications/components that are still in all but deployment part of the same logical application unit. - Message Driven Beans: Again, as a facade to do switching from JMS transport, into Java-objects. So, when and if EJB's have been used, it has mostly been as a facade (or switch from messaging to Java), and all the actual business logic has been implemented in POJO's. That way the actual performance can be tested outside of a container, as it is all implemented in POJO's. The only real performance issue as far as EJB's are concerned has been with sizing of EJB pools (and other similar configuration tweaks), and the potential performance of the underlying messaging system in the case of MDB use. / Wille Blog: Buzzword Bingo
  3. Re: Profiling EJBs in the real world[ Go to top ]

    We use different tools/approaches for different aspects of this problem (EJB 2.0): - method code-level issues are best done by developers with a traditional Java profiler; but these normally cause the least performance issues - in-container tuning of EJB pools, worker threads, JMS queues etc is done in performance labs that are modeled after real production environments (deployment topology-wise if not the actual processing capacity), and under numerous load scenarios; mostly using built-in container and JVM profiling tools - but for really important areas, where we need to understand exactly what's going on, execution paths, transaction payloads/timings, EJB component interation, resources involved, invocation counts, SQL issued by CMPs etc etc, we use a specialized tool (JXInsight). Organizationally, we have a group dedicated to nothing else but product performance and stability improvement. Ivan
  4. That way the actual performance can be tested outside of a container, as it is all implemented in POJO's.
    Personally, I've never really had the need to test/profile/debug/whatever outside the container. Maybe its because I've only worked with JBoss the past 5.5 years, I don't know... JBoss hotdeploy works great for unit testing (and development for that matter). A few seconds or less for most tests. If you look at the JBoss build scripts you'll see that JBoss is started once and a multitude of tests are deployed/undeployed through the automated testsuite/build. We have also some nice extensions to JUnit that make it very easy to deploy/undeploy components for the duration of your unit test. I always thought debuggers were for pussies, but on those rare occasions that I use a debugger it is ridiculously easy to boot up JBoss and attach Intellij to the JVM process. Its also been extremely easy to use JProfiler or Optimizeit to profile any app or even app server code with a running JBoss instance. Clebert Suconic's JBoss Profiler project is also great for automation of profiling and memory leak testing. I never really assume something is "tested" until it runs in the environment it is supposed to run in... I dont know, maybe (probably), this whole testing outside the container movement got so popular because other app servers are so crappy and slow at booting up or incapable of performing a simple hot deploy. Or maybe people are drinking too much koolaide and not trying things out for themselves... Bill
  5. When performing Java EE component performance unit testing whether it be automated (JUnit/TestNG) or manual (clicking within a web page or Java UI form) the focus should not necessarily but on how slow the execution is but on the count of various important resource interactions. One should look for metrics that are NOT dependent on the underlying hardware, deployment topology, or concurrent workload on a developer workstation system. I initially advocate that developers collect the following measurements for all major business/user operations using typical data patterns and distributions. 1. The number of client-to-server round trips 2. The number of global and local resource transactions (JCA, JDBC, JMS) per trip and per business operation. 3. The number of component-to-resource interactions (EJB->JDBC, Servlet->JDBC, EJB->JMS, EJB->JCA, EJB->EnterpriseCache, EJB->CORBA, EJB->WS,..) per trip and per business operation 4. The number of component-to-component (ejb-to-ejb, servlet-to-ejb...) interactions per trip and per business operation 5. The (average) number of object allocations during the execution of each trip and business operation 6. The number of explicit/direct I/O calls made per trip and business operation The above list is not a complete list but I hope you understand the type of application execution behavior that I try to measure (COUNT) and the application execution behavior that is being collected and modeled. Wall clock and CPU times are also looked at but these are subject to changes in the underlying runtime, os and hardware platform. Of course poor response times at this stage of testing should be cause for alarm but good response times might be misleading especially to project management when you have a large number of JUnit tests that are performance specific and all have this feel good factor - PASSED. The following articles touches on this approach. http://www.jinspired.com/products/jxinsight/usertesting.html Kind regards, William Louth JXInsight Product Architect JInspired "Java EE tuning, testing, tracing, and monitoring with JXInsight" http://www.jinspired.com
  6. Re: Profiling EJBs in the real world[ Go to top ]

    +1 Hire that man!
  7. Re: Profiling EJBs in the real world[ Go to top ]

    I'm aware of the fact that most people on TSS don't like IBM WebSphere (for its complexity etc.) but this is one of the areas where WebSphere (with or without additional Tivoli tools) is really good. Oracle OC4J also allows collecting performance statistics. http://www.enterpriseware.eu
  8. Re: Profiling EJBs in the real world[ Go to top ]

    Hi Adam, I am not sure that is the view from outside of IBM. One must question this statement when one looks at the growing number of APM vendors providing solutions for IBM WebSphere. One leading vendor was recently acquired by CA for more than 350 million. IBM does offer much more compared to other vendors such as JBoss and BEA but that is not saying much when you consider what is on offer at present. "Fred Grott, in "A Question About profiling EJB applications," says that it "seems a better proposition to start at the coding steps to use junit frameworks to assist in profiling EJB apps than waiting until the application is completely developed to profile." I believe it is important to start early as possible but not for the sake of publishing performance benchmarks but to gain an insight into the underlying component execution patterns during the software construction for behavioral verification and application knowledge acquisition across team members and teams that will be of great value when problems do arise in production. Regards, William
  9. JBoss does collect some helpful call statistics that can be viewed with the webconsole. But the way this data is presented does not answer questions like 'which service has been called most often?' or 'What is my top ten ms eater list, by average and total'? This is where the llameSituando.jsp (to be copied into the folder of the deploy/jmx-console.war). Caution: It has only been testet with jboss 3.2.6 <% Iterator mbeans = Server.getDomainData("jboss.j2ee*:service=EJB,*"); while (mbeans.hasNext()) { MBeanData lMBeanData[] = (MBeanData[]) ((DomainData) mbeans.next()).getData(); for (int d = 0; d < lMBeanData.length; d ++) { ObjectName name = lMBeanData[d].getObjectName(); if (name.getKeyProperty("plugin") == null) { // hack? another way of filtering out the pools? Map lStatMap = ((InvocationStatistics) Server.getMBeanAttributeObject(name.toString(), "InvokeStats")).getStats(); Iterator lIter = lStatMap.keySet().iterator(); while (lIter.hasNext()) { Object lKey = lIter.next(); InvocationStatistics.TimeStatistic lTimeStat = (InvocationStatistics.TimeStatistic) lStatMap.get(lKey); %> <%=lMBeanData[d].getMetaData().getClassName() + '\t' + name.getKeyProperty("jndiName") + "\t\"" + lKey + "\"\t" + lTimeStat.count + '\t' + lTimeStat.minTime + '\t' + lTimeStat.maxTime+ '\t' + lTimeStat.totalTime %> Got a second jsp that makes excel open directly where one can filter, group, sort and format the collected data. type jndiName method count min max total I'll spare out the Dojo based AJAX-Client here. JBoss users can use this without any additional runtime overhead. Together with databases statement statistics this helps finding performance problems a lot. Sometimes this hight-level view is more helpfull that the overwelming amount of data a real profile delivers.
  10. Instrumentation != APM Solution[ Go to top ]

    Hi Marc, Such information (ignoring the obvious lack of tooling) is useful for high level monitoring of applications during pre-production testing and production. It is in general not useful in assisting engineers, architects, and testers gain a better understanding of the underlying execution workflow. This understanding is of paramount importance when a problem does arise in production and every person involved has his own theory of the underlying performance problem. By the way in JXInsight we control the amount of information collected and provided via 3 analysis modes (Timeline, Profile and Metrics) along with 500 system properties that allow fine grain control over our agent collection and instrumentation. Our metrics analysis mode already includes JMX integration for JBoss, BEA, Apache Tomcat, and Tangosol Coherence. http://www.jinspired.com/products/jxinsight/new-in-4.2.html I think it best we stick to defining the software performance engineering process and techniques independent of what a particular application server provides or does not provide which is largely the case. Regards, William Louth CTO, JInspired "Java EE tuning, testing, tracing, and monitoring with JXInsight" http://www.jinspired.com
  11. would have been surprising if they could, would'nt it? Looking really jealous on the stack trace down to the JDBC statement your tool provides I kind of think that the shop I am currently working for spend the APM bucks for the wrong tool (not the one one above). @Bill: No, jboss does not deploy everything in an instant.
  12. Profiling Ejbs in the real world is quite a dispensible job.The readily available profiling tools in the market like OptimizeIt perform its job very well but it is primarily the application server which should doll out all the statistics about ejb application which resides in there.We need a full set of profiling features to cofigure the container for best performance.We need stats about pool sizes, thier usage over the run, the major ms eating sessions, how container is using database connections?what are the queries fired, are they optimized?How heap is increasing?And above all how the server responding to requests?After all its the response time that matters, you don't want users to wonder after a click. Well! Users of Pramati Application Server will be very happy in getting all they want about profiling.The server has a rich set of diagnostic and monitoring tools which will give runtime statistics about everything we need to start tuning the container.If we want all the queries that has been fired, the current database connection usage, the pool sizes of all the ejbs, the execution time of service methods,you can gather all the information.The diagnostic tool gives very fine grained statistical details about each and every processing that is being running in the server.Now you how much time your business method took to execute, what are the queries fired inside the method and practically everything related.The running of diagnostics may add little overhead but the features are worth of it. JBoss too does colect some useful information and viewing in the webconsole adds an ease of task. cheers, http://javaicillusion.blogspot.com/
  13. How does this help the many other developers using leading application server platforms? What is the performance profiling technique you are advocating? Note it is becoming much more common for developers to deploy and test against multiple application servers during development, testing, and pre-production. This is especially true for companies delivering products built on Java EE where the customer has a choice of support application server platforms. Having customers report back to the product support team with output from different application server consoles is a nightmare for training and knowledge acquisition (root-cause analysis database). Such companies should instead look at embedding a mature and vendor independent application performance management solution within their own products to reduce support costs and problem resolution turnaround times thereby improving overall customer satisfaction. Having the same "extensible" APM solution used by developers, architects, testers, and operations (customers staff) has many benefits and the costs are not that high once you select based on quality and stay away from the big shops with the big windows. Regards, William Louth JXInsight Product Architect JInspired "Java EE tuning, testing, tracing, and monitoring with JXInsight" http://www.jinspired.com
  14. It is not about advocating any particular profiling technique or backing some application server.The whole point is that what are the options that we are left with when it comes to profiling ejbs and other server side components. Commercially available profiling tools are always vendor independent, you just need to integrate them with the applicatio server you are using during "code and port" phase or during production phase.But the real benefit will be when your application server gives you a hook to get the most of profiling stats from the container itself. And we need to wait offcourse to get some profiling solution which comes shipped with some vendor and yet vendor indepenedent.