If transaction times jump from less than one second to more than 60 seconds after a software upgrade, it is evident that there is a problem that needs investigation. This scenario was experienced by a customer with its time-tracking application. Our customer identified this performance regression while executing extensive performance tests prior to its production upgrade. The company’s “traditional” approach of analyzing these types of problems was analyzing application and database log files as well as infrastructure metrics such as CPU, Memory and Network Throughput. Our customer couldn’t find a single slow database query, nor were there any indications of an error in the log files. The infrastructure also seemed to be fine because the Application, the Database Server and the Network didn’t show any bottlenecks. Analyzing the End-to-End Transaction, such as the click on search, however revealed a problem which turned out to be a “data-driven” performance issue.
Problem: Loading and Processing Large Result Sets
The following screenshot shows parts of the transaction execution highlighting where most of the time is spent within the transaction: