Far too often “the database” is blamed for impacting performance and scalability of applications. In many cases, however, it turns out that it’s not the database itself, but the way the database is accessed by the application. Common problem patterns are requesting too much data, inefficient queries, lack of data caching, and waste connection usage, for example.
But, there are cases where the database is to be blamed. This is the scenario of the following story one of our customers shared with me:
The company runs 4 JVMs in a WebLogic cluster with each of them having a connection pool to its Oracle database with 20 max connections. They have setup proper application monitoring which includes both system monitoring and application and real user monitoring. Here is what happened on December 16th at 2PM.
Step 1: Alerting on Connection Pool Exhaustion
As part of its application monitoring, the team members are watching connection pool stats from WebLogic via JMX. They look at measures such as ActiveConnectionsCurrentCount, ActiveConnectionsHighCount as well as ConnectionDelayTime. Besides having these metrics on a dynaTrace dashboard that shows them pool usage per JVM over time they also have an automatic alert configured that triggers when a pool is exhausted. This alert triggered at 2:06PM