Don‘t Trust Your Log Files: How and Why to Monitor ALL Exceptions


News: Don‘t Trust Your Log Files: How and Why to Monitor ALL Exceptions

  1. I would say that only one out of a million exceptions thrown in an application actually makes it to a log file – unless you run your application in verbose logging mode  – Do you agree? No? Here is why I think that is: Because most exceptions are handled by your code or by the frameworks your app uses. Here is a chart from an enterprise application showing that there are about 4000x more custom application exception objects thrown than important log messages written.

    So why worry about these exceptions that nobody cares to write to a log file? Two reasons:

    1. They are typically thrown for a good reason and therefore indicate a problem, e.g: configuration issues in frameworks or runtime problems
    2. Every Exception object is a potential performance problem because it means the JVM needs to allocate memory, get the stack trace and dispose the object soon after

    Reason #1: Configuration Problems

    The following shows a transaction where the method getImagePath makes a web service call to a backend server using HttpClient. getImagePath uses a an HTTP Endpoint URL. The Web Service however only supports HTTPS (SSL). The web service call therefore fails with an SSLException. getImagePath retries 3 times until it gives up and just returns a default value to the caller. No log entry written, no exception thrown to the caller, everything seems OK to the outside world even though we have a severe impact on an end user who is waiting longer than necessary for an image that he doesn’t get to see:

    Continue reading the rest of the blog ...

  2. Can you please help me in finding customers as hopeless as the ones you have written about at length? I long for the day that performance problems could be so obvious that I could use any performance monitoring, no matter how bad, and still be a winner. I want to be a winner like you and if that means losing first to become a real winner I'm there :-).

    Interestly enough we have to aposing posters posting on TSS lately. Those saying don't log it hurts performance and others say log to solve performance problems. I should be confused, right? But I'm not. Does that mean I've a more serious problem?

  3. Its always "refreshing" when I see a comment from you Will. How are you? :-)Well - if you follow our blog posts where we tend to write about real customers that have real issues you will see that these problems really exist and that people struggle with them. They may seem basic and simple for you - but - fact is that a lot of software out there is facing performance problems that boil down to a handful of problem patterns. Looking forward seeing you again soon and will be happy to discuss this in more detail with you - just as we did in the past when we met :-)