In an article published on SearchSoftwareQuality.com
, Joe Ponczak and John Miller question common beliefs on code coverage. In the article they assert that even with 100% code coverage, one cannot be assured that their application is devoid of critical bugs.
The meat of the article starts with a description of code coverage and then expands on this with a definition of statement and branch coverage. With this foundation, Joe and John explain how these two different types of coverage are inadequate.
A branch is the outcome of a decision, so branch coverage simply measures which decision outcomes have been tested. This sounds great because it takes a more in-depth view of the source code than simple statement coverage, but branch coverage can also leave you wanting more.
In the accompanying example, the authors demonstrate how to use branch testing to achieve 100% coverage. However, as complete as the tests are, they still miss a bug that is lurking in the code. To top it all off, this is just a small example and things start getting much more difficult very quickly as this size of the problem increases.
So, achieving 100% statement and 100% branch coverage may not be adequate, and testing every possible path exhaustively is probably not feasible for a complex method either. What's the alternative? Enter basis path coverage.
What is path coverage? To answer that question, the authors start by offering a definition of an execution path. The claim made is that path testing allows one to find the smallest subset of paths through the code that tests every decision outcome independently of each other. Once again the authors turn to their example to demonstrate the steps one needs to take to define test cases. The result is that testing now finds the bug that had been previously missed.
The authors conclude that although any type of coverage testing is a step in the right direction, there are advantages to moving beyond statement and branch coverage. Is coverage an important part of your QA process?