Exhaustive testing of software is generally impossible; in all but the most simple of cases the number of possible paths is effectively infinite. By that I mean that while the number is finite, it is so large that only a tiny percentage could ever be tested. So in practice the number is infinite. Since humans are prone to error and the number of possible paths where a bug can be introduced into a program is also very large, the bug-free (perfect) program is an unattainable goal. It is no fun, but that is just the way things are. Being driven by optimism and necessity, humans still create millions of lines of code every year and many of these programs are released into the untamed wilderness known as the customer base. Therefore, conscientious developers have no choice but to develop a testing program that will have a reasonable chance of identifying the most significant and likely bugs in the software. As Weinberg points out, some people do make the choice to do very little testing, sometimes relying on an approach like "If we get lucky, no one will experience that bug." Others simply redefine the bug as a feature and move on. Neither of these approaches is ethical nor will it provide you with legal cover. In a very witty yet understated style, Weinberg describes some of the circumstances he has encountered where people simply denied there was a problem and if that failed tried to hoist the problem onto someone else. As is the case in most areas of software development, the solution lies more in the realm of group psychology rather than technical expertise. Having authored the classis "The Psychology of Computer Programming" Weinberg is an expert at finding what solutions are available to this seemingly impossible problem. It starts with eliminating all illusions and delusions and accepting the inherent limitations of what software testing can do. After that, one develops a testing process that is executed concurrently with the development of the software. The first step is to test the modules by creating a set of reasonable coverage test values. Varying from module to module, this set covers some of the most likely entries as well as the bizarre ones and the ones on the limits. Once the tests on the modules are complete, the next step is to integrate the modules into a functional unit. Once the unit is stable and can be run, an additional set of reasonable coverage test operations is created and executed. In each case, detailed knowledge of what the software is supposed to is used to develop the sets of test values. Weinberg is also forceful in debunking the current state of the automated testing of software, a tactic that he dismisses. The strongest argument against using software to test software is grounded in human psychology, namely the easily adopted assumption that if it came from a computer then it must be right. This belief in the infallibility of technology has a strong tendency to dumb-down humans and that is to be avoided if at all possible. As stated before, the quality software developer has no choice but to test their product. If you want to maximize your likelihood of releasing a quality product, then your first step is for everyone in your team to read and discuss this book. Doing so will clear a lot of the yucky stuff out of your path. So while you may still step in it, the pile you squish is either of your own making or one of the few that Weinberg does not cover. Bookreview by Charles Ashbacher: "IBM Rational Unified Process Reference and Certification Guide Solution Designer", in Journal of Object Technology, vol. 8, no. 1, January-February, pp 133-134 http://www.jot.fm/books/review24/ |
