Testing Extreme Programming

by Lisa Crispin and Tip House, Addison-Wesley, Boston, MA, 2003. 306 pp., $34.99(paper). ISBN 0-321-11355-1.

Reviewed by Charles Ashbacher

BOOK
REVIEW

While I yield to no one in recognizing the value of testing software, my first reaction to reading the title of this book was one of skepticism. One of the fundamental principles of extreme programming (XP), is that the software is developed in small increments, each of which must pass a unit test before the next change is made. In fact, in many cases the unit test is written by the developers before they write the code. These incremental tests are also carried out by the two-person coding team, so it seemed strange to be reading about testing XP. From the title and blurbs on the covers, it was a natural assumption that the focus would be on testing other than that done by the development teams.

After reading the book, that skepticism has largely gone, although I do possess some residual doubts about XP and how it scales. The basic point is that programmers are very good at testing their code at the unit level, but weak when asked to verify it at the system level. I agree with the authors that there should be a dedicated tester who examines the code at a level higher than the unit. However, I am also of the opinion that this is a confirmation of the doubt about XP expressed by so many observers, namely that it does not scale up to large projects well. The testers that they are proposing are more in the realm of a manager responsible for testing rather than a tester.

This is of course very sensible. Once the programmers start producing code tested at the unit level and the integration process begins, someone must be responsible for the smooth flow and testing of the integration. This is also the level where the ever-present customer, another fundamental principle of XP, really sees the functionality of the code for the first time. While XP proponents speak a great deal about having the customer at the side of the coding team, realism dictates that they will generally be restricted from that level. Only the most technically sophisticated customer will be able to glean any useful information from most of the unit tests that will be performed. This is where the additional layer of the test manager is of use. By creating and demonstrating the higher level tests, the test manager can give the customer information that they will understand and can respond to.

The authors also put forward a very controversial statement, “No manual tests. All acceptance tests on an Extreme Programming project must be automated.” While I am in general agreement with the principle that tests should be automated for easy repetition at each level, the reality is that nearly every use of words such as { “no”, “never”, “all”} is too extreme. Especially when you are describing something as subjective as the behavior of computer programs and the human response to them. How one can automate the response of a customer to the appearance of a GUI interface is something I do not yet understand, and this is mentioned, but not examined in the book.

One very positive aspect of the book is the exercises at the end of the chapters, and the authors do the very commendable thing and provide solutions at the end. After years of frustration with math and computer books that list exercises but avoid solutions, any book where they are included must be given a higher rating.

After reading this book, my confidence in the value of XP has increased, ironically because one of the fundamental weaknesses is examined with an explanation of how to overcome it. The uber-tester is a concession to the problems of scaling, and the inclusion of such individuals will definitely make the development process run smoother. If you are going to use XP and your project is of any size, then you should read this book.

Cite this book review as follows: Charles Ashbacher: Review of “Testing Extreme Programming”, in Journal of Object Technology, vol. 2, no. 3, May-June 2003, pp. 153-154. http://www.jot.fm/books/review9