Finding Bugs: No Pain, No Gain
by Pat Craig 
 

From the Fall 1996 issue of the Complexity Management Chronicles 

Our last newsletter detailed three reasons that cause staff turnover. Staff losses can result in software delays and reduced creative capacity. The reasons for staff losses, cited previously, were:

- Snowballing employee resignations caused by managers who push too hard,

- Shoddy work being shoved from stage to stage in the system development life cycle, and

- Balance of power wars between Development and Test.

Wars: In this newsletter we would like to touch upon the balance of power wars and offer some client tested solutions. We have witnessed some particularly ugly conflicts revolving around QA¹s role as the institutionalized defender of quality. In one case, a key tester quit the firm rather than succumb to pressure and lie about the status of an important bug. In another case, two developers regarded a tester's bug reports as a personal affront and they retaliated by attacking her professionalism. She was unjustly put on probation and subsequently resigned. This newsletter also details how a client solved their Development/QA wars after an important developer and an outstanding tester, embroiled in a bitter dispute about recording bugs, both quit their jobs within days of each other.

Minimizing Rework Lowers Costs (and Conflict): Coincident with these staff losses, the President hired a new Development VP. The VP knew about QAI published data demonstrating that the cost to remove defects increases exponentially throughout the life cycle. (In February 1994 QAI quoted defect correction costs ranging from less than $10 per bug to more than $1,000. Costs were strongly correlated with when bugs are found.) Rework is expensive. The further downstream in the development cycle bugs are found, the more they cost to correct. To reduce development costs the VP and his staff implemented many techniques to discover the bugs earlier "upstream" while still in the requirements, detailed design or coding phases. The added benefit of discovering bugs earlier was less conflict between Development and Test.

Group Walkthroughs Ensure Deadlines Met: In the 1987 IEEE Transactions on Software Engineering, the computer scientists V. Basili, R. Selby, and F. Baker published bug removal research results. Their research assessed the merit of "testless" testing techniques: code reading, inspection, and group walkthroughs. They had experimental groups of developers using "testless" techniques only. To ensure scientific validity, they also had control groups using traditional methods. The study affirmed "testless" techniques. A full 100% of the experimental teams met their deliverable dates, while only 40% of the control groups did as well. (The IBM Systems Journal, Vol. 29, No 1, 1990, authors Mays et al. also discusses the value of these techniques.)

The VP used the Basili study to drive home his statement, "Software development is tough. We are going to make it a part of our corporate culture that developers enlist their friends on bug finding missions." Group walkthroughs became the norm.

"Zero Defects" to Test Mean Less Bugs Installed: Dave Moore, Development Director for Microsoft writes in S. Maguire's book "Writing Solid Code" that test groups can only find about 60% of all bugs. To ship cleaner code Microsoft has made Development responsible for turning over zero defect code to Test. Microsoft still has a test group and beta tests, to catch what Development misses.

We hope this brief review of bug discovery ideas will help minimize conflict between your Development and Test groups. 

©Complexity Management 1996 
Somerville, Massachusetts
Located in Metropolitan Boston
 


Complexity Management Chronicles, a newsletter for software quality assurance professionals, is published in print form four times a year. Send your name and snail-mail address to the e-mail address below if you would like to be on the mailing list - at no cost to USA mailing addresses. 

*********************************************************************** 

Return to Complexity Management Home
Contact Pat Craig at patcraig@alum.mit.edu