Finding Bugs: No Pain, No
Gain
by Pat Craig
From the Fall 1996 issue of the Complexity Management Chronicles
Our last newsletter detailed three reasons that cause staff turnover. Staff losses can
result in software delays and reduced creative capacity. The reasons for staff losses,
cited previously, were:
- Snowballing employee resignations caused by managers who push too hard,
- Shoddy work being shoved from stage to stage in the system development life cycle,
and
- Balance of power wars between Development and Test.
Wars: In this newsletter we would like to touch upon the balance of power wars
and offer some client tested solutions. We have witnessed some particularly ugly conflicts
revolving around QA¹s role as the institutionalized defender of quality. In one case, a
key tester quit the firm rather than succumb to pressure and lie about the status of an
important bug. In another case, two developers regarded a tester's bug reports as a
personal affront and they retaliated by attacking her professionalism. She was unjustly
put on probation and subsequently resigned. This newsletter also details how a client
solved their Development/QA wars after an important developer and an outstanding tester,
embroiled in a bitter dispute about recording bugs, both quit their jobs within days of
each other.
Minimizing Rework Lowers Costs (and Conflict): Coincident with these staff
losses, the President hired a new Development VP. The VP knew about QAI published data
demonstrating that the cost to remove defects increases exponentially throughout the life
cycle. (In February 1994 QAI quoted defect correction costs ranging from less than $10 per
bug to more than $1,000. Costs were strongly correlated with when bugs are found.) Rework
is expensive. The further downstream in the development cycle bugs are found, the more
they cost to correct. To reduce development costs the VP and his staff implemented many
techniques to discover the bugs earlier "upstream" while still in the
requirements, detailed design or coding phases. The added benefit of discovering bugs
earlier was less conflict between Development and Test.
Group Walkthroughs Ensure Deadlines Met: In the 1987 IEEE Transactions on
Software Engineering, the computer scientists V. Basili, R. Selby, and F. Baker published
bug removal research results. Their research assessed the merit of "testless"
testing techniques: code reading, inspection, and group walkthroughs. They had
experimental groups of developers using "testless" techniques only. To ensure
scientific validity, they also had control groups using traditional methods. The study
affirmed "testless" techniques. A full 100% of the experimental teams met their
deliverable dates, while only 40% of the control groups did as well. (The IBM Systems
Journal, Vol. 29, No 1, 1990, authors Mays et al. also discusses the value of these
techniques.)
The VP used the Basili study to drive home his statement, "Software development is
tough. We are going to make it a part of our corporate culture that developers enlist
their friends on bug finding missions." Group walkthroughs became the norm.
"Zero Defects" to Test Mean Less Bugs Installed: Dave Moore, Development
Director for Microsoft writes in S. Maguire's book "Writing Solid Code" that
test groups can only find about 60% of all bugs. To ship cleaner code Microsoft has made
Development responsible for turning over zero defect code to Test. Microsoft still has a
test group and beta tests, to catch what Development misses.
We hope this brief review of bug discovery ideas will help minimize conflict between
your Development and Test groups.
©Complexity Management 1996
Somerville, Massachusetts
Located in Metropolitan Boston
Complexity Management Chronicles, a newsletter for software quality assurance
professionals, is published in print form four times a year. Send your name and snail-mail
address to the e-mail address below if you would like to be on the mailing list - at no
cost to USA mailing addresses.
***********************************************************************
Return to Complexity Management Home
Contact Pat Craig at patcraig@alum.mit.edu .
|