QA Managers Caught in the Middle
by Pat Craig 

From the Winter 1995 issue of the Complexity Management Chronicles 

We see many QA managers in anguish about providing thorough testing with limited resources. These managers seek a way to justify hiring additional staff so they can test better. Their pain is very real. To begin addressing this issue, we have written this newsletter. 

We simplify their dilemma into the following analogy. Paying for staff to audit/test the quality of software is similar to buying health insurance. Just as people get sick over time (with the flu) or extended use (tennis elbow), so too, software will break down with time and use. 

As with a health insurance purchaser, development management must define the level of risk they are willing to assume. They can take a large risk by spending little on insurance (testing), take a moderate risk, or take a small risk by spending a lot on testing. 

You as a QA manager can only provide as much insurance as management will fund. If you struggle to release a quality product using minimal resources, we have these six suggestions to help you justify staff (and ease your pain in the meantime): 

Use past bug history as a staffing guideline. Bugs found in prior releases may be your surest indicator of the number of bugs in this new release. Compare your organization's cost for correcting bugs in the field versus the cost to identify bugs earlier. IBM/Raleigh reports that a bug found in the field can cost 25 times more to correct than a bug found in test. Contact us for more detail on this study and other cost data. Past bug history has prompted two of our clients, a bank and a software vendor, to spend more money on testing. 

Build credibility. As a rule, upper management will not spend money on managers or groups that have questionable worth. An important part of your job is selling the value of your department to them. 

Inform management in writing of the risks they incur by not spending more on testing. Report quality disasters such as the Intel's $475 million write off for Pentium chips. QA groups, such as ASQC, SPIN, and NESQAF, can help quantify risks. Call us for information on these groups. 

A SQA manager we worked with at a financial service institution has had success getting more resources. She explains the risks by saying "If you do not get me these labor and computer resources -- here are your risks: to your bottom line, to your service levels, and to your department's credibility." 

Hone in on the bugs. Your search for bugs is similar to a Coast Guard search for lost fishermen. You want to search every inch of the software for bugs, just as they want to search every inch of the ocean. But you need to start searching for bugs in the most likely places. Your search, like the Coast Guard's, is constrained by the staff at your disposal, the state of technology (automated test tool technology), and the amount of time you have. Providing good value for the testing dollars expended should help you justify additional staff in the future. 

Temper upper management's expectations of how much software your group can test in the allotted time. Write weekly reports informing them about the testing progress and the bugs discovered. Call their attention to the areas of software code that you cannot test due to resource constraints. 

Shift some testing to customers by starting a Beta Test Program. Nurturing a close relationship with customers willing to beta test your software is an extremely cost effective way of testing. 

In conclusion, agonizing because you cannot thoroughly test on a shoestring budget just wastes your energy. On the other hand, following our hints should bring you more success in the future. 

©Complexity Management 1995 
Somerville, Massachusetts
Located in Metropolitan Boston

Complexity Management Chronicles, a newsletter for software quality assurance professionals, is published in print form four times a year. Send your name and snail-mail address to the e-mail address below if you would like to be on the mailing list - at no cost to USA mailing addresses.

*********************************************************************** 

Return to Complexity Management Home
Contact Pat Craig at patcraig@alum.mit.edu