Software Engineering
Home Planning Requirements Writing Hazard Analysis Requirement Analysis Config Control Software Design Software Testing Software Standards Basic Logic

Software Testing

No amount of testing can demonstrate that software contains no defects; it can only demonstrate that a particular defect exists.

It’s like flipping a coin. Once flipped, the outcome is a past event and cannot be changed. The only thing I don’t know is how the event came out. To know that, I must look at the coin. Once the software is compiled, its performance is a past event and cannot be changed. Testing only reveals its performance.

Value Proposition:

When one tests a program, it should add value to the product, otherwise testing is a waste of resources.

Improving a program’s quality and reliability increases it’s value.

If the testing process found no errors, then the program's quality and reliability remain unchanged - no value was added.

Therefore: Testing is the process of executing a program with the intent of finding errors, i.e. to show that the program does NOT work!
 

Testing does not improve quality.
Correcting defects improves quality.
Testing reveals defects, so they can be corrected.

Prerequisite

Basic Logic Fundamental to software test design and requirements analysis.

Pre-Course Test

Write the tests for the following program:

  • Program reads three integers from the keyboard.

  • The three integers are interpreted as being the lengths of the sides of a triangle.

  • Program tells the user whether the triangle is scalene (no sides equal), isosceles (2 sides equal) or equilateral (all sides equal).

In the Summary section there is a defect seeded program to try out your test cases.

Common Problems in Software Testing

  • Test data does not actually support the conclusion.

  • Descriptive, not Prescriptive testing.

  • Documentation based testing philosophy. This commonly occurs in conjunction with descriptive testing.

  • Requirements analysis not performed. This causes many test conditions to be missed.

True Horror Story

While managing a medical device software team, a key Quality Assurance person insisted that we perform more exhaustive testing of an operating system security patch before allowing our customers to install the security patch onto our systems. Since the patch was written in response to an Internet worm, the customers were frantic to install the patch. However, since our system was a medical device, customers could not install anything without our written permission.

I asked the QA person point blank: "Are you willing to increase the risk to the customers so you can reduce the risk to the test documentation?"

"Yes" was the QA answer. QA's test philosophy was clearly documentation-based, not product-based.

Thankfully, they were overridden by everyone else (including the FDA, when we called them!) and the customers began installing the security patch the next day.