Mahmoud Abas, Full-Stack Software Engineer & AI

Software Construction - Software Construction - Lab 3: Software Testing Techniques

In the print dialog, choose "Save as PDF" as the destination.

Software Construction - Lab 3

Software Testing Techniques

Validation, test-first programming, black-box and white-box techniques, and coverage.

Agenda

Validation and validation vs verification
Why software testing is hard
Test-first programming
Software testing technique types
Black-box and white-box testing
Choosing test cases by partitioning
Coverage
Unit testing vs integration testing

Validation

Testing is one example of the broader process of validation: uncovering problems in a program to increase confidence in its correctness

Validation

Verification

A formal proof of correctness; tedious by hand, tooling still a research area

Code Review

Someone else reads your code and reasons about it informally

Testing

Run the program on selected inputs and check the results

Validation vs Verification

Verification

Asks: are we building the product RIGHT?
Checks documents, design, code and program against the spec
Does not involve executing the code
Finds bugs early in the development cycle
Comes before validation

Validation

Asks: are we building the RIGHT product?
Dynamic: always involves executing the code
Checks the software meets the customer's needs
Catches bugs that verification cannot
Comes after verification

Perfect Quality in Software

Even with the best validation, perfect quality is very hard to achieve
Typical residual defect rates, measured per kloc (1,000 lines of source)

Defects per kloc	Quality level	Examples
`1` - `10`	Typical industry software	Most commercial software
`0.1` - `1`	High-quality validation	Java libraries
`0.01` - `0.1`	Safety-critical validation	NASA, Praxis

Why Software Testing Is Hard

Exhaustive Testing: Infeasible: the space of possible test cases is generally too big to cover. A 32-bit floating-point multiply a*b alone has 2^64 possible test cases.

Haphazard Testing: Just trying it and seeing if it works is less likely to find bugs, and it does not increase confidence in the program's correctness.

Random or Statistical Testing: Other engineering disciplines can sample a small percentage of a physical product and infer a defect rate, because physical defects are continuous and uniform across the space. Software bugs are discrete, not continuous, so that assumption fails.

Test-First Programming

When you're coding, the goal is to make the program work. When you're testing, the goal is to make it fail.

Test early and often: it is far more pleasant to test code as you develop it than to leave testing until the end

Write a specification for the function
Write tests that exercise the specification
Write the code: once it passes the tests you wrote, you're done

Writing tests first is a good way to understand the spec: a spec can be buggy too, incorrect, incomplete, ambiguous, or missing corner cases
Trying to write tests surfaces those problems early, before you have wasted time implementing a buggy spec

Software Testing Technique Types

Techniques help design better test cases: fewer of them, with more coverage, and they surface conditions that are otherwise hard to notice

Testing Techniques

Specification-Based

Black-box: test cases chosen only from the spec

Structure-Based

White-box: test cases chosen with knowledge of the implementation

Experience-Based

Driven by tester skill, intuition and past failures

Specification-Based Techniques

Boundary Value Analysis (BVA): Explore errors at the edges of the input domain: the minimum, just above the minimum, the maximum, just below the maximum, and invalid values.

Equivalence Partitioning (EP): Split inputs into classes that should behave the same way, then test one representative value from each class.

Decision Table Testing: Systematically test every combination of conditions against its expected outcome.

State Transition Diagrams: Model the system as states with rule-guided transitions; test that valid transitions work and invalid ones are rejected.

Experience-Based Techniques

Highly dependent on the tester's experience, skill, and intuition to know where the software is most likely to fail

Error Guessing: Anticipate errors based on experience, availability of data, and knowledge of past product failures.

Exploratory Testing: No formal documentation: minimal planning time, maximal execution time, with test design and execution happening concurrently.

Worked Example

Partitioning `BigInteger.multiply`

java

/**
 * @param val another BigInteger
 * @return a BigInteger whose value is (this * val)
 */
public BigInteger multiply(BigInteger val)

The method under test.

multiply takes a two-dimensional input space: every pair of integers (a, b)
How would you partition this space into representative classes?

Sign partitions: both positive, both negative, mixed signs
Special values: a or b is 0, 1, or -1
Size partitions: a or b is small, or its absolute value exceeds Long.MAX_VALUE (roughly 2^63), since a suspicious tester assumes the implementation optimizes with int/long internally
Partitioning finds representative classes so you get good coverage without exhaustive testing

Coverage

Statement Coverage: The percentage of statements that have actually been executed by the test suite.

python

def absolute_value(x):
    if x < 0:
        x = -x
    return x

One test alone does not reach every line.

Test with x = 5: how many of the three statements does it execute?

It executes lines 1 and 3 only, skipping the negation on line 2: 66% statement coverage (2 of 3)
A test with x = -5 executes all three lines: 100% statement coverage

Branch Coverage: Stronger than statement coverage: every decision point is tested in both its True and False directions.

Path Coverage: All possible paths through the code; often impractical, since the number of paths grows exponentially with the number of decisions.

Unit Testing vs Integration Testing

Unit Test: Tests one module (a method or a class) in isolation, using stubs to replace real dependencies; when it fails, the bug is almost certainly inside that module.

Integration Test: Tests a combination of modules, or the whole program; catches bugs at the connections between modules, such as one module expecting different inputs than another actually sends it.

Aspect	Unit test	Integration test
Scope	Single module	Multiple modules
Speed	Fast	Slower
Debugging	Easy	Harder
When it fails	Bug is in this module	Bug could be anywhere
Purpose	Test logic	Test connections

Key Takeaways

Validation (verification, code review, testing) builds confidence, but never guarantees perfect quality
Exhaustive and random/haphazard testing don't work for software: use deliberate, intelligent strategies instead
Test-first programming writes specs and tests before code, catching spec bugs early
Black-box (spec-only) and white-box (implementation-aware) techniques are complementary; experience-based techniques add tester intuition
Partition the input space (sign, special values, size, boundaries) to pick representative, high-value test cases
Coverage (statement, branch, path) measures test thoroughness but never proves correctness; balance unit tests (isolation) with integration tests (connections)

Open this lesson

Mahmoud AbasSoftware Construction - Lab 3: Software Testing Techniques

Defects per kloc

Quality level

Examples

1 - 10

Typical industry software

Most commercial software

0.1 - 1

High-quality validation

Java libraries

0.01 - 0.1

Safety-critical validation

NASA, Praxis

Aspect

Unit test

Integration test

Scope

Single module

Multiple modules

Speed

Fast

Slower

Debugging

Easy

Harder

When it fails

Bug is in this module

Bug could be anywhere

Purpose

Test logic

Test connections

Software Testing Techniques

Agenda

Validation

Validation vs Verification

Verification

Validation

Perfect Quality in Software

Why Software Testing Is Hard

Test-First Programming

Software Testing Technique Types

Specification-Based Techniques

Experience-Based Techniques

Partitioning BigInteger.multiply

Coverage

Unit Testing vs Integration Testing

Key Takeaways

Open this lesson

Software Testing Techniques

Agenda

Validation

Validation vs Verification

Verification

Validation

Perfect Quality in Software

Why Software Testing Is Hard

Test-First Programming

Software Testing Technique Types

Specification-Based Techniques

Experience-Based Techniques

Partitioning BigInteger.multiply

Coverage

Unit Testing vs Integration Testing

Key Takeaways

Open this lesson

Partitioning `BigInteger.multiply`

Partitioning `BigInteger.multiply`