Why passing tests is not enough

A green test suite is useful evidence, but its meaning is narrower than its color suggests. It says that the checks which executed matched their expected outcomes. It does not tell you whether the suite distinguishes the required behavior from the mistakes a maintainer is likely to make.

The difference becomes clear when you stop counting tests and start asking which defects they would detect. One assertion at the right boundary can be more informative than many examples that all follow the same easy path.

Start with a rule precise enough to contradict

Consider an illustrative discount function: an order of at least 100 units receives a 10% discount, and negative totals are rejected. The example uses simple numeric inputs to discuss boundaries; production monetary calculations would need an explicit decimal and rounding policy.

def discounted_total(total):
    if total < 0:
        raise ValueError("negative total")
    return total * 0.9 if total >= 100 else total

A test that checks only a total of 200 will pass for both >= 100 and the incorrect > 100. The test executes the discount branch, but it does not distinguish the boundary rule. Code coverage alone cannot supply that missing distinction.

Choose examples around the decision

Use values just below, exactly at, and just above an important threshold. Then add a separate check for invalid inputs. With floating-point arithmetic, use an appropriate tolerance for numeric comparisons rather than depending on every decimal value having an exact binary representation.

import math

assert discounted_total(99) == 99
assert math.isclose(discounted_total(100), 90)
assert math.isclose(discounted_total(101), 90.9)

try:
    discounted_total(-1)
except ValueError:
    pass
else:
    raise AssertionError("negative totals must be rejected")

Each assertion now has a reason to exist. The test at 100 protects inclusivity; the negative example protects validation. Adding another ordinary discounted value may be less valuable than covering a different rule.

Make a plausible mistake on purpose

Temporarily change >= to >, remove validation, or return the original total from the discounted branch. If the suite still passes, investigate whether the altered behavior violates a requirement. This is the idea behind mutation testing: introduce small code changes and observe whether tests detect them.

Google's account of mutation testing also explains why not every generated mutation is useful. A surviving mutation can reveal a missing assertion, an equivalent implementation, unreachable behavior, or a requirement that was never intended. Treat the result as a review prompt rather than a target to maximize mechanically.

Always establish that the correct baseline passes first. If the original implementation is already failing, a failure in a modified version does not show that the suite detected the modification.

Assert the effect, not only the call

In service code, a common weak test verifies that a mocked helper was called and stops there. That may protect a particular implementation while leaving the business outcome unspecified. If an operation reserves stock, inspect the resulting reservation and available quantity. If failure should roll back both changes, verify the persisted state after the failure.

Mocks remain useful for controlling an external boundary or reproducing an error. The question is what the assertion proves. A mock returning success cannot establish that a real database constraint behaves correctly under concurrent writers.

Choose different tests for different uncertainty

Unit tests are well suited to isolated rules and many boundary combinations. Integration tests help verify framework, persistence, and protocol behavior that a substitute cannot establish. A small end-to-end path can check that the important pieces are connected, but it is usually an expensive place to exercise every arithmetic edge case.

Tests should also make failures easy to interpret. Use independent setup, explicit expectations, and a focused purpose. If a failure requires reconstructing several unrelated scenarios, the suite is spending the maintainer's attention inefficiently.

A practical next step

Choose one important function in a real codebase. Write down two plausible defects before opening the test file. Predict which assertions will fail for each defect, then check your prediction. If the answer is unclear, the test may be documenting implementation detail rather than behavior.

You can practice the same process in the Test Quality Lab. Run the starting suite, inspect surviving mutations, and add an assertion that explains a missing rule. A useful suite earns trust by exposing meaningful mistakes, not by remaining green at all costs.