M07-L02 — Prove tests catch a meaningful defect

Recording transcript

A passing test becomes more informative when you know a meaningful defect it can detect. In this lesson, we will deliberately change one freshness comparison, show that the wrong version still compiles, and inspect the resulting behavioral failure. Use a new checkpoint-seven exercise copy. Preserve the good source and its result before introducing the fault.

Read the requirement first. A sample remains current at an age of exactly three thousand milliseconds. It becomes stale only when its age is greater than that limit. This distinction defines the correct comparison. The test's expected result must come from that contract, not from whichever operator happens to be in the code today.

Open the shared policy implementation and locate Monitor poll. Inspect the whole condition before editing. It checks that a value is current and compares unsigned elapsed time with the stale limit. The intentional mutation changes the greater-than operator to greater-than-or-equal. Nothing else should change. We want to isolate one semantic error that a reader can explain from a small diff.

This mutation is different from the supplied fault-stale checkpoint. That checkpoint disables expiration entirely. Here, expiration still happens, but it happens one boundary step too early. Keeping the two faults distinct prevents us from attributing the wrong cause to the observed output. Both can teach useful lessons, but they answer different questions about the tests.

First run the unmodified basic tests and preserve the actual result. If the baseline does not pass, investigate that condition before adding a deliberate defect. Otherwise, you cannot separate an existing problem from the mutation you are about to introduce. Record the source identity and command so the starting evidence remains connected to the right exercise copy.

Now make the one-character change in the disposable source. Review the diff immediately. Confirm that the changed comparison is inside the polling expiration guard rather than in another input-validation path. An accidental additional edit would make the experiment harder to interpret. Preserve the good copy instead of relying on memory to reconstruct it later.

Run the same maintained test wrapper. The wrong version should compile and then be rejected by the relevant behavioral assertions. If you accidentally create a syntax error, fix that typing mistake before interpreting the experiment. A compiler error proves that the compiler rejected invalid source; it does not prove that the freshness assertions distinguish the wrong equality behavior.

Read the assertion failure and connect it to the requirement. The ordinary case uses a sample at timestamp one thousand and checks it at four thousand. Its age is exactly three thousand, so it must remain current. The mutated comparison marks it stale. That is the concrete trigger and consequence, stated independently of the test framework's wording or line number.

The rollover equality case can expose the same mistake across the unsigned clock boundary. It checks the contract under another relevant timestamp arrangement. Do not treat multiple failed assertions as unrelated bugs automatically. A single wrong comparison can cause several observations to disagree with the same requirement. Explain their shared cause before broadening the investigation.

Do not change the expected values to make the wrong source pass. That would erase the independent requirement and teach the tests to accept the defect. Tests can be wrong, but correcting them requires an argument from the specification. Here the equality condition is explicit, so the implementation is the thing that must return to the documented behavior.

Restore exactly the original greater-than comparison. Inspect the diff again and run the same test command. The original good version should still pass. Preserve the bad result as well as the restored result. A final passing screenshot alone would hide the useful demonstration that the suite rejected the specific defect.

Because the exercise changed shared firmware source, reproduce the firmware cross-build after restoration using the maintained build wrapper. Record its actual outcome separately from the host assertion result. The cross-build checks the restored source against the selected target inputs. It does not establish an upload, a physical reading, or real device timing.

Your evidence record should contain the baseline result, the seeded diff, successful compilation of the bad version, the relevant assertion failure, the restoration, and the passing rerun. Explain which condition failed and why. You do not need to claim that every possible defect would be detected. The evidence supports this particular test capability.

Preserve standard output and diagnostic output when recording the run. The process exit communicates success or failure, while the assertion message explains the disagreement. A screenshot that shows only a command prompt after execution can lose both pieces of information. Keep the receipt readable enough to establish which source ran and why the behavioral check rejected it.

Pause now to complete the exercise. If the result differs from the expected sequence, preserve it and inspect the active checkpoint, exact changed line, and command. A different source version or accidentally modified assertion can explain the discrepancy. Do not replace the actual output with the course's expected result just to make the demonstration appear complete.

When reviewing your submission, ask whether a person can follow the chain from requirement to mutation to observed failure to recovery. A tiny code change can have a precise behavioral consequence. The test is useful because it makes that consequence visible, not because it produces a large amount of terminal output or an impressive number of assertions.

Keep the meaningful rejection case with your verification notes. After a relevant refactor, it can help confirm that the suite still distinguishes premature staleness from the required boundary. The general habit transfers beyond this project: preserve a good baseline, seed a realistic defect in isolation, prove behavioral rejection, and restore the original result without concealing any part of the evidence.

Download transcript