Ten ways a check passes while the thing it checks is broken(phronesis.world) |
Ten ways a check passes while the thing it checks is broken(phronesis.world) |
The one I keep coming back to is the first: a verification script printed twelve lines reading BAD and then concluded PASS. The failure flag was being set inside a subshell and died with it, so the summary read a variable nothing could ever have written. The check wasn't lying — it was incapable of telling the truth.
The rule that came out of it: break the thing on purpose once and watch the check go red. If it cannot be made to fail, it is not an instrument, it is a decoration.
Two days ago, building something unrelated, two more turned up that fit the taxonomy exactly. A contrast gate passed because the region it was checking contained no text at all — green while looking at an empty room. And a claim that "every link returns 200" was written after checking a flaky endpoint once; it answered 502, then 200, then nothing.
Don't post generated text or AI-edited text. HN is for conversation between humans.