Compliance
Test and retest: what one run proves
A passing run is a sample, not a guarantee, and the discipline of deciding how much repetition a conclusion needs is older than electronics.
What one run proves
A single compliance run proves that a device met its limits once, under one set of conditions, on one bench. It does not prove the device will do it again, and the difference between the two claims is the entire subject of test and retest.
The first run answers the question it was asked: did this device, on this day, in this fixture, at this temperature, meet the figure? A passing run is evidence, and it is bounded evidence. The device under test is one sample of a population; the bench is one configuration of a methodology; the conditions are one point in an envelope. The run's conclusion travels no further than its frame.
The habit of asking what a repetition would add is shared by every field that measures performance. The sport-science desk that keeps the test-retest habit treats a single countermovement jump the same way a lab treats a single eye diagram: a measurement whose meaning lives in its repeatability, and whose reported value should carry the variation between runs, not just the run. The jump and the eye are the same epistemology with different instruments.
What does the second run add?
Information about variance rather than about the mean. If the first run gave a figure near a limit, the second run tells whether that figure was the device or the noise. Runs that repeat tightly make the measurement trustworthy; runs that scatter make it a sample from a distribution the first run did not show. In compliance work this matters most at the margin: a device that passes by a wide margin needs few runs to characterize, while a device that passes by a hair needs enough runs to show which side of the line the population actually sits on.
The rerun also interrogates the bench. When a repeated run moves, the movement is shared between the device and the measurement chain, and the page on what the bench controls is where that division is made. A retest that moves within the bench's own variation is not evidence the device moved.
Why does the program specify runs at all?
Because the alternative is each vendor defining its own confidence. The compliance program described in how a compliance program works fixes the test points, the fixtures and the pass criteria precisely so that a pass means the same thing across vendors. Within that frame, the number of samples and the number of runs become a statistical question about populations: one golden device passing once says little about a production run, and the program's sampling expectations exist to make the result speak for more than the unit on the bench.
| Run | What it adds | What it still cannot show |
|---|---|---|
| First | A point estimate at one condition | Whether the figure repeats |
| Second | A first look at variance | Whether the variance is the device or the bench |
| Third and beyond | A distribution, if conditions held constant | Whether the sample represents production |
| Across devices | A population estimate | Whether the population is stable over time |
How do you read a margin from one run?
Carefully, and with the rerun in mind. A margin measured once is a margin measured at the bench's own noise level, and the same device measured again may show a different residue. The honest reading of a single-run margin is not the number but the number plus the bench's repeatability, which is why the bench page and this one belong together: the figure's frame is set by what the bench controls, and the figure's reliability is set by what the rerun shows.
Where a device sits near a limit, the professional answer is not a stronger opinion about the single run but more data: another run, another unit, another temperature. The instinct is the one the verification literature formalizes in verification and validation, read on September 6, 2026: a result that will not bear repetition is a result that has not yet been measured.
Checks before trusting a single run
- Ask where the figure sits against the limit: a wide pass needs fewer reruns than a marginal one.
- Repeat before concluding: the second run is the first information about variance.
- Hold the bench constant between runs, or the rerun measures the bench.
- Remember the frame: one device, one bench, one condition is the claim's whole territory.
Common mistakes
- Reading one pass as a property of the part rather than a property of the run.
- Rerunning with a changed bench and calling the difference device variance.
- Treating a marginal pass and a wide pass as the same confidence.
- Averaging runs without asking whether the conditions stayed fixed.
The first run is a hypothesis about the device; the second run is the first test of the hypothesis. Everything after that is statistics, and statistics is the only honest way a single bench speaks about a population it has sampled once.
The page behind these facts
The framing of measurement as evidence with a bounded frame follows the verification and validation literature linked above, read on September 6, 2026. The compliance context, the test points and the program's role in standardizing the bench are described on this desk's page about how a compliance program works, and the measurement itself on the page about reading a receiver eye.