Absence of Evidence Is Not Evidence of Absence: Reading a Nonsignificant Result

When a study reports no significant difference, that is a statement about the evidence, not about reality. It usually means the study was unable to detect an effect, not that it proved the effect is zero. The way to tell the two apart is to read the confidence interval instead of the p value, because a wide interval that still reaches into meaningful benefit is compatible with a real effect the study simply lacked the power to find. Only when the interval is narrow and sits close to zero can you say the study genuinely rules out an important difference.

The mistake in the word negative

When a study finds no statistically significant difference, it is often called negative, and the word does real damage. Negative sounds like proof that the treatments are equal, but usually all the study has shown is that it failed to find a difference.

Failing to find something and showing it is absent are not the same claim. A quiet room might mean no one is there, or it might mean you did not listen long enough.

Read the interval, not the verdict

The p value gives you a yes or no about statistical significance, and in doing so it throws away the information you most need. The confidence interval keeps it. The interval shows the range of effects compatible with the data, and its width tells you what the study could and could not rule out.

A nonsignificant result with a wide interval that still stretches into meaningful benefit is entirely consistent with a real effect the study was too small to detect. The same verdict with a narrow interval sitting close to zero is a much stronger statement.

Power is why big questions need big trials

Statistical power is the ability of a study to detect an effect that is really there. A small study has low power, so it can miss a genuine effect and report no significant difference simply for lack of participants.

This is why a single small trial finding nothing should rarely settle a question, and why pooling studies in a meta-analysis can reveal an effect that each study alone was too underpowered to show. Absence of a signal in a small study is often a statement about the study's size, not about the treatment.

When you can legitimately claim no effect

There is a right way to show that a meaningful difference is absent, and it takes deliberate design. Equivalence and noninferiority studies set, in advance, a margin for what counts as a meaningful difference, and they are built large enough that the confidence interval can fall entirely inside that margin.

When the whole interval excludes any important effect, you can reasonably say the study ruled one out. That is a positive demonstration of similarity, not the accidental silence of an underpowered trial.

How this shows up in practice

You will meet this whenever a headline announces that a treatment did not work or that a risk was not linked to an exposure. Before accepting it, find the confidence interval and ask what effects it still allows.

Ask how many people were studied and whether the study was ever large enough to detect the effect in question. Reading a nonsignificant result well is mostly a habit of refusing to let a p value stand in for a conclusion the data cannot support.