Small-Study Effects: What the Egger Test and Trim-and-Fill Can and Cannot Show

Small-study effects describe a common pattern in meta-analysis where smaller trials report systematically larger effects than big ones. Funnel plots display it, Egger's test puts a number on the asymmetry, and trim-and-fill estimates what the pooled result might look like if suppressed studies were added back. All three are useful screens, but asymmetry has several causes besides publication bias, so treat them as prompts for investigation rather than proof that evidence was hidden.

Small-study effects, and why publication bias is only one cause

In many meta-analyses, if you sort the trials by size, the small ones report bigger benefits than the large ones. That tendency has a name, small-study effects, and it is worth naming carefully because it is often mislabeled as publication bias.

Publication bias is one explanation: small trials with disappointing results are more likely to stay unpublished, so the small studies that do surface are a flattering selection. But it is not the only explanation. Small trials are sometimes run in higher-risk patients who have more to gain, which is real heterogeneity, not bias. Smaller studies also tend to have weaker methods, which can inflate effects. And with few studies, plain chance can produce a lopsided picture. Keeping these apart matters, because the fix for each is different.

Reading the funnel plot

The funnel plot is the standard picture. Each study is a dot, plotted with its effect estimate on one axis and its precision, usually the standard error, on the other. Large, precise studies sit near the top and cluster tightly around the pooled effect. Small, imprecise studies scatter widely at the bottom.

If nothing is skewing the evidence, the dots should form a roughly symmetric inverted funnel. When the bottom of the funnel is lopsided, with a gap where small unfavorable studies should be, that asymmetry is the visual signature of small-study effects. The plot is only a screen, and it becomes hard to read when there are few studies, but it frames every formal test that follows.

Egger's test: putting a number on asymmetry

Eyeballing a funnel plot is subjective, so Egger and colleagues proposed a regression that measures its asymmetry. In essence it regresses the standardized effect against precision and checks whether the intercept departs from zero. A significant intercept indicates that the smaller, less precise studies are pulling in a systematic direction.

The test has real limits. Its power is low when only a handful of trials are pooled, so a nonsignificant result is weak reassurance rather than an all-clear. Guidance recommends applying it only when there are at least ten studies. It can also flag asymmetry for reasons unrelated to bias, and for some binary outcomes the original version is unreliable, which is why alternatives designed for those situations exist. A significant Egger test says the funnel is lopsided, not why.

Trim-and-fill: a sensitivity analysis, not a correction

Trim-and-fill goes one step further and asks what the answer might be if the missing studies were restored. It is a nonparametric method that estimates how many studies would need to be removed to make the funnel symmetric, then imputes their mirror images on the empty side and recomputes the pooled effect.

It is easy to misread the adjusted number as the corrected truth. It is not. Trim-and-fill assumes the asymmetry comes from suppression and that the missing studies mirror the observed ones, and when the real cause is heterogeneity rather than bias it can over-correct or under-correct badly. The right way to use it is as a sensitivity analysis: if the pooled effect survives the adjustment more or less intact, that is reassuring, and if it collapses, the original result was fragile.

Distinguishing causes with contour-enhanced plots

Because asymmetry has several causes, a plain funnel plot cannot tell you whether you are looking at publication bias or something more innocent. A contour-enhanced funnel plot helps. It shades the regions of the plot by statistical significance, so you can see where the apparent gaps fall.

If the missing studies would have landed in the zone of nonsignificant results, suppression of unwelcome findings becomes a plausible reading. If instead the gaps fall in significant regions, publication bias is a poor fit and you should look to heterogeneity or study quality. This small addition turns a vague impression of lopsidedness into a more specific question about mechanism.

What to take from an asymmetry analysis

The honest summary is that these tools detect a pattern, not its cause. Funnel asymmetry, a positive Egger test, and a shifting trim-and-fill estimate together tell you that the smaller studies are behaving differently from the larger ones and that the pooled effect may be optimistic.

What they do not deliver is a verdict that evidence was hidden. When you meet these analyses in a review, read them as an invitation to ask harder questions: were small trials run in different patients, were their methods weaker, were enough studies included for the tests to mean anything? A meta-analysis that reports these screens and interprets them cautiously is being candid about its own fragility, which is exactly what you want.