Why Psychotherapy Trials Are Hard to Blind and What That Does to the Evidence

The short answer

A psychotherapy trial cannot be run the way a drug trial can, because in talk therapy both the patient and the therapist always know exactly what is being delivered. That single structural fact, layered on top of who designed the study and what the treatment was compared against, tends to push reported effects upward. None of it proves a therapy does not work, but it means the headline number should be read with the design in mind. Reading these trials well is largely a matter of asking how much of the measured benefit could come from the setup rather than the therapy itself.

Why blinding breaks down

Blinding exists to keep expectation from masquerading as effect. When a patient believes they are receiving a powerful treatment, they often feel and report improvement regardless of the treatment's specific action, and when a clinician believes in what they are delivering, that belief can shape how they rate the outcome. Double-blinding severs both links at once.

Psychotherapy offers almost nowhere to hide this. The therapist has to know the protocol in order to deliver it. The patient experiences the sessions and can usually tell whether they are getting active treatment or sitting in a control condition. Dragioti and colleagues, writing in Annals of General Psychiatry in 2015, put the constraint plainly: in psychotherapy research, double-blind designs cannot be performed, so the usual experimenter biases are very difficult to address. This is not a flaw in any one study. It is a permanent feature of the field, and it means the safeguards that anchor pharmaceutical evidence are simply unavailable.

Outcome assessment is where some ground can be recovered. Even when patients and therapists cannot be blinded, the person rating symptoms sometimes can be, and self-report scales can be supplemented with independent evaluators kept unaware of group assignment. Trials that take this step are more trustworthy than trials that let the treating therapist score the result.

The allegiance problem

The people most motivated to run a psychotherapy trial are frequently the people who developed or champion the therapy being tested. That enthusiasm has a measurable footprint. Researcher allegiance describes the tendency for a study to favor the treatment its investigators believe in, and the evidence that it moves results is strong.

Munder and colleagues, in a 2013 overview of reviews in Clinical Psychology Review, pooled findings across roughly 30 meta-analyses and found a consistent association between researcher allegiance and reported outcomes, on the order of a correlation of about 0.26, a moderate and robust effect that persisted across different ways of measuring allegiance. Dragioti and colleagues later examined this directly in randomized controlled trials and reported a summary odds ratio around 1.31 in favor of treatments the researchers were aligned with, a figure whose confidence interval sat just above no effect. The pattern was strongest when the same team both developed the therapy and trained or supervised the therapists delivering it. Allegiance is best understood as a non-financial conflict of interest, a form of optimism bias, and it does not require anyone to behave dishonestly.

What the control group is really doing

The comparison group decides what a trial's effect size means, and in psychotherapy the most convenient comparator is often the most misleading one. A waitlist control assigns some participants to simply wait for treatment. On paper this looks like a neutral no-treatment baseline. In practice it may be worse than neutral.

Furukawa and colleagues, in a 2014 network meta-analysis of cognitive behavioral therapy for depression in Acta Psychiatrica Scandinavica, argued that waiting may act as a nocebo condition. People told they must wait may hold off on other coping efforts and settle into the expectation that nothing will improve until their turn arrives, so their symptoms stagnate. In that analysis the odds of response were higher for people assigned to no treatment or to a psychological placebo than for people assigned to a waitlist, which is the opposite of what a neutral baseline should show. When the treatment group is measured against a comparator that is drifting the wrong way, the gap between them widens for reasons that have nothing to do with the therapy.

The size of this distortion is not small. Laws and colleagues, reanalyzing trials of exposure therapy for obsessive compulsive disorder in Frontiers in Psychiatry in 2022, reported effect estimates several times larger when the therapy was compared against a waitlist than against other control conditions, and found that whether a trial used a waitlist accounted for roughly a third of the variance in the effect sizes. Same therapy, different yardstick, very different headline number.

How to read these trials

A few questions do most of the work. Who developed the treatment, and were they involved in running the trial and training the therapists? Was anyone kept unaware of group assignment when outcomes were scored, or did the treating clinician also rate the result? What was the therapy compared against, and if it was a waitlist, would the more demanding comparison against usual care or an active control likely have shrunk the effect? Was treatment fidelity checked?

None of this means psychotherapy does not work. Many psychological treatments show genuine, replicated benefit against strong comparators. It means the specific number attached to a given trial should be read as an upper bound shaped by design as much as by therapy, and that effects measured against a waitlist by an allegiant team deserve the most caution. This article is educational and is not medical advice.