The Dodo Bird, the Archbishop, and the Squirrel: A Review and Meta-Analysis of Psychotherapy Outcome Equivalence

Abstract

The Dodo Bird Verdict (DBV)—the proposition that all psychotherapies are equally effective—remains controversial. Although meta-analyses have been the primary tool used to arbitrate disagreements about relative psychotherapy efficacy, the meta-analytic literature itself has not reached consensus as to whether psychotherapeutic treatments are differentially effective. In this thesis by articles, I explored the extent to which conceptual and methodological decisions in meta-analytic studies have impacted meta-analytic conclusions about the DBV. The first chapter—a general introduction—frames the history and philosophical issues associated with meta-analysis of the DBV. The second chapter—a review article—examines several meta-analyses of psychotherapy outcomes in order to highlight key methodological issues in the DBV meta-analytic literature, including the inclusion of direct versus indirect comparisons; the restriction of analyses to bona fide treatments; the outcomes to be included and distinguished in the analysis; statistical considerations; and possible moderators for use in meta-regression analysis, including treatment class, disorder, allegiance, and methodological quality. The third chapter—the meta-analytic paper—implements those methodological recommendations in meta-analytic tests of the DBV. Eligible studies for inclusion were English-language comparisons of psychotherapies for adults, compared directly to one another head-to-head, and published after the year 1980. To be retained, studies were required to (a) be randomized controlled trials, (b) involve direct comparisons of (c) “bona fide” psychotherapies, (d) evaluate therapies in populations with diagnosed psychological disorders, and (e) have no apparent confounds with medication use across the therapies being evaluated. Studies were excluded if they (a) reanalyzed a dataset that has already been included or (b) did not include information from which effect sizes can be calculated (e.g., means, sample sizes, variances). Studies were collected via a hand-search of key journals (Behavior Therapy, Behaviour Research and Therapy, Journal of Consulting and Clinical Psychology, Journal of Counseling Psychology, JAMA Psychiatry, and Cognitive Therapy and Research, Psychotherapy, Psychotherapy Research), a systematic term search of the ProQuest Dissertations and Theses Global database, and a search of the reference list of extant meta-analyses, collected via a systematic search of indexed subject-terms and title/abstracts in the PSYCInfo, Medline, and Proquest Dissertations and Theses databases. Risk of bias was assessed using the Risk of Bias 2 (RoB2) tool. Studies were meta-analyzed (1) all together, with randomly distributed signs, using a homogeneity test to assess significant differences, (2) all together, using the absolute value of the effect size to determine the magnitude of the effect of Treatment A vs. Treatment B, and (3) divided into treatment families (e.g., CBT, Psychodynamic), and compared using standard meta-analytic methods with CBT as the reference class. These three approaches revealed significant differences among individual treatments, (e.g., for the overall analysis of the absolute valued effect size between treatments, an average effect of g = 0.27, 95% CI [0.23, 0.30]). However, when divided into treatment families, the effect size within families (e.g., CBT vs. CBT), g = 0.24, 95% CI [0.19, 0.29], was not meaningfully different from the effect of one family vs. another (e.g., CBT vs. other), g = 0.15, 95% CI [0.06, 0.23]. The final chapter—a general conclusion—reviews important limitations of the data (e.g., overall high risk of bias) and explores the implications of these results both for public health and for the conceptual foundations of the DBV.