Electroconvulsive Therapy for Depression: A Review of the Quality of ECT vs Sham ECT Trials and Meta-Analyses

Background: Electroconvulsive Therapy (ECT) is still being administered to approximately a million people annually. There have been no ECT vs simulated ECT (SECT) studies since 1985. The five meta-analyses of ECT vs SECT studies all claim that ECT is more effective than SECT for its primary target, severe depression. This review assesses the quality of those meta-analyses and of the 11 studies on which they are based.
Methods: The meta-analyses were evaluated primarily in terms of whether they considered the quality of the studies they included, but also in terms of whether they addressed efficacy beyond end of treatment. The methodological rigour of the 11 studies included by one or more of the meta-analyses was assessed using a 24-point Quality scale developed for this review.
Results: The five meta-analyses include between one and seven of the 11 studies. The meta-analyses pay little or no attention to the multiple limitations of the studies they include. The 11 studies have a mean Quality score of 12.3 out of 24. Eight scored 13 or less. Only four studies describe their processes of randomisation and testing the blinding. None convincingly demonstrate that they are double-blind. Five selectively report their findings. Only four report any ratings by patients. None assess Quality of Life. The studies are small, involving an average of 37 people. Four of the 11 found ECT significantly superior to SECT at the end of treatment, five found no significant difference and two found mixed results (including one where the psychiatrists reported a difference but patients did not). Only two higher Quality
studies report follow up data, one produced a near-zero effect size (.065) in the direction of ECT, and the other a small effect size (.299) in favour of SECT.
Conclusions: The quality of most SECT-ECT studies is so poor that the meta-analyses were wrong to conclude anything about efficacy, either during or beyond the treatment period. There is no evidence that ECT is effective for its target demographic – older women, or its target diagnostic group – severely depressed people, or for suicidal people, people who have unsuccessfully tried other treatments first, involuntary patients, or adolescents. Given the high risk of permanent memory loss and the small mortality risk, this longstanding failure to determine whether or not ECT works means that its use should be immediately suspended until a series of well designed, randomised, placebo controlled studies have investigated whether there really are any significant benefits against which the proven significant risks can be weighed.