See reply by Dongen-Boomsma et al and related article by Dongen-Boomsma et al.
Doubting the Efficacy/Effectiveness of Electroencephalographic Neurofeedback in Treating Children With Attention-Deficit/Hyperactivity Disorder Is As Yet Unjustified
To the Editor: Van Dongen-Boomsma et al1 assessed the efficacy of electroencephalographic (EEG) neurofeedback in children with attention-deficit/hyperactivity disorder by comparing an experimental group (n = 22) receiving EEG neurofeedback to a placebo group (inactive neurofeedback) (n = 19). Results did not show significant differences between the 2 groups. The authors report that, post hoc, the "sample had 80% power to detect a treatment effect of 0.90" and that "it is unlikely that the negative results were due to limited statistical power,"1(p826) concluding that their study sheds serious doubt on the effectiveness of EEG neurofeedback as a way of treating children with attention-deficit/hyperactivity disorder.
First, we want to acknowledge the robust design and methodology of this study. However, we are not sure what the authors meant by stating in their discussion that "the sample had 80% power to detect a treatment effect of 0.90," and we cannot find an explanation of the rationale behind such a statement. How would statistical power be an issue if the effect size was as high as 0.90 in a sample of 41 participants? We ran a post hoc power analysis, using G*Power Version 3.1.5,2,3 for a sample size of 41 participants, α of 5% and an effect size of 0.90 resulting from repeated-measures analysis of variance within-group and between-groups interaction tests for 2 groups and 2 measuring times, and found that this sample would have achieved 100% power to detect a treatment effect size of 0.90, not 80% as reported by the authors. Therefore, there is a 100% chance of finding statistically significant differences (P ≤ .05) between the EEG neurofeedback group and the placebo group,4 although none were found. Impossible, unless the effect size equaled zero. We suspect that the authors made a calculation error.
To verify this, we first calculated η2 effect size for all the comparisons in the group ×— time effect (interaction) using the following formula5: (F ×— df1)/[(F ×— df1) + df2)]. We found that 0.13% ≤ η2 ≤ 5.48% (Table 1). Then, with these effect sizes, we ran a post hoc power analysis for each comparison, following the same procedure, using G*Power Version 3.1.5.2,3 Our calculations show that the achieved power varies between 5% and 11% (Table 1). On the basis of these findings, the risk of a type II error in this study varies between 89% and 95%. Hence, the authors’ conclusion that it is unlikely that the negative results were due to a lack of statistical power is conceptually wrong.
Click figure to enlarge
We read van Dongen-Boomsma and colleagues’ 1 study with great interest, and, given the η2 effect sizes we calculated, we agree that the differences between the effect of EEG neurofeedback and the placebo neurofeedback are small, and we believe that they probably would not be much higher in a larger sample if inclusion criteria were based on the same demographic and clinical characteristics. Nonetheless, given the lack of statistical power we showed above, it is not methodologically sound to describe the study as one with negative results. Clarification from the authors of the rationale and what they meant in saying that "the sample had 80% power to detect a treatment effect of 0.90" would be useful. Further research with sound statistical power and validity is needed to corroborate van Dongen-Boomsma and colleagues’ conclusion doubting the efficacy/effectiveness of EEG neurofeedback, which has so far been demonstrated in other methodologically sound studies.6,7
References
1. van Dongen-Boomsma M, Vollebregt MA, Slaats-Willemse D, et al. A randomized placebo-controlled trial of electroencephalographic (EEG) neurofeedback in children with attention-deficit/hyperactivity disorder. J Clin Psychiatry. 2013;74(8):821-827. PubMed doi:10.4088/JCP.12m08321
2. Faul F, Erdfelder E, Lang A-G, et al. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191. PubMed doi:10.3758/BF03193146
3. Faul F, Erdfelder E, Buchner A, et al. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149-1160. PubMed doi:10.3758/BRM.41.4.1149
4. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum; 1988.
5. Guilford JP. Fundamental Statistics in Psychology and Education. New York, NY: McGraw-Hill; 1965.
6. Arns M, de Ridder S, Strehl U, et al. Efficacy of neurofeedback treatment in ADHD: the effects on inattention, impulsivity and hyperactivity: a meta-analysis. Clin EEG Neurosci. 2009;40(3):180-189. PubMed doi:10.1177/155005940904000311
7. Gevensleben H, Holl B, Albrecht B, et al. Is neurofeedback an efficacious treatment for ADHD? a randomised controlled clinical trial. J Child Psychol Psychiatry. 2009;50(7):780-789. PubMed doi:10.1111/j.1469-7610.2008.02033.x
Author affiliations: Department of Psychology, University of Quebec (all authors), and Faculty of Medicine, McGill University (Dr El-Baalbaki), Montreal, Quebec, Canada.
Potential conflicts of interest: None reported.
Funding/support: None reported.
J Clin Psychiatry 2014;75(7):778-779 (doi:10.4088/JCP.14lr09043).
© Copyright 2014 Physicians Postgraduate Press, Inc.
