Number Needed to Treat Can Be Helpful: A Response to Andrade

Number Needed to Treat Can Be Helpful: A Response to Andrade

To the Editor: We read with interest the March 2015 Clinical and Practical Psychopharmacology column by Dr Andrade regarding numbers needed to treat and harm (NNT, NNH).1 We applaud Dr Andrade for writing an exceptionally clear explanation of what NNT and NNH are and how they may be calculated. In our own experience, NNT and NNH are the simplest ways of explaining effect sizes in a clinically relevant manner to medical practitioners who would otherwise mistakenly believe that all they need to be aware of are P values. Although P values can help convince us that we are most likely dealing with a truth, effect sizes are essential in helping determine if such a truth is clinically important. Therefore, in our view, the statement "The NNT is an academically useful statistic, but it has limited value for the practicing clinician"1(p e332) is unduly pessimistic, as NNT is easy to calculate and can help practicing clinicians appraise benefits and harms in a meaningful way. This is demonstrated in a number of published works examining different interventions, as, for example, in bipolar depression.2 Thus, we contend that clinicians can rapidly calculate NNT from published randomized controlled studies, easily comprehend this effect size (which reflects magnitude of therapeutic benefit in "patient units"), and intuitively integrate it into practice.

We agree with Dr Andrade’s statement that "a lot of information is lost when outcomes are dichotomized into response and nonresponse categories,"1(p e332) but emphasize that NNT and NNH are tools of particular value to clinicians and not intended to replace the usual statistical analytic techniques when designing and reporting on clinical trials. We advocate that NNT and NNH based on well-accepted and clinically relevant dichotomous benefits (such as response and remission) and harms (such as ≥ 7% weight gain) can provide a "birds-eye" view of real-world clinical outcomes that can be expected with a potential intervention. Although Dr Andrade suggests that "it is far better to directly examine by what margin drug outperforms placebo on a rating scale than to see by what margin drug outperforms placebo on an arbitrary cutoff value that defines response on that rating scale,"1(p e332) this more granular and esoteric approach implies a greater knowledge about statistics and rating scales than many clinicians possess and minimizes the importance of the "outliers" who respond by a clinically relevant amount. We contend that most clinicians will find it more difficult to understand the clinical relevance of a mean ± SD difference of 3.5 ± 1.6 points between groups on a rating scale, compared to understanding a 25% advantage in response (≥ 50% improvement) rate (ie, an NNT for response of 4).

By adhering to best practices when reporting NNT or NNH values, we can avoid the important potential problems that Dr Andrade wisely describes. These practices include (1) reporting 95% confidence intervals (CIs) for NNT and NNH and noting if the CI includes infinity (a CI that includes infinity means that the NNT and NNH estimates are not statistically significant at the P value threshold selected); (2) reporting the time frame from which data were obtained—the effect of time on benefits such as treatment response can be profound, and the longer the clinical trial, the greater the opportunity for harms such as adverse events to occur or resolve3; and (3) reporting the absolute rates from which NNT or NNH estimates were calculated—an NNT of 10 calculated from responder rates of 95% versus 85% is a very different clinical scenario compared to the same NNT calculated from responder rates of 15% versus 5%. Moreover, the individual baseline characteristics of the person being treated, and their values and preferences, will be important to know in order to optimize the use of NNT and NNH in clinical decision making.

Lastly, it needs to be emphasized that NNT values of 1 or −1 are mere theoretical constructs, as they imply absolutely perfect or absolutely imperfect therapeutic outcomes, respectively, which clearly do not have real-world clinical correlates. Because whole numbers are preferred when describing NNT or NNH, the lowest numeric absolute value (most robust effect size) encountered in clinical trials is 2, and such a value is indeed rare.

References

1. Andrade C. The numbers needed to treat and harm (NNT, NNH) statistics: what they tell us and what they do not. J Clin Psychiatry. 2015;76(3):e330-e333. PubMed doi:10.4088/JCP.15f09870

2. Ketter TA, Miller S, Dell’ Osso B, et al. Balancing benefits and harms of treatments for acute bipolar depression. J Affect Disord. 2014;169(suppl 1):S24-S33. PubMed doi:10.1016/S0165-0327(14)70006-0

3. Citrome L, Ketter TA. When does a difference make a difference? interpretation of number needed to treat, number needed to harm, and likelihood to be helped or harmed. Int J Clin Pract. 2013;67(5):407-411. PubMed doi:10.1111/ijcp.12142

Leslie Citrome, MD, MPH

[email protected]

Terence A. Ketter, MD

Author affiliations: New York Medical College, Psychiatry and Behavioral Sciences, Valhalla (Dr Citrome); and Stanford University, Psychiatry and Behavioral Sciences, Stanford, California (Dr Ketter).

Potential conflicts of interest: In the past 36 months, Dr Citrome has engaged in collaborative research with or received consulting or speaking fees from Actavis (Forest), Alexza, Alkermes, AstraZeneca, Avanir, Bristol-Myers Squibb, Eli Lilly, Forum, Genentech, Janssen, Jazz, Lundbeck, Merck, Medivation, Mylan, Novartis, Noven, Otsuka, Pfizer, Reckitt Benckiser, Reviva, Shire, Sunovion, Takeda, Teva, and Valeant. In the past 36 months, Dr Ketter has engaged in collaborative research with or received consulting or speaking fees from Abbott, Allergan, AstraZeneca, Avanir, Cephalon, Depotmed, Eli Lilly, GlaxoSmithKline, Janssen, Merck, Otsuka, Pfizer, and Sunovion. In addition, Dr Ketter’s spouse is an employee and stockholder of Janssen. No writing assistance was utilized in the production of this letter.

Funding/support: None reported.

J Clin Psychiatry 2015;76(9):e1136

dx.doi.org/10.4088/JCP.15lr10001

© Copyright 2015 Physicians Postgraduate Press, Inc.