See original letter by Carroll and related article by Gibbons et al.
Dr Gibbons and Colleagues Reply
To the Editor: As indicated in the recent review by Maurer,1(p139) "depression has an estimated prevalence of 5.4 to 8.9 percent in the US general population2 and affects 5 to 13 percent of patients in primary care settings.3" The condition accounts for more than $43 billion in medical care costs and $17 billion in lost productivity annually.4 Depression is projected to become the second largest cause of worldwide disability by 2020.5 The most frequently used tools for screening of depression in primary care are the Patient Health Questionnaire-2 (PHQ-2) and PHQ-9; however, both suffer from poor sensitivity: 61% and 74%, respectively,6 in primary care settings and much lower in cardiology settings (eg, 39% for the PHQ-2 in cardiology settings7). The development of new statistical tools with higher sensitivity for screening depression in primary care is critical. The Computerized Adaptive Diagnostic Test for Major Depressive Disorder (CAD-MDD)8 is clearly such a tool. In the mental health settings in which we have studied the CAD-MDD (a mixture of academic medical center and community mental health settings), it dramatically increases sensitivity to 0.95 with comparable specificity to existing methods. Assuming the CAD-MDD is comparably sensitive in primary care settings, for every 100 cases of major depressive disorder the PHQ-2 will miss 39, the PHQ-9 will miss 26, but the CAD-MDD will miss only 5. As we have shown in our article, all of this can be achieved using an average of only 4 items in less than 1 minute anywhere on the planet via the Internet or in a kiosk in a health care provider’s office. We think that this is a good thing and will continue to study the properties of the CAD-MDD in different populations in which, as we noted in our article, variation in case mix and prevalence may well affect sensitivity. The use of this new statistical approach to develop screening instruments for other disorders should remain a high priority on our nation’s mental health agenda.
Dr Carroll raises the conjecture that "specificity was probably no better than 0.50 in the clinical subsample and close to 1.0 in the ‘ scrubbed’ control subsample." In fact, for the clinical subsample (Western Psychiatric Institute and Clinic, Pittsburgh, Pennsylvania), both sensitivity and specificity remained high at 0.92 and 0.85, respectively, and did not degrade as Dr Carroll suggests. In the community mental health clinic (DuBois Community Mental Health Center, DuBois, Pennsylvania), sensitivity was 1.00 and specificity was 0.88. These findings reveal that the CAD-MDD is robust to both psychiatric comorbidity and the overall incidence of depression. We note that the incidence was 30% in the "clinical" sample and 15% in the community mental health center. The community mental health center has an incidence that is not dissimilar to what is typically found in primary care settings.3 Nevertheless, sensitivity was perfect and specificity remained high.
As Dr Carroll suggests, positive predictive value (PPV) is strongly related to prevalence. The PPV value that we report (0.66) is based on an overall prevalence rate of 20.4%. In our medical center sample the prevalence of depression was 30.2% and PPV was 0.72 and negative predictive value (NPV) was 0.96. In our community mental health center sample, for which rates approach those observed in primary care (14.8%), PPV was 0.59 and NPV was 1.00. These estimates based on actual data are a far cry from Dr Carroll’s estimates of PPV in the range of 0.05 to 0.20. Of course, the best estimates will be based on application of the CAD-MDD in primary care. Collection of those data is currently underway.
As a final note, there are numerous ways of describing the information contained in a 4-fold table. PPV is only one. The choice of method depends, in large part, on whether there are differential consequences of false positive and false negative rates. With respect to DSM diagnostic criteria, there are patients with considerable depression who do not meet DSM criteria for MDD. However, a good screening tool would include these individuals as predicted cases to be further evaluated. This will, of course, lead to decreased PPV and specificity, which is the case for any imperfect "gold standard." In our view, the unique contribution of the CAD-MDD is its ability to produce high sensitivity while maintaining reasonably high specificity. In this way, we do not miss true cases, but include a small proportion of borderline cases that warrant further evaluation but might otherwise go undetected. The alternative is to require, for example, that all primary care patients undergo a complete diagnostic interview performed by a trained clinician. Of course, this could never happen. The CAD-MDD represents a highly reliable screening tool that provides the clinician with a way to determine which patients require further evaluation.
References
1. Maurer DM. Screening for depression. Am Fam Physician. 2012;85(2):139-144. PubMed
2. Narrow WE, Rae DS, Robins LN, et al. Revised prevalence estimates of mental disorders in the United States: using a clinical significance criterion to reconcile 2 surveys’ estimates. Arch Gen Psychiatry. 2002;59(2):115-123. PubMed doi:10.1001/archpsyc.59.2.115
3. Coyne JC, Fechner-Bates S, Schwenk TL. Prevalence, nature, and comorbidity of depressive disorders in primary care. Gen Hosp Psychiatry. 1994;16(4):267-276. PubMed doi:10.1016/0163-8343(94)90006-X
4. Chisholm D. The economic costs of depression. In: Dawson A, Tylee A, eds. Depression: Social and Economic Timebomb. London, United Kingdom: BMJ Books; 2001.
5. Murray CJ, Lopez AD. Evidence-based health policy—lessons from the Global Burden of Disease Study. Science. 1996;274(5288):740-743. PubMed doi:10.1126/science.274.5288.740
6. Arroll B, Goodyear-Smith F, Crengle S, et al. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med. 2010;8(4):348-353. PubMed doi:10.1370/afm.1139
7. Gilbody S, Richards D, Brealey S, et al. Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med. 2007;22(11):1596-1602. PubMed doi:10.1007/s11606-007-0333-y
8. Gibbons RD, Hooker G, Finkelman MD, et al. The computerized adaptive diagnostic test for major depressive disorder (CAD-MDD): a screening tool for depression. J Clin Psychiatry. 2013;74(7):669-674. PubMed doi:10.4088/JCP.12m08338
Author affiliations: Center for Health Statistics, University of Chicago, Illinois (Dr Gibbons); Department of Statistics, Cornell University, Ithaca, New York (Dr Hooker); Department of Public Health and Community Service, Tufts University School of Dental Medicine, Boston, Massachusetts (Dr Finkelman); Department of Psychology, University of Minnesota, Minneapolis (Dr Weiss); and Western Psychiatric Institute, University of Pittsburgh, Pennsylvania (Drs Pilkonis, Frank, and Kupfer and Ms Moore).
Potential conflicts of interest: The CAD-MDD will ultimately be made available for routine administration through Adaptive Testing Technologies, in which Drs Gibbons, Weiss, Pilkonis, Frank, and Kupfer have financial interests. Dr Gibbons has been an expert witness on cases related to issues involving drugs and suicide for the US Department of Justice, Wyeth, and Pfizer over the past 10 years. Dr Frank has been a speaker or a member of advisory boards for Servier International, Guilford Press, and the American Psychological Association. Dr Kupfer is a consultant for the American Psychiatric Association. Drs Hooker and Finkelman and Ms Moore report no potential conflicts of interest relevant to the subject of this letter.
Funding/support: This work was supported by National Institute of Mental Health grant R01-MH66302.
J Clin Psychiatry 2014;75(1):85-86 (doi:10.4088/JCP.13lr08758a).
© Copyright 2014 Physicians Postgraduate Press, Inc.
