Generalizability of Clinical Trial Results for Bipolar Disorder to Community Samples: Findings From the National Epidemiologic Survey on Alcohol and Related Conditions

Generalizability of Clinical Trial Results for Bipolar Disorder to Community Samples: Findings From the National Epidemiologic Survey on Alcohol and Related Conditions

Nicolas Hoertel, MD; Yann Le Strat, MD; Pierre Lavaud, MD; Caroline Dubertret, MD, PhD; and Frédéric Limosin, MD, PhD

The practice of evidence-based medicine depends on the availability of double-blind, randomized, placebo-controlled trials (RCTs), which are widely viewed as the “gold standard” of clinical research.1,2 RCTs apply strict eligibility criteria that exclude patients with comorbid medical, psychiatric, and addictive disorders to evaluate the efficacy of the treatment on the disorder under study and to comply with guidelines by regulatory agencies. However, the high internal validity of the RCTs is achieved at the cost of diminished external validity.3 In fact, restrictive eligibility criteria may affect the external validity of clinical studies,4 resulting in a selection bias and perpetuating the gap between research and clinical practice.5 Over the last several years, concerns have emerged regarding the relevance of RCTs to typical patients in community settings who may often have complex clinical presentations.6 As a consequence, there has been a call to quantify the generalizability of RCT results to the broader target population suffering from the disorder under study.7-9

The impact of eligibility criteria on the generalizability of clinical trials has been previously examined in efficacy trials for major depressive disorder3,10-12 and psychosis13-15 and clinical trials for alcohol dependence,16-18 cannabis dependence,19 nicotine dependence,20 panic disorder,21 and generalized anxiety disorder.22,23 Previous studies conclude that the percentage of participants excluded by these criteria ranges from 50.5% to 80.5%,3,16 suggesting that results of RCTs cannot be directly extrapolated to patients in real-life settings.

Research on the generalizability of clinical trials for bipolar disorder is limited.9 Zarin et al24 compared data from published reports of 2 key RCTs underlying recent pharmacologic treatment for bipolar disorder with data on routine psychiatric practice collected through a Practice Research Network. They found that 55% of Practice Research Network patients with bipolar I disorder would have been ineligible for the corresponding RCT. This study underscores the relevance of quantifying the generalizability of RCTs in bipolar disorder and provides a framework for assessment. However, a limitation of this study is that it relies on a sample of participants with bipolar I disorder who seek treatment and therefore cannot be extrapolated to all individuals with bipolar disorder in the community. Examining the application of eligibility criteria to a large, nationally representative sample of individuals with bipolar disorder is required6-8,22 and may help quantify the impact of eligibility criteria on the generalizability of clinical trial results as well as guide eligibility criteria operationalization for future clinical trials in bipolar disorder.9

The present study assessed the effect of applying exclusion criteria commonly used in clinical trials for bipolar disorder to a large (N = 43,093), nationally representative general population sample of the United States, the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). To estimate the generalizability of the results of clinical trials for bipolar disorder, we used a method previously described by Blanco et al in clinical trials for major depression3 and alcohol dependence.16 We examined the proportion of all cases of bipolar disorder in the NESARC that would be eligible if the traditional eligibility criteria were applied to this sample. Because most RCTs examine efficacy of treatments for bipolar depression and acute mania separately, we applied standard exclusion criteria to individuals (1) with a current diagnosis of bipolar depression and (2) with acute mania separately. Because individuals who seek treatment for a disorder may differ from those who do not, we applied the exclusion criteria first to all participants with a current diagnosis of bipolar depression and then to a subsample of participants who sought treatment.

METHOD

Sample

Data were drawn from the 2001-2002 NESARC, a nationally representative face-to-face survey of 43,093 civilian noninstitutionalized US residents aged 18 years and older conducted by the US Census Bureau under the direction of the National Institute on Alcoholism and Alcohol Abuse (NIAAA), as described in detail elsewhere.25,26 The overall survey response rate was 81%. The research protocol, including informed consent procedures, received full ethical review and approval from the US Census Bureau and the Office of Management and Budget.27

Diagnoses were made according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) using the NIAAA Alcohol Use Disorder and Associated Disabilities Interview Schedule-IV (AUDADIS-IV), a fully structured diagnostic interview designed for experienced interviewers who are not clinicians.27 Respondents with mania occurring during the year preceding the interview were classified as having a current mania. Those with a major depressive episode occurring during the year preceding the interview and reporting a lifetime history of mania or hypomania were considered to have a current bipolar depression. Participants with current bipolar depression who “went anywhere or saw anyone to get help for low mood” during the year preceding the interview were considered to be seeking treatment. Information to approximate treatment-seeking behavior in participants with current mania was not available in the NESARC.

• Eligibility criteria applied in clinical trials for bipolar disorder exclude a majority of participants, particularly those seeking treatment, thus limiting the generalizability of their results.
• Individuals with a current significant risk of suicide are underrepresented in clinical trials for bipolar disorder.
• Clinicians and researchers should carefully consider eligibility criteria and their impact on the representativeness of clinical trials.

The reliability of the AUDADIS-IV, including clinical reappraisal studies conducted by psychiatrists, is well documented.27-29 The test-retest reliability of AUDADIS-IV in assessing DSM-IV mood, anxiety, and personality disorders was fair to good (κ = 0.40-0.62)30,31 and good (κ = 0.54-0.76) for substance use disorders.25,31

Participants with substance-induced mood disorders and mood symptoms due to a general medical condition may be difficult to detect routinely in eligibility assessments of clinical trials for bipolar disorder. As a control, we did a sensitivity analysis to examine the impact on the results if cases due to these 2 conditions were not ruled out. In addition, we examined whether distinguishing bipolar I and bipolar II depression or participants’ age might lead to different exclusion rates.

Measures

Exclusion criteria commonly used in clinical trials for bipolar depression and acute mania were applied to a sample representative of the general population to examine the proportion of individuals with a current DSM-IV diagnosis of bipolar depression or mania that would be eligible for a typical clinical trial. To investigate potential differences in eligibility between treatment-seeking and non-treatment-seeking individuals, we applied the same criteria to the subsample of individuals with current bipolar depression who sought treatment.

We investigated the exclusion criteria from clinical trials included in a recent literature review of articles on bipolar depression32 and a recent meta-analysis of articles on acute mania,33 both examining efficacy of pharmacologic treatments for bipolar disorder. Of the 32 trials included in the literature review on bipolar depression (comprising trials included in the meta-analyses cited in the references), all were included in the present analysis. Of the 56 published trials included in the meta-analysis on acute mania, 55 studies were included in the present analysis, because data from 1 trial were not available.

Two coders independently collected all exclusion criteria from the 32 bipolar depression studies and the 55 acute mania clinical trials and kept criteria that were present in more than 10% of the studies. Intercoder reliability was adequate (intraclass correlation coefficient = 0.84; 95% confidence interval [CI], 0.54-0.94). In cases of disagreement, the 2 coders discussed exclusion criteria and reached agreement. The median number of exclusion criteria used in both the 32 bipolar depression and the 55 acute mania studies was 7 (including criteria present in less than 10% of the studies). The 7 most commonly used exclusion criteria were similar in clinical trials for bipolar disorder and acute mania (Tables 1 and 2). Therefore, in order to reproduce a clinical trial with typical exclusion criteria, we applied the 7 most commonly used criteria to individuals with a DSM-IV diagnosis of bipolar depression within the last 12 months and then to those with a DSM-IV diagnosis of acute mania within the last 12 months in the NESARC sample.

The percentages of individuals excluded were estimated from data collected by the AUDADIS-IV. The criterion “significant risk of suicide” was considered present if the person reported a suicide attempt in the past year, the time frame used by the AUDADIS-IV when assessing the presence of “current” symptoms. The criterion “significant medical condition” was approximated by a series of questions on 12-month chest pain, angina pectoris, myocardial infarction or any other form of heart disease, and cirrhosis or hepatic disease and whether the diagnosis was confirmed by a physician. Information to approximate the criterion “currently taking any psychotropic medication” was not available in the NESARC.

Analysis Methods

We first determined the percentage (and 95% CI) of survey participants with a current DSM-IV diagnosis of bipolar depression or mania who would have been excluded by individually applying each exclusion criterion in clinical trials. Because individuals might have been excluded by more than 1 criterion, we also calculated the overall percentage of subjects who would have been excluded by the simultaneous application of all criteria. We conducted these analyses for all participants with a current DSM-IV diagnosis of bipolar depression (n = 785), for the subsample of individuals who sought treatment (n = 276), and for participants with a current diagnosis of mania (n = 724). Because of the weighting and clustering used in the NESARC design, all statistical analyses were performed using the Taylor series linearization method, a design-based method implemented using SUDAAN, version 10 (RTI International, Research Triangle Park, North Carolina). Intraclass coefficient correlation measuring intercoder reliability was performed using R software, version 2.12.2 (The R Foundation for Statistical Computing, Vienna, Austria).34

RESULTS

The percentage of participants excluded by at least 1 of the 6 most common and available criteria in efficacy trials was 58.17% (95% CI, 53.66%-62.54%) in the full sample of 785 individuals who met DSM-IV criteria for bipolar depression and 63.87% (95% CI, 56.23%-70.87%) in the subsample of 276 individuals who sought treatment (Table 3). This percentage was substantially lower in efficacy trials for current mania, falling to 55.75% (95% CI, 51.43%-59.99%) in the sample of 724 participants with mania (Table 4). The criterion leading to the highest percentage of exclusion was “significant suicidal risk” in trials for bipolar depression and DSM-IV “alcohol abuse or dependence” in trials for acute mania.

Individuals with bipolar I depression were significantly more often excluded by at least 1 exclusion criterion than those with bipolar II depression (OR = 1.79; 95% CI, 1.21-2.64; Wald F = 8.95, P < .01) (data available on request).

Because current prevalence of most mental disorders decreases with age,35 we examined whether the participants’ age might impact the overall exclusion rate in bipolar disorder clinical trials. We found no significant association between age and overall exclusion rate in bipolar depression (Wald F = 0.49, P = .487) and acute mania (Wald F = 3.55, P = .064) clinical trials.

DISCUSSION

The present study examines the proportion of adults with bipolar disorder in the community who would not have been eligible for a typical clinical trial for current bipolar depression or acute mania. Findings indicate that, in a typical efficacy trial for bipolar depression, more than 5 of 10 respondents with bipolar depression in the overall sample and more than 6 of 10 in the subsample of individuals who sought treatment would have been excluded by at least 1 criterion. In addition, more than 5 of 10 participants would have been excluded in a typical efficacy trial for acute mania. Consistent with a previous study,24 we found that traditional criteria for clinical trials for bipolar disorder tend to exclude a majority of individuals from participation, particularly those seeking treatment.

The overall exclusion rate was not found to be influenced by participants’ age, while participants with bipolar I depression were significantly more likely to be preferentially excluded in a typical trial for bipolar depression than those with bipolar II disorder. Comparably with patients with nicotine20 and alcohol dependence,16 individuals with bipolar disorder, particularly those with bipolar I disorder, often presented with comorbid medical and addictive disorders.32 These results support that clinical trial results are not generalizable to community settings and have implications for the design of clinical trials. As a consequence, clinical trials for bipolar disorder using typical eligibility criteria tend to recruit “pure” rather than “typical” patients.36

Findings indicate that having a current significant risk of suicide explains a substantial part of ineligibility in all samples. Having a current diagnosis of alcohol abuse or dependence and a significant medical condition also excluded a substantial proportion of individuals in all samples. Application of the eligibility criteria to the treatment-seeking subsample excluded substantially more subjects with bipolar depression than from the full sample. As previously suggested,37,38 these results support that individuals seeking treatment do present with greater illness severity and more psychiatric and medical comorbidities. As a consequence, these patients may perceive a greater need for treatment, which may favor treatment-seeking behaviors. Paradoxically, clinical trials tend to preferentially exclude individuals who have the greatest overall disease severity and therefore the greatest need for treatment.16,23

As an internal control of our approach, we did a sensitivity analysis. We examined the impact on the results if substance-induced depression or mania or mood symptoms due to a general medical condition were not ruled out. Excluding the cases of illness-induced and substance-induced depression and mania only slightly decreased by 0.77% in the overall bipolar depression sample, by 1.02% in the treatment-seeking subsample, and by 0.48% in the mania sample the percentage of participants excluded because of at least 1 criterion (data available on request). This result suggests that illness-induced and substance-induced criteria used to define the DSM-IV diagnosis of bipolar depression and mania have little impact on the overall exclusion rate.

While some exclusion criteria have been implemented for safety reasons (eg, pregnancy, significant medical condition), some others exclude a significant proportion of the population likely to seek care in clinical setting (eg, significant risk of suicide, current diagnosis of alcohol or drug abuse or dependence). The exclusion of participants currently presenting with a significant risk of suicide may be particularly important, since suicidal risk is highly prevalent in patients with bipolar disorder.39 Thus, it is questionable whether the results of clinical trials apply in a community setting.9 While the use of more restrictive eligibility criteria may be appropriate in early efficacy trials, the designers of clinical trials should carefully consider the trade-offs between the application of each exclusion criterion and its impact on generalizability.16

This study has several limitations. First, we adopted specific conventions to operationalize eligibility criteria and to translate those to the NESARC sample. We followed a methodology described by Blanco et al3,16 and considered eligibility criteria from 32 clinical trials included in a recent literature review on bipolar depression32 and from 55 trials in a recent meta-analysis on acute mania.33 Other conventions might have led to different exclusion estimates. For example, we excluded all individuals with suicide attempts within the last 12 months, considering this question as providing the closest available data to approximate the somewhat vague criterion “significant risk of suicide,” which is mostly used in trials. In addition, the time frame considered for the exclusion criterion “any alcohol or drug abuse/dependence” ranged from 2 weeks to 6 months before inclusion in the clinical trials we included. We thus approximated this criterion as having “current/past 6 months’ any alcohol or drug abuse/dependence.” The time frame used could have led to an overestimation of the exclusion rate. Furthermore, we considered the criterion “significant medical condition” met if the trial excluded individuals with a particular general medical condition (eg, hepatic disease, angina pectoris) and used a series of questions on 12-month chest pain, angina pectoris, myocardial infarction, or any other form of heart disease, and cirrhosis or hepatic disease to approximate it in the NESARC sample. However, the percentage of excluded participants was high in both bipolar depression and mania samples as well as in the subsample of treatment-seekers and was consistent with those observed in a previous study,24 suggesting that commonly applied criteria are likely to exclude a majority of individuals with bipolar disorder. Development of procedures to operationalize eligibility criteria selection might help refine future generalizability estimates.

Second, one exclusion criterion (ie, currently taking any psychotropic medication) was not included in the NESARC. The percentage of adults taking a psychotropic medication is likely to be high40 and might theoretically lead to an underestimation of the proportion of patients excluded in clinical trials.

Third, although intercoder reliability was adequate (intraclass correlation coefficient = 0.84; 95% CI, 0.54-0.94), some discrepancies in interpreting certain eligibility criteria existed between the 2 coders. This finding supports the need of operationalization in criteria selection, since some all-encompassing exclusion criteria (eg, any Axis I or Axis II disorder that would interfere with compliance) might lead to different exclusion rates according to an investigator’s interpretation.

Lastly, the 12-month prevalence rate of bipolar disorder (bipolar I and II disorders) was 2.8% (SE = 0.1) in the NESARC,41,42 substantially higher than previous prevalence estimates but within the same range as the one reported in the National Comorbidity Survey Replication (NCS-R) (2.6%, SE = 0.2).43 Since factors associated with exclusion from efficacy trials (eg, medical conditions and substance use disorders) and bipolar disorder have in common several symptoms (eg, asthenia, insomnia, motor agitation), it is possible that the lay interviewers who performed evaluations in NESARC may have overestimated the prevalence of bipolar disorder itself because of symptom overlaps, resulting in false positive diagnoses. If present, this bias presumably led to an artificial increase in the estimated associations of bipolar depression and mania with medical conditions and substance use disorders.

Despite these limitations, this study suggests that the current criteria for eligibility applied in clinical trials for bipolar disorder are restrictive and exclude a majority of participants, thus limiting the generalizability of their results. Particularly, individuals with a current significant risk of suicide are underrepresented in clinical trials. Future studies should benefit from examining treatment efficacy for bipolar disorder in this group. Although several criteria are widely implemented, not all trials use all criteria. Future trials should report how their exclusion criteria were operationalized and how they would likely influence patient eligibility, which will help refine estimates of the proportion of individuals ineligible for clinical trials in bipolar disorder. As required by Consolidated Standards of Reporting Trials (CONSORT) guidelines, both clinical trials and meta-analyses should report exclusion rate estimates and reasons for eligibility.

Author affiliations: Assistance Publique-H×´pitaux de Paris, Department of Psychiatry, Corentin-Celton Hospital, France; Paris Descartes University, PRES Sorbonne Paris Cité, Paris (Drs Hoertel and Limosin); INSERM UMR 894, Psychiatry and Neurosciences Center; Paris Descartes University, PRES Sorbonne Paris Cité, Paris (Drs Hoertel, Le Strat, Dubertret, and Limosin); Assistance Publique-H×´pitaux de Paris, Department of Psychiatry and Addiction Medicine, Louis Mourier Hospital, Colombes, France; Paris Diderot University, PRES Sorbonne Paris Cité (Drs Le Strat and Dubertret); and Assistance Publique-H×´pitaux de Paris, Department of Psychiatry, Kremlin Bicêtre Hospital, Paris, France (Dr Lavaud).

Potential conflicts of interest: Dr Limosin is a member of the speakers/advisory boards for Bristol-Myers Squibb, Euthérapie, Janssen, and Lundbeck. Drs Hoertel, Le Strat, Lavaud, and Dubertret report no conflicts of interest.

Funding/support: None reported.

Additional information: Original data set for the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) is available from the National Institute on Alcohol Abuse and Alcoholism (http://www.niaaa.nih.gov).

REFERENCES

1. Bucher HC, Guyatt GH, Cook DJ, et al; Evidence-Based Medicine Working Group. Users’ guides to the medical literature, 29: applying clinical trial results. a: how to use an article measuring the effect of an intervention on surrogate end points. JAMA. 1999;282(8):771-778. PubMed doi:10.1001/jama.282.8.771

2. Oxman AD, Sackett DL, Guyatt GH. The Evidence-Based Medicine Working Group. Users’ guides to the medical literature, 1: how to get started. JAMA. 1993;270(17):2093-2095. PubMed doi:10.1001/jama.1993.03510170083036

3. Blanco C, Olfson M, Goodwin RD, et al. Generalizability of clinical trial results for major depression to community samples: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2008;69(8):1276-1280. PubMed doi:10.4088/JCP.v69n0810

4. Weisberg HI, Hayden VC, Pontes VP. Selection criteria and generalizability within the counterfactual framework: explaining the paradox of antidepressant-induced suicidality? Clin Trials. 2009;6(2):109-118. PubMed doi:10.1177/1740774509102563

5. Dzewaltowski DA, Estabrooks PA, Klesges LM, et al. Behavior change intervention research in community settings: how generalizable are the results? Health Promot Int. 2004;19(2):235-245. PubMed doi:10.1093/heapro/dah211

6. March JS, Silva SG, Compton S, et al. The case for practical clinical trials in psychiatry. Am J Psychiatry. 2005;162(5):836-846. PubMed doi:10.1176/appi.ajp.162.5.836

7. Geddes J, Carney S. Recent advances in evidence-based psychiatry. Can J Psychiatry. 2001;46(5):403-406. PubMed

8. Wells KB. Treatment research at the crossroads: the scientific interface of clinical trials and effectiveness research. Am J Psychiatry. 1999;156(1):5-10. PubMed

9. Licht RW. Limits of the applicability and generalizability of drug trials in mania. Bipolar Disord. 2002;4(suppl 1):66-68. PubMed doi:10.1034/j.1399-5618.4.s1.27.x

10. Gaynes BN, Warden D, Trivedi MH, et al. What did STAR*D teach us? results from a large-scale, practical, clinical trial for patients with depression. Psychiatr Serv. 2009;60(11):1439-1445. PubMed doi:10.1176/appi.ps.60.11.1439

11. Zimmerman M, Chelminski I, Posternak MA. Generalizability of antidepressant efficacy trials: differences between depressed psychiatric outpatients who would or would not qualify for an efficacy trial. Am J Psychiatry. 2005;162(7):1370-1372. PubMed doi:10.1176/appi.ajp.162.7.1370

12. Zimmerman M, Mattia JI, Posternak MA. Are subjects in pharmacological treatment trials of depression representative of patients in routine clinical practice? Am J Psychiatry. 2002;159(3):469-473. PubMed doi:10.1176/appi.ajp.159.3.469

13. Boter H, Derks EM, Fleischhacker WW, et al. Generalizability of the results of efficacy trials in first-episode schizophrenia: comparisons between subgroups of participants of the European First Episode Schizophrenia Trial (EUFEST). J Clin Psychiatry. 2010;71(1):58-65. PubMed

14. Leucht S, Heres S, Hamann J, et al. Methodological issues in current antipsychotic drug trials. Schizophr Bull. 2008;34(2):275-285. PubMed doi:10.1093/schbul/sbm159

15. Khan AY, Preskorn SH, Baker B. Effect of study criteria on recruitment and generalizability of the results. J Clin Psychopharmacol. 2005;25(3):271-275. PubMed doi:10.1097/01.jcp.0000161497.73514.80

16. Blanco C, Olfson M, Okuda M, et al. Generalizability of clinical trials for alcohol dependence to community samples. Drug Alcohol Depend. 2008;98(1-2):123-128. PubMed doi:10.1016/j.drugalcdep.2008.05.002

17. Humphreys K, Weingardt KR, Harris AH. Influence of subject eligibility criteria on compliance with National Institutes of Health guidelines for inclusion of women, minorities, and children in treatment research. Alcohol Clin Exp Res. 2007;31(6):988-995. PubMed doi:10.1111/j.1530-0277.2007.00391.x

18. Humphreys K, Weingardt KR, Horst D, et al. Prevalence and predictors of research participant eligibility criteria in alcohol treatment outcome studies, 1970-98. Addiction. 2005;100(9):1249-1257. PubMed doi:10.1111/j.1360-0443.2005.01175.x

19. Okuda M, Hasin DS, Olfson M, et al. Generalizability of clinical trials for cannabis dependence to community samples. Drug Alcohol Depend. 2010;111(1-2):177-181. PubMed doi:10.1016/j.drugalcdep.2010.04.009

20. Le Strat Y, Rehm J, Le Foll B. How generalisable to community samples are clinical trial results for treatment of nicotine dependence: a comparison of common eligibility criteria with respondents of a large representative general population survey. Tob Control. 2011;20(5):338-343. PubMed doi:10.1136/tc.2010.038703

21. Hoertel N, Le Strat Y, De Maricourt P, et al. Are subjects in treatment trials of panic disorder representative of patients in routine clinical practice? results from a national sample [published online ahead of print October 17, 2012]. J Affect Disord. doi:10.1016/j jad.2012.bMed doi:10.1016/j.jad.2012.09.023.

22. Westen D, Morrison K. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: an empirical examination of the status of empirically supported therapies. J Consult Clin Psychol. 2001;69(6):875-899. PubMed doi:10.1037/0022-006X.69.6.875

23. Hoertel N, Le Strat Y, Blanco C, et al. Generalizability of clinical trial results for generalized anxiety disorder to community samples. Depress Anxiety. 2012;29(7):614-620. PubMed doi:10.1002/da.21937

24. Zarin DA, Young JL, West JC. Challenges to evidence-based medicine: a comparison of patients and treatments in randomized controlled trials with patients and treatments in a practice research network. Soc Psychiatry Psychiatr Epidemiol. 2005;40(1):27-35. PubMed doi:10.1007/s00127-005-0838-9

25. Grant BF, Dawson DA, Stinson FS, et al. The Alcohol Use Disorder and Associated Disabilities Interview Schedule-IV (AUDADIS-IV): reliability of alcohol consumption, tobacco use, family history of depression and psychiatric diagnostic modules in a general population sample. Drug Alcohol Depend. 2003;71(1):7-16. PubMed doi:10.1016/S0376-8716(03)00070-X

26. Grant BF, Dawson DA, Stinson FS, et al. The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug Alcohol Depend. 2004;74(3):223-234. PubMed doi:10.1016/j.drugalcdep.2004.02.004

27. Grant BF, Harford TC, Dawson DA, et al. The Alcohol Use Disorder and Associated Disabilities Interview schedule (AUDADIS): reliability of alcohol and drug modules in a general population sample. Drug Alcohol Depend. 1995;39(1):37-44. PubMed doi:10.1016/0376-8716(95)01134-K

28. Hasin D, Carpenter KM, McCloud S, et al. The alcohol use disorder and associated disabilities interview schedule (AUDADIS): reliability of alcohol and drug modules in a clinical sample. Drug Alcohol Depend. 1997;44(2-3):133-141. PubMed doi:10.1016/S0376-8716(97)01332-X

29. Canino G, Bravo M, Ram×­rez R, et al. The Spanish Alcohol Use Disorder and Associated Disabilities Interview Schedule (AUDADIS): reliability and concordance with clinical diagnoses in a Hispanic population. J Stud Alcohol. 1999;60(6):790-799. PubMed

30. Grant BF, Stinson FS, Dawson DA, et al. Prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry. 2004;61(8):807-816. PubMed doi:10.1001/archpsyc.61.8.807

31. Hasin DS, Stinson FS, Ogburn E, et al. Prevalence, correlates, disability, and comorbidity of DSM-IV alcohol abuse and dependence in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry. 2007;64(7):830-842. PubMed doi:10.1001/archpsyc.64.7.830

32. Frye MA. Clinical practice: bipolar disorder—a focus on depression. N Engl J Med. 2011;364(1):51-59. PubMed doi:10.1056/NEJMcp1000402

33. Cipriani A, Barbui C, Salanti G, et al. Comparative efficacy and acceptability of antimanic drugs in acute mania: a multiple-treatments meta-analysis. Lancet. 2011;378(9799):1306-1315. PubMed doi:10.1016/S0140-6736(11)60873-8

34. Team. RDC. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. http://www.R-project.org. 2008. Accessed October 5, 2012.

35. Blazer DG 2nd, Hybels CF. Origins of depression in later life. Psychol Med. 2005;35(9):1241-1252. PubMed doi:10.1017/S0033291705004411

36. Goldenberg IM, White K, Yonkers K, et al. The infrequency of “pure culture” diagnoses among the anxiety disorders. J Clin Psychiatry. 1996;57(11):528-533. PubMed doi:10.4088/JCP.v57n1105

37. Cohen P, Cohen J. The clinician’s illusion. Arch Gen Psychiatry. 1984;41(12):1178-1182. PubMed doi:10.1001/archpsyc.1984.01790230064010

38. Kirchner JE, Booth BM, Owen RR, et al. Predictors of patient entry into alcohol treatment after initial diagnosis. J Behav Health Serv Res. 2000;27(3):339-346. PubMed doi:10.1007/BF02291745

39. Oquendo MA, Currier D, Liu SM, et al. Increased risk for suicidal behavior in comorbid bipolar disorder and alcohol use disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). J Clin Psychiatry. 2010;71(7):902-909. PubMed doi:10.4088/JCP.09m05198gry

40. Verdoux H, Bégaud B. Pharmaco-epidemiology: what do (and don’ t) we know about utilisation and impact of psychotropic medications in real-life conditions? Br J Psychiatry. 2004;185(2):93-94. PubMed doi:10.1192/bjp.185.2.93

41. Hasin DS, Goodwin RD, Stinson FS, et al. Epidemiology of major depressive disorder: results from the National Epidemiologic Survey on Alcoholism and Related Conditions. Arch Gen Psychiatry. 2005;62(10):1097-1106. PubMed doi:10.1001/archpsyc.62.10.1097

42. Hoertel N, Le Strat Y, Angst J, et al. Subthreshold bipolar disorder in a US national representative sample: prevalence, correlates and perspectives for psychiatric nosography. [published online ahead of print October 3, 2012.] J Affect Disord. doi:10.1016/j jad.2012. doi:1016/j.jad.2012.09.016 PubMed.

43. Kessler RC, Chiu WT, Demler O, et al. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617-627. PubMed doi:10.1001/archpsyc.62.6.617