Antidepressant Discontinuation and Risk of Suicide Attempt: A Retrospective, Nested Case-Control Study

Antidepressant Discontinuation and Risk of Suicide Attempt: A Retrospective, Nested Case-Control Study

Robert J. Valuck, PhD, RPh; Heather D. Orton, PhD; and Anne M. Libby, PhD

Objective: Prior efforts to assess the impact of antidepressant use on risk of suicide attempt focused on antidepressant initiation or duration of use. Gaps remain in understanding risks associated with antidepressant discontinuation in the context of the drug regimen. We assessed the effects of antidepressant discontinuation on the risk of suicide attempt.

Method: We report a nested case-control study of suicide attempt with at least 12 months of prior observation. A retrospective cohort of 2.4 million patients with depression (ICD-9 codes 296.2, 296.3, 300.4, and 311), aged 5-89 years, was created using standard Healthcare Effectiveness Data and Information Set (HEDIS) criteria; from this cohort, cases (n = 10,456) and controls (n = 41,815) were selected for study. Data were from a large, national, longitudinal, integrated claims database of managed care enrollees in the United States from calendar years 1999 through 2006.

Results: Compared to controls, cases were more likely to have used antidepressants, to have had multiple antidepressants, and to have had prior depressive episodes and inpatient stays that involved depression. After adjusting for confounding due to depression severity, comorbidities, and other medications, antidepressant use showed a protective effect for suicide attempt (OR = 0.62, P < .001). Compared to prior therapy, antidepressant discontinuation had a significant risk for suicide attempt (OR = 1.61, P < .05). Antidepressant initiation had the highest risk for suicide attempt (OR = 3.42, P < .05), followed by titration (titration up, OR = 2.62; down, OR = 2.19; P < .05).

Conclusions: Substantial confounding exists in examining the link between antidepressant use and suicide attempt, specifically regarding those factors associated with characteristics of depression. Antidepressant discontinuation showed a significant risk for suicide attempt, as did the period of an abbreviated trial, that is, stopping before a therapeutic regimen of 56 days had been reached. The highest risk was associated with initiation, a finding consistent with other studies, closely followed by periods of dosing changes and discontinuation. Patients should be closely monitored during these periods.

J Clin Psychiatry 2009;70(8):1069-1077

© Copyright 2009 Physicians Postgraduate Press, Inc.

Submitted: December 12, 2008; accepted April 21, 2009 (doi:10.4088/JCP.08m04943).

Corresponding author: Robert J. Valuck, PhD, RPh, School of Pharmacy, Anschutz Medical Campus at Fitzsimons, University of Colorado Denver, PO Box 6508, Aurora, CO 80045 ([email protected]).

Suicidality is a term that encompasses a large range of behaviors and thoughts, including completed suicide, suicide attempt, planning or other preparatory action, and suicidal ideation.1 Concern over a possible link between suicidality and antidepressant use has led the US Food and Drug Administration (FDA) to take regulatory action in the form of a black box warning for this class of drugs for children and adolescents (under age 19 years) in 20042 and for young adults (aged 19-24 years) in 2005.3 In their meta-analyses of antidepressant clinical trials, however, the FDA noted that there were no completed suicides and only infrequent suicide attempts reported in the trials.4,5 Only when the endpoints of suicidal ideation and "possible" suicidal ideation were included did the meta-analyses yield significant effects.6 Further, the FDA meta-analyses did not include information on antidepressant drug exposure (dose, duration) or timing of reported suicidal events in relation to initiation or phases of antidepressant treatment.

Experimental research cannot answer key questions on the relationship between antidepressants and suicidal behaviors because of ethical issues involved in studying this severe outcome with randomized, placebo-controlled trials. Because of the rarity of reported suicidal behaviors in clinical trials and the difficulty in accurately classifying such behaviors from case report forms,1 little is known about the demographic or clinical characteristics of subjects who attempt suicide or how often such attempts occur in large "real world" populations. Thus, observational study designs must be employed to answer specific questions pertaining to this possible drug-effect relationship.

A gap exists in the literature in characterizing antidepressant regimens as something more than "ever or never" use or simple duration of use. Burning questions remain about suicide attempt risk during antidepressant discontinuation and other phases of the antidepressant regimen such as dose titration. One cohort study showed an increase in suicide attempt risk in the month immediately prior to initiation of antidepressant therapy and in the first 4 to 8 weeks of therapy.7 Another cohort study showed that suicide attempt risk decreased as duration of treatment increased.8 Clinicians and researchers have postulated that during initiation of antidepressant therapy and during any "abrupt" changes in therapy (including upward or downward dose titration, or discontinuation without a prescribed period of tapering), the risk of suicide-related behaviors may increase. Rates of prescribed titration have not been well studied, and suicidal risk during discontinuation is essentially unknown. This research study addresses these gaps.

This study contributes to the debate about antidepressant use and suicide attempt by using a national, community-based observational sample of patients with depression to compare cases with a suicide attempt to matched controls who did not attempt suicide. We report a nested case-control study of a large, nationally representative, managed-care dataset of subjects with a prior episode of depression who have attempted suicide (n ≈ 10,000), and a set of matched control subjects (n ≈ 40,000). We present crude and adjusted estimated risk ratios for suicide attempts in this cohort of patients diagnosed with depression. The primary strengths of the study are that it (1) represents the largest known population of suicide attempt cases available, (2) is the only study to measure the relationship between suicide attempt risk and antidepressant discontinuation, and (3) introduces a typology of antidepressant regimen phases associated with differential suicide attempt risk.

METHOD

Data came from a commercially available database provided by PHARMetrics, Inc (Watertown, Massachusetts), a unit of Intercontinental Marketing Services (IMS), the largest patient-centric database of longitudinal, integrated medical, facility, and pharmacy claims data. These integrated data include paid claims from 85 managed care plans nationally, representing 47 million patients with insurance coverage in the United States from 1999 through 2006. A retrospective, nested case-control study was conducted. In this design, the observation periods looked back in time prior to the suicide attempt or, for matched controls, were anchored to the case’s suicide attempt date. An expedited review was obtained due to unidentified and anonymous records, and the study was approved by the Colorado Multiple Institutional Review Board.

Base Population

Claims data were used to create a cohort of patients with new episodes of depression treatment (2.4 million patients). A new episode was defined using the specifications of the National Committee for Quality Assurance’s Healthcare Effectiveness Data and Information Set (HEDIS): an International Classification of Diseases, Ninth Revision (ICD-9) code of 296.2, 296.3, 300.4, or 311; a period of 120 days before diagnosis during which no other depression-related diagnoses appeared in the claims history; and a period of 90 days before diagnosis during which no other antidepressant medication claims appeared in the history.9,10

Case Definition: Suicide Attempt Cases

Suicide attempt cases were identified from insurance claims coded for a suicide attempt associated with any visit to a provider or facility. We followed the Centers for Disease Control and Prevention guidelines in defining suicide attempts as claims with ICD-9 codes E950-E959 and ICD-10 codes X60-X84 and Y87.0.11 Our previous work using the PHARMetrics database has focused on the outcome of suicide attempt, as noted previously (ICD-9 codes E950-E959).8,12 This outcome, we believe, is the most suitable measure of suicidal risk in observational studies for 2 reasons: (1) completed suicides are very rare and represent only a small fraction of the attempts that are made and (2) suicidal ideation/thinking is very subjective and rarely coded on medical claims. Suicide attempt is a clear indicator of suicidal behavior, and as an outcome is more common and yields sufficient power to study issues related to antidepressant drug use patterns. For each suicide attempt case, the date of the attempt was labeled the index date (in the case of multiple attempts per subject, the date of the first attempt was labeled the index date, and subsequent descriptive analyses focused only on the initial attempt). Twelve months of observation prior to the event was the minimum requirement (mean prior observation time was 30 months, or 2.5 years), although there were subjects who did not meet this inclusion criterion. This restriction was employed to enable accurate description of demographic and clinical characteristics of identified case subjects in the year prior to, as well as at the time of, the suicide attempt (Figure 1). Prior observation periods were trimmed beyond 12 months to equate observation periods for cases and controls.

Selection of Controls

For each subject defined as a case, up to 4 control subjects were matched by age (± 1 year), gender (male or female), and region (East, Midwest, South, or West); if more than 4 were available, matches were randomly selected. These variables were previously shown to correlate with suicidal risk in the study cohort.8 Controls were assigned an event date (corresponding to the index date of suicide attempt for cases) that was the same number of days after the start of their depression episode as their matched counterpart. The number of controls/subjects available for matching was 2.4 million. Subjects and their matches were excluded if they did not have populated data in the fields required to operationalize drug exposure definitions (fill date, drug dispensed, dose dispensed, quantity dispensed, and days supplied).

Measures

Antidepressant exposure and phases of treatment. A simple measure of antidepressant exposure was the zero-one indicator of any use (ever-never) if there was at least 1 antidepressant prescription filled before the event date and during the observation period. For each subject who had any antidepressant use, mutually exclusive indicators of antidepressant treatment phases at the time of the event were assigned. A flowchart illustrates the antidepressant regimen typology and sample sizes (Figure 2). The following definitions were used based on the start date of antidepressant drug exposure, ie, first prescription fill date, and the stop date, equal to the last date of antidepressant drug exposure, ie, last fill date plus last days’ supply.

Current therapy. Prescription records indicate that the antidepressant interval contains the suicide attempt event date. Current short term is less than 90 days on antidepressant treatment when the event date occurs; current long term indicates more than 90 days of antidepressant treatment at the event date. The following phases fall into current therapy.

Initiation. Initiation refers to the first 55 days of antidepressant therapy following the start date.

Maintenance. Early maintenance was 56-179 days (late maintenance was 180 or more days) of prescribed antidepressant treatment after start date, but not occurring during a titration or discontinuation period as defined below.

Titration. Titration up or down is noted for any days during the days’ supply period of a given prescription for which the antidepressant daily dose is changed (increased or decreased) from the previous daily dose in a maintenance phase.

Past therapy. Past therapy applies if there is at least a 1-day gap between the stop date and the event date. Recent past indicated a gap of less than 90 days; distant past indicated a gap of more than 90 days between the stop date and the event date. Past therapy includes the following phases.

Abbreviated trial. The last fill date occurred within 56 days of antidepressant initiation.

Discontinuation. The first 14 days after the stop date, without a preceding prescribed titration down.

Prior therapy. Any days after the stop date of therapy for a subject with a titration down; any days after the last fill date at least 56 days after initiation; or any days occurring more than 14 days after the stop date for subjects with a discontinuation.

Covariates

Age category. Age groupings were based on risk populations as defined by the FDA: pediatric, 5-18 years; young adult, 19-24 years; adult, 25-64 years; and elderly, 65 years and older.

Service use before the index date as proxies for depression severity. Service use before the index date was used as a proxy for depression severity and was determined using the following criteria: the count of prior depressive episodes using HEDIS criteria defined previously; the number of inpatient stays of any length that had at least 1 discharge diagnosis of depression; any health care visit for psychotherapy using Current Procedural Terminology, Fourth Revision (CPT-4) procedure codes from provider or facility claims.

Medical and psychiatric comorbidities. The level of chronic medical comorbidity for cases and controls at the time of index date was measured using the Chronic Disease Indicator score, which indicates the total number of chronic diseases a subject has according to pharmacy claims data.13 Pregnancy was noted for any female patient in the observation period using ICD-9 codes and pregnancy-related procedure codes.14 The following psychiatric disorders were included as indicator variables: bipolar disorder, schizophrenia, anxiety spectrum, substance use disorder, and other mental health problem.

Nonantidepressant medication use. Indicators were used to adjust for the use of prescription medications for which there is some risk of suicidality on the package insert or for which a concern has been raised publicly by the FDA for the drug or drug class. These are isotretinoin, varenicline, oseltamivir, 11 antiepileptic drugs specified by the FDA,15 leukotriene receptor antagonist asthma medications, and mood stabilizers/lithium. Atypical antipsychotics and anxiolytics were included as possible antidepressant substitutes.

Antidepressant characteristics. Antidepressant drug groups were also used as indicators: selective serotonin reuptake inhibitors (SSRIs), serotonin-norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants, other antidepressants such as monoamine oxidase inhibitors, and multiple antidepressants of any type either concurrently or consecutively. Prescriber type for the antidepressant was indicated as primary care physician (internist, family medicine practitioner, obstetrician/gynecologist), psychiatrist, other mental health specialist (psychologist, social worker, psychiatric nurse), other non-mental health specialist, or other physician/unknown specialty type as defined in previous work.16

Statistical Analysis

Descriptive statistics were used to describe case and control subjects on clinical and demographic variables. As cases and controls were matched on age, gender, and region, these measures were statistically not different for cases and controls by design. Other clinical and demographic covariates were compared using t tests and χ2 tests of association, as appropriate for the underlying distribution of the data. A case-control analysis was performed with suicide attempt cases and their matched controls. The dependent variable was case/control (ie, suicide attempt), and the independent variables were the following: any (vs no) antidepressant drug exposure; among antidepressant users, current and past short- and long-term; and separately, the phase of antidepressant treatment (discontinuation, titration up or down, initiation, early or late maintenance, or prior therapy). The effects of drug exposure and phases were modeled in separate conditional logistic regressions in order to accommodate the violation of independence created by matching.

Subset analyses were performed excluding the female patients who were pregnant in the period prior to the index date. Roughly the same percent of female patients with pregnancy codes were cases (13.23%) and controls (13.07%); their exclusion did not change any estimates, so they were retained in the models. Age groupings of subjects corresponding to the most recent FDA review of antidepressants and suicidality were included to determine if any observed elevations in risk of suicide attempt followed the possible "age gradient" suggested by FDA scientists at the December 13, 2006 meeting of the Psychopharmacologic Drugs Advisory Committee.17 Statistical significance was set at α = .05 (2-tailed) for all comparisons. All analyses were performed using SAS version 9.1.18

RESULTS

The total case-control sample included 52,271 patients with depression episodes from 1999 to 2006. Controls were matched 4:1 to cases, yielding 41,815 controls matched to 10,456 cases of suicide attempt following a new, treated depression episode. Controls were matched to cases on the basis of gender (70% female), age in years (mean age, 31), and geographic region (χ2 tests of difference not statistically different [P values = .36 to .86]). Proxy measures for depression severity and other psychiatric and medical comorbidities were compared for cases and controls (Table 1). Cases were more likely to have had a prior depressive episode and an inpatient stay associated with depression, as well as diagnoses of bipolar disorder, schizophrenia, anxiety, and substance use disorder. Cases were also more likely to have been prescribed antiepileptic medications, atypical antipsychotics, or anxiolytics.

Cases were more likely than controls to have ever had an antidepressant during the observation period (71% vs 59%, P < .001) (Table 2). Among those 32,070 cases and controls who had ever had an antidepressant during the observation period, cases were more likely to have had multiple antidepressants either consecutively or concurrently (54% vs 33%, P < .001), and were more likely to have received that antidepressant prescription from a psychiatrist (30% vs 20%, P < .001). Cases who had received multiple antidepressants and their matched controls (n = 12,106) were excluded from the timing and phase analyses due to the inability to aggregate dosing and establish equivalency, or to attribute suicide attempt to a specific drug when more than 1 was used. Cases were more likely to have been on current short-term therapy at the time of the suicide attempt than their controls (29% vs 18%, P = .004). Cases were less likely to have been on current long-term therapy at the time of the suicide attempt than their controls (71% vs 82%, P = .004). Twice as many cases as controls were in the initiation phase of antidepressant therapy at the time of the suicide attempt (14% vs 7%, P < .001), as well as in the (prescribed) titration up and titration down phases (see Table 2). There was no statistical difference in the proportion of cases and controls in the early maintenance phase of 56-179 days supplied.

Logistic regression analyses were conducted using data from cases and controls, with the main effect of antidepressant use. When antidepressant exposure was characterized in its simplest form—ever or never—and no adjustments for confounding by indication were made, the risk of suicide attempt was 1.74 times higher (P < .001) for those receiving any antidepressant(s) compared to those receiving none. After adjusting for confounding factors, including depression severity, comorbidities, and other medication use, the risk of suicide attempt was 0.62 for those receiving any antidepressant(s) compared to those receiving none (P < .001). Thus, controlling for these confounding variables decreased the empirical relationship between antidepressant use and suicide attempt from 1.74 to 0.62, changing from positive and significant to negative and significant; ie, any antidepressant use had a significant protective effect on suicide attempt in this national sample.

Among those ever on an antidepressant, and excluding users of multiple antidepressants, an analysis of the phases of antidepressant use was conducted (Table 3). Patients previously but not currently on an antidepressant on the index date (prior therapy) composed the referent group in the logistic regression analysis. This choice of referent group is important because the referent group should be among the "ever" antidepressant users, that is, during the observation period; they were prescribed antidepressants but had stopped by at least 1 day prior to the index date. In this way, prior antidepressant users are an ideal comparison group, akin to a propensity adjustment, because those patients had observed characteristics that led to a prescribed antidepressant at one time; thus, they are the best nonexperimental comparison group.

Younger groups, both pediatric and young adult, had elevated risk compared to adults, adjusting for other factors. Patients on treatment with SNRIs, as well as people with prior episodes and psychiatric comorbidity, had higher risk of suicide attempt, adjusting for other factors.

During the discontinuation phase, risk for suicide attempt was increased by 61% compared to prior use (OR = 1.61, P < .05). Patients were twice as likely to attempt suicide if they were in a titration up phase (OR = 2.62, P < .05) or titration down phase (OR = 2.19, P < .05). Patients in the initiation phase of antidepressant treatment were 3 times as likely to attempt suicide (OR = 3.42, P < .05). Antidepressant duration was associated with lower risk for suicide attempt, with late maintenance phase a lower risk period than early maintenance (late OR = 1.36, early OR = 1.77, P < .05). Adjusted odds ratios are plotted for timing and phases of antidepressant treatment (Figure 3).

DISCUSSION

Our key finding is that discontinuation, that is, the period of 2 weeks after stopping antidepressants, is a period of elevated risk for suicide attempt. An abbreviated trial, that is, early stopping in the days before a patient has reached a full pharmacologic effect, is also a period of increased risk, but less so. Periods of prescribed changes in antidepressants—whether up or down—double the risk of suicide attempt compared to prior therapy. These periods require careful monitoring no matter when in the duration of treatment the changes occur, nor whether dose is titrated up or down. These figures likely underestimate risk because we observed only prescribed changes in dosing; providers commonly give the same dose, however, and instruct patients to take the pills every other day in order to titrate down, for example. Indicators of depression severity such as prior depression episodes and psychiatric comorbidities increased risk of suicide attempt, adjusting for other factors, which is an expected finding. Although younger ages showed higher adjusted relations to suicide risk compared to adults overall, the extent to which age interacts with the phasing of relative risks is important future work. Limitations related to incomplete measurement of key risk factors, such as suicide attempts prior to the observation period, are nevertheless present in such observational studies.

A simple indicator of antidepressant exposure (ever/never) has been the state of the field for too long. The work of Simon and colleagues7,19 was the first to extend the empirical work on timing of antidepressant treatment relative to suicide attempt or death, finding the highest risk in the month prior to starting treatment, followed by the month after treatment start, and subsequently declining over the next 5 months. This work uses a new typology of antidepressant exposure that goes further by characterizing the regimen by examining risk of suicide attempt in clinically meaningful periods of mutually exclusive treatment phases: initiation (1-55 days), early maintenance (56-179 days), late maintenance (180 days or more), discontinuation, titration up, titration down, and prior (but currently off) therapy. Consistent with Simon and colleagues,7,19 our findings reveal the entire 55-day initiation period as the highest risk for suicide attempt, and consistent with our earlier pediatric work, longer duration reduced risk of suicide attempt.8

The association between antidepressant use and the risk of suicidality is a classic example of confounding by indication. Antidepressants are the first-line treatment for acute-phase depression, and depression is the major risk factor for suicide attempt. It stands to reason, then, that the sickest and most at-risk patients would be given antidepressants. Accounting for this likelihood of antidepressant treatment would likely account for much of the measured variation in the relationship between antidepressant use and suicide attempt, since it would be confounded by unmeasured factors associated with underlying depression and illness. Indeed, this case-control study illustrated just that. This large national sample over several years permitted sufficient observation of suicide attempts among depressed patients and allowed accounting for unmeasured clinical factors. Subsequently, the crude odds ratio for the likelihood of suicide attempt among those ever on an antidepressant went from a large significant risk (1.74) to a substantial protective factor (0.62) after accounting for confounding. The limitation remains of unmeasured variables and the inability to tie effectiveness of treatment to safety outcomes. We must use persistence of therapy as a proxy for response, which should lead to remission. We are unable to directly test for effectiveness of antidepressants in this study.

This work takes a methodological "middle ground" between ecological studies that cannot link people to treatment at the individual level and experimental assignment that is not ethically possible. A number of ecological studies have been conducted to evaluate the possible link between antidepressant consumption and risk of suicide in large populations.20 These studies have generally found that rates of suicide have decreased as the consumption (at the population level) of antidepressants has increased over the past 20 years. While useful in general terms, these studies do not directly study individual subjects or their consumption of antidepressants and are thus useful primarily for drawing more general conclusions. Observational studies (eg, case-control and cohort studies) use patient-level data from real-world clinical treatment populations and allow for linkage of individual medication consumption patterns and outcomes (including suicide and suicide-related behaviors). These data allowed large-scale samples to permit the empirical study of rare events like suicide attempt, in fact, the largest sample of this outcome to date. These data also allowed for individual-level measurement to specify variables previously allocated as measurement error, obviously upwardly biasing the estimated effect of antidepressant use on suicide attempt risk. Better measurement using individual-level data permitted better estimates and specific criteria for increased suicide attempt risk during antidepressant treatment, showing exactly when patients need enhanced monitoring.

Drug names: isotretinoin (Accutane, Sotret, and others), lithium (Eskalith, Lithobid, and others), oseltamivir (Tamiflu), varenicline (Chantix).

Author affiliations: School of Pharmacy (Dr Valuck) and School of Medicine (Drs Libby and Orton), University of Colorado Denver.

Financial disclosure: Drs Valuck, Orton, and Libby report unrestricted investigator-initiated research grants from Eli Lilly, Forest, Lundbeck A/G, the Agency for Healthcare Research and Quality, and the American Foundation for Suicide Prevention.

Funding/support: This study was supported by the American Foundation for Suicide Prevention Distinguished Investigator Grant Program (Dr. Valuck). Open-access data licensing for the commercial PHARMetrics Patient-Centric Database was made possible through a prior grant from Eli Lilly. No sponsor had any input into any stage of the research (design, analysis, or interpretation of the findings) or preparation of the manuscript.

REFERENCES

1. Posner K, Oquendo M, Gould M, et al. Columbia Classification Algorithm of Suicide Assessment (C-CASA): classification of suicidal events in the FDA’s pediatric suicidal risk analysis of antidepressants. Am J Psychiatry. 2007;164(7):1035-1043. PubMed doi:10.1176/appi.ajp.164.7.1035

2. US Food and Drug Administration. FDA Updates Its Review of Antidepressant Drugs in Children [FDA Talk Paper]. http://www.fda.gov/bbs/topics/ANSWERS/2004/ANS01306.html. Published August 20, 2004. Accessed August 20, 2004.

3. US Food and Drug Administration. FDA reviews data for antidepressant use in adults [FDA Talk Paper]. Published July 1, 2005. http://www.fda.gov/bbs/topics/ANSWERS/2005/ANS01362.html. Accessed December 8, 2005.

4. Bridge JA, Iyengar S, Salary CB, et al. Clinical response and risk for reported suicidal ideation and suicide attempts in pediatric antidepressant treatment: a meta-analysis of randomized controlled trials. JAMA. 2007;297(15):1683-1696. PubMed doi:10.1001/jama.297.15.1683

5. Hammad TA, Laughren T, Racoosin J. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry. 2006;63(3):332-339. PubMed doi:10.1001/archpsyc.63.3.332

6. Klein DF. The flawed basis for FDA postmarketing safety decisions: the example of antidepressants and children. Neuropsychopharmacology. 2006;31(4):689-699. PubMed doi:10.1038/sj.npp.1300996

7. Simon GE, Savarino J. Suicide attempts among patients starting depression treatment with medications or psychotherapy. Am J Psychiatry. 2007;164(7):1029-1034. PubMed doi:10.1176/appi.ajp.164.7.1029

8. Valuck RJ, Libby AM, Sills MR, et al. Antidepressant treatment and risk of suicide attempt by adolescents with major depressive disorder: a propensity-adjusted retrospective cohort study. CNS Drugs. 2004;18(15):1119-1132. PubMed doi:10.2165/00023210-200418150-00006

9. National Committee for Quality Assurance. The State of Health Care Quality: 2005. http://www.ncqa.org/Docs/SOHCQ_2005.pdf. Accessed December 9, 2005.

10. Scholle SH. NCQA behavioral health measurement efforts. J Manag Care Pharm. 2005;11(3):S9-S11. PubMed

11. O’ Carroll PW, Potter LB. Centers for Disease Control and Prevention. Suicide Contagion and the Reporting of Suicide: Recommendations from a National Workshop. http://www.cdc.gov/mmwr/preview/mmwrhtml/00031539.htm. Published April 22, 1994. Accessibility verified April 30, 2009.

12. Valuck RJ, Libby AM, Benton T, et al. A descriptive analysis of 10,000 suicide attempters in United States managed care plans, 1998-2005. Prim Psychiatry. 2007;14(11):52-60.

13. Malone DC, Billups SJ, Valuck RJ, et al for the IMPROVE Investigators. Development of a chronic disease indicator score using a Veterans Affairs Medical Center medication database. J Clin Epidemiol. 1999;52(6):551-557. PubMed doi:10.1016/S0895-4356(99)00029-3

14. Berard A, Azoulay L, Koren G, et al. Isotretinoin, pregnancies, abortions and birth defects: a population-based perspective. Br J Clin Pharmacol. 2007;63(2):196-205. PubMed doi:10.1111/j.1365-2125.2006.02837.x

15. US Food and Drug Administration. Information for healthcare professionals: suicidality and antiepileptic drugs. http://www.fda.gov/cder/drug/InfoSheets/HCP/antiepilepticsHCP.htm. Published January 31, 2008. Accessed January 31, 2008.

16. Libby AM, Brent DA, Morrato EH, et al. Decline in treatment of pediatric depression after FDA advisory on risk of suicidality with SSRIs. Am J Psychiatry. 2007;164(6):884-891. PubMed doi:10.1176/appi.ajp.164.6.884

17. US Food and Drug Administration. FDA proposes new warnings about suicidal thinking, behavior in young adults who take antidepressant medications [FDA news]. http://www.fda.gov/bbs/topics/NEWS/2007/NEW01624.html. Published May 2, 2007. Accessed September 12, 2007.

18. SAS Institute Inc. SAS Language, Version 9.1. Cary, NC: SAS Institute; 2004.

19. Simon GE, Savarino J, Operskalski B, et al. Suicide risk during antidepressant treatment. Am J Psychiatry. 2006;163(1):41-47. PubMed doi:10.1176/appi.ajp.163.1.41

20. Baldessarini RJ, Tondo L, Strombom IM, et al. Ecological studies of antidepressant treatment and suicidal risks. Harv Rev Psychiatry. 2007;15(4):133-145. PubMed doi:10.1080/10673220701551102