Evaluation of Guideline-Concordant Care for Bipolar Disorder Among Privately Insured Youth

It has been noted that there is a “knowledge and practice gap”1 regarding the treatment of youth with bipolar disorder suggesting that the evidence and clinical guidelines available are not being applied in community settings. This issue may not be specific to bipolar disorder,1 but, given questions regarding bipolar phenomenology and the lack of evidence of the effectiveness of treatment among youth with this disorder, variation in care is likely to be prominent.2

The American Academy of Child and Adolescent Psychiatry3 practice parameters for the assessment and treatment of children and adolescents with bipolar disorder are based on expert consensus and literature review. Use of multimodal treatment, combining pharmacotherapy and adjunctive psychosocial interventions, is emphasized. Despite small samples and some methodological issues with lithium trials, this medication has the most evidence and the longest history of use in children and adolescents with early-onset bipolar disorder.4-8 Use of mood stabilizers or other psychopharmacologic agents may be warranted; however, care must be taken to avoid unnecessary polypharmacy.3 Maintenance and follow-up treatment are recommended to prevent relapse, although the evidence for this in children is still lacking.3

Additional bipolar treatment guidelines for children and adolescents have been published in the Journal of the American Academy of Child and Adolescent Psychiatry,3,9 as well as by the National Institute for Health and Clinical Excellence (NICE)10 and the Canadian Network for Mood and Anxiety Treatments (CANMAT),11 which include only a brief section pertaining to children and adolescents. Despite the limited evidence base for bipolar treatment in this population, there are many similarities across the various guidelines.12 The intent of treatment guidelines is to provide clinicians with the best available scientific and clinical evidence to support the use of therapeutic interventions. However, adoption and use of the recommended interventions in clinical practice are not well understood, particularly for a controversial disorder such as early-onset bipolar disorder. Additionally, while the evidence for improved outcomes due to improved guideline concordance is clear for other behavioral health disorders, this association has not been shown in youth with bipolar disorder.13,14 This study focuses on the 1997 American Academy of Child and Adolescent Psychiatry guidelines,3 which were the most recently published guidelines in the United States at the time of the study period (2000-2001). The aims of this study were to (1) estimate the prevalence of guideline concordance in a sample of youth with bipolar disorder; (2) describe relationships between each of the guideline variables; and (3) examine the relationship between guideline concordance and likelihood of an inpatient hospitalization or an emergency department visit.

The following was hypothesized: (1) the majority of youth diagnosed with bipolar disorder will not be receiving care concordant with current guidelines; (2) there will be a weak correlation between concordance with each guideline variable; and (3) guideline concordance will be associated with a decreased likelihood of inpatient hospitalization or emergency department use. This project was approved by the Johns Hopkins University Bloomberg School of Public Health Institutional Review Board, Baltimore, Maryland.

METHOD

Data Source

The data are from the Thomson Medstat MarketScan (2000-2001) database (Thomson Medstat, Ann Arbor, Michigan), a national (US) dataset containing standardized, detailed, enrollee-specific clinical utilization information across inpatient and outpatient services, and prescription drug information from approximately 45 employer-sponsored health plans. Medication utilization was obtained from pharmacy files that include all manual and electronically submitted claims for prescriptions filled through outpatient community, mail order, and specialty pharmacies. Information from the outpatient pharmacy claims data that was used in the present study included enrollee-specific deidentified prescription dispensing date, quantity dispensed, and National Drug Code (NDC). The NDC, a unique number assigned by the US Food and Drug Administration (FDA), identifies manufacturer and medication product formulation. Because the medication name is not recorded on the pharmacy claims file, the Thomson Medstat MarketScan database was linked to the Multum Lexicon database using the NDC to extract the drug name.15

Sample Characteristics

Behavioral health claims of an initial cohort (N = 423) of privately insured youth 6-18 years of age who had at least 2 first listed outpatient claims or 1 first listed inpatient claim associated with a bipolar disorder diagnosis (ICD-9-CM codes: 2960; 2964-2968) were examined. This method of case identification has been associated with a high positive predictive value (94%) and modest sensitivity (80%).16 In studies of diagnostic validity, bipolar disorder and depression diagnoses obtained from claims data have been found to be accurate 75% to 87% of the time.17-19

The initial cohort was identified as youth who did not have any type of bipolar disorder diagnosis during the first 6 months of the study period. Only youth having complete coverage for the 2-year period were included to ensure they had the opportunity to access all types of care, addressing other factors such as copayment or distance to facilities that may vary among individuals. Youth with 2 years of continuous enrollment were likely to differ from those insured for shorter periods of time. Indeed, compared with youth enrolled for 1 year or less, those with 2 years of continuous enrollment were more likely to be in a health maintenance organization (HMO) and less likely to be in a point of service plan (POS). Also, those with 2 years versus 1 year or less of continuous enrollment were significantly more likely to live in the Northeast (27.7% vs 9.4%) and less likely to live in the West (22.1% vs 4.9%) (P <.001). There were, however, no age or gender differences between these groups. So, although this is not a probabilistically representative sample of the entire nation, it is a large sample of youth from all regions of the country who were enrolled during 2000 and 2001 in all types of private health insurance plans. Youth who did not have 6 months of data following the index diagnosis of bipolar disorder were excluded because they would have insufficient data to assess adherence. Youth with seizure disorders who may receive the same medications given to youth with bipolar disorder were also excluded because it would be difficult to determine for which diagnosis the medication had been prescribed. Youth with any claims for schizophrenia or schizoaffective disorder were also excluded from the analysis.

Age was categorized into 4 groups: 6-11 years (n = 67), 12-14 years (n = 108), 15-16 years (n = 136), and 17-18 years (n = 112) years. The group aged 6-11 years encompasses a wider range than the others because there were few children younger than 12 years in the study sample. To address concerns that the 6- to 9-year-olds (n = 33) are developmentally different from 10- to 11-year-olds (n = 38) and their service utilization might be different, several analyses were conducted and showed no consistent pattern of differences between these subgroups.

Study Variables

Health status and comorbidity. The Johns Hopkins Adjusted Clinical Groups (ACG) Case-Mix System, Version 8.0, controlled for individual health status and co-occurring medical and psychiatric diagnoses and has been shown to be a valid tool for case-mix adjustment of morbidity and severity.20-22 The ACG system is an empirically derived algorithm that uses administrative data of health care services. Adjustments are made for age and gender for individuals with similar morbidity and anticipated health care resource consumption. Individuals are stratified into resource utilization bands (RUBs) with scores ranging from 0 (none) to 5 (high) anticipated use of future health care resources. These categories have been empirically validated in the United States22-24 and internationally,25,26 and it has been demonstrated that people with higher RUBs have more severe comorbidity and use more medical care resources.20,27

Density of Child Psychiatrists

The 2001 Area Resource File (ARF) was used to obtain the availability of child psychiatrists in each county.28 The total number of child psychiatrists in the youth’s county of residence was divided by the number of 6- to 18-year-olds living in the county (also obtained through the ARF) to construct the variable density of child psychiatrists for youth. ARF data were linked to the claims data using the county of residence for each youth.

Guideline Concordance

Services received by each youth for treatment of bipolar disorder were compared to what was specified in the practice guidelines.3 Operationalization of each guideline measure is outlined later in the article. Although the guidelines are clear that long-term maintenance therapy is needed to prevent relapse, they note that evidence for the length of treatment needed is unclear. Therefore, for maintenance treatment variables, a conservative estimate of 6 months’ maintenance was used. Guidelines measured include (1) prescription of at least 1 mood stabilizer or atypical antipsychotic; (2) no more than 2 mood stabilizers and 1 antipsychotic at a time; (3) long-term maintenance therapy with an antimanic agent; (4) establishment of a strong therapeutic relationship including regular follow-up assessments; (5) use of concurrent psychotherapy; and (6) antidepressants are not to be used as monotherapy and must be used in conjunction with a mood stabilizer. For guidelines 1 and 3, the prevalence analysis examines 2 groups of youth: (1) youth using a mood stabilizer (though they may also have adjunctive antipsychotic use) and (2) youth not using a mood stabilizer, but having a prescription of an antipsychotic (guideline 1) or maintenance therapy with an antipsychotic (guideline 3). For the additional analysis of guideline concordance, only the more conservatively defined group of youth who used a mood stabilizer (group 1) was analyzed.

Dependent Variables

The 2 main dependent variables were at least 1 claim or encounter associated with (1) an emergency department visit and (2) a hospital admission during the 2-year study period. These services were identified from outpatient claims files and hospital admission records.

Statistical Analysis

The unit of analysis for this study was the youth enrollee. Generalized estimating equations (GEE) with an exchangeable correlation structure and robust variance estimates were used to estimate odds ratios and 95% confidence for the likelihood of an emergency department visit or inpatient hospitalization. The GEE model is a general linear model used for clustered data. It accounts for multiple events (ie, ED visit and hospitalization) and adjusts for within-subject correlation. GEE does not require follow-up observations for all individuals.29 The odds ratio was adjusted for sociodemographic characteristics (age, sex, urbanicity), clinical characteristics (comorbidity), and type of insurance plan, as these may be related to service use. Analyses were done to assess collinearity, interaction, and confounding among the study variables and covariates. Data were analyzed using SAS v 9.1 (SAS Institute, Cary, North Carolina). Statistical significance was set at the 5% level.

RESULTS

Youth had a mean age of 14.4 years. About half of the youth were male (52%), while 12% were from a rural area and 88% were from an urban/suburban area. More than three-quarters of the youth (77.5%) were enrolled in a managed care plan (Table 1).

Guideline Concordance

All 423 youth had complete insurance coverage for the 2-year period, which ensured that they had equal opportunity to access care as recommended in the treatment guidelines. Prevalence rates and 95% confidence intervals of guideline concordance for all youth are summarized in Table 2 for each guideline criterion. Most youth were not receiving more than the recommended number of mood-stabilizing medications or antipsychotics concomitantly (89%). As recommended, two-thirds of youth did not receive antidepressant monotherapy. About half received monthly visits with a psychiatrist, and more than one third of youth received any type of psychotherapy, but many fewer were receiving continuous psychotherapy (on a monthly basis) (15%). Twenty-six percent (n = 109) received a mood-stabilizing medication or atypical antipsychotic within a month of diagnosis, but only 5% maintained this treatment over the 6-month follow-up period.

Correlation Among Guidelines

The associations between the guidelines (ie, the likelihood that complying with any one guideline was associated with complying with another) ranged from low (0.01) to moderate (r = −0.3 to 0.4). It appears that some correlations between guideline variables are present because of the structure of the guideline. For instance, there is a moderately positive correlation (r = 0.4) between guidelines 4 (establishing a strong therapeutic relationship) and 5 (use of concurrent psychotherapy), which is most likely related to the fact that guideline 4 calls for monthly visits and guideline 5 for use of psychotherapy, and if one is involved in psychotherapy, monthly visits would be more likely. Guideline 2 (no more than 2 mood stabilizers and 1 antipsychotic at a time) is moderately negatively correlated with guidelines 1 (prescription of at least 1 mood stabilizer) and 3 (long-term maintenance therapy with a mood stabilizer) (ie, those who meet guideline 2 are less likely to meet 1 as well as less likely to meet 3). Guidelines 1 and 3, however, are structurally very similar and positively correlated at 0.4. Guideline 6 (antidepressants are not to be used as monotherapy) is not strongly related to any of the other guidelines.

Association of Guideline Concordance With Emergency Department Visits and Inpatient Hospitalizations

For 4 of the 6 guidelines, compliance was associated with a significantly lower likelihood of an emergency department visit. This association also was found, but to a lesser extent, with an inpatient hospitalization. Receiving no more than 2 mood stabilizers and 1 antipsychotic at a time was significantly associated with a lower likelihood of an inpatient hospitalization. Use of any psychotherapy (OR = 2.08, 95% CI = 1.32-3.19) and use of continuous psychotherapy (OR = 2.34, 95% CI = 1.19-4.60) were statistically significantly associated with a higher likelihood of inpatient hospitalizations, although these variables were not significantly associated with emergency department visits. Female gender, lower density of child psychiatrists, and lower RUB score were also significantly associated with a lower likelihood of emergency department use (P ≤ .05). Female gender, lower density of child psychiatrists, and being enrolled in a managed care plan were associated with a lower likelihood of an inpatient hospitalization (P ≤ .05).

DISCUSSION

This study estimated the overall prevalence of concordance with bipolar treatment guidelines among a privately insured youth population and found that better concordance was associated with fewer hospitalizations and emergency department visits during the 2-year study period. The guideline recommendation to use at most 2 mood stabilizers and 1 antipsychotic simultaneously was followed quite consistently. However, concordance was lower for prescription and maintenance use of mood-stabilizing or antipsychotic medications, concurrent use of psychotherapy, and use of antidepressants as monotherapy. About one third of youth received antidepressants as monotherapy, which is supported by other research.30

Relationships among receipt of concordant treatment were examined to better understand the extent to which certain treatment recommendations were more likely or less likely to co-occur (eg, whether those who get psychotherapy are more likely or less likely to be getting other guideline-recommended care). It was found that concordance with one guideline, however, did not ensure that other recommended treatment guidelines were followed. Correlations were only low to moderate (range, 0.05-0.4); if guideline compliance overall were high, it is likely that the correlations would have been higher.

The low guideline concordance demonstrated by this study is very likely attributable to both patient and provider behaviors since psychotropic medication prevalence is based on prescription fills. Patient adherence to treatment is a significant issue in patients with bipolar disorder and very likely influences the rates of guideline-concordant care in this sample. Overall adherence in medical practice has been shown to be around 50%,31 and Perlick and colleagues32 have shown the median rate of nonadherence in adults (age 16 or over) with bipolar disorder to be around 42%. Interestingly, our data also showed that although several youth (56%) filled a prescription for a mood-stabilizing or antipsychotic medication over the 6-month follow-up period, only 26% filled a prescription within the first month of diagnosis, and only 5% filled a prescription on a continuous monthly basis over the 6-month follow-up period.

Guideline concordance is very likely influenced by physicians with prescribing practices that are not concordant with treatment guidelines, patients who are noncompliant with prescribed treatments, and possibly patients who have an idiosyncratic presentation or a more complex or severe form of bipolar disorder that is more difficult to treat and that might warrant alternative types of treatment. The overall association of emergency department visits and inpatient hospitalizations with guideline compliance, however, suggests that those children receiving guideline-concordant care may have a lower chance of experiencing an adverse behavioral health event leading to the use of the emergency department or inpatient services.

In isolation, most guidelines were associated with a lower likelihood of an emergency department visit and, to a lesser degree, inpatient hospitalizations. Table 3 suggests that concordance with certain guidelines might be protective in terms of emergency department use and inpatient hospitalizations and that more effort in determining the relative impact of guidelines on these high cost services might be clinically useful. Use of concurrent psychotherapy is associated with a higher likelihood of an inpatient hospitalization. Although this is counterintuitive given the positive evidence for treatment of bipolar disorder with psychotherapy, this may be explained by youth with more severe or complex cases being more likely to receive psychotherapy. Additionally, this may reflect the range of type and quality of psychotherapy provided and coded as “psychotherapy.” It is clear that much more information is needed to better understand how to best treat youth with bipolar disorder, but this study provides preliminary evidence that the standards of care that we do have available may significantly impact the outcomes for youth with bipolar disorder.

As with any analysis of claims data, it is important to note that claims data provide only an estimate of the treated prevalence of bipolar disorder in children and adolescents and that the validity of the diagnoses cannot be verified. This study population excluded those without insurance and was not representative of those with insurance provided through some mechanism other than employer. Additionally, children and adolescents with a validated diagnosis of bipolar disorder are a heterogeneous group1,33 varying in type, number, and periodicity of classical bipolar symptoms, and claims data cannot capture all aspects of their clinical profile. These data, however, allow us to construct a picture of a substantial number of youth treated “as usual” in a community setting, and this is a strength given the low prevalence of the disorder and issues around generalizability of small clinical samples.

Although these data provide novel preliminary evidence about treatments and services received by youth diagnosed with bipolar disorder in the community, we cannot yet make firm conclusions about why a gap in quality exists. Claims data offer only information on whether the treatment was received by the patient. Nevertheless, all children in the cohort examined were diagnosed with and are presumably being treated for bipolar disorder, and therefore the guideline recommendations should be relevant for them. Further research is needed to explore why these patterns are occurring and to what extent they result from physicians’ lack of awareness of treatment guidelines, physicians who knowingly prescribe an alternative treatment due to an idiosyncratic clinical presentation, or patients or parents who avoid or refuse prescribed treatments. Finally, although conformance to some of the guidelines was not associated with a significant reduction in emergency department visits or inpatient hospitalizations, this does not mean that the recommendations in those guidelines are not clinically important parts of treatment. Emergency department visits and hospitalizations are useful utilization-based indicators of poor outcomes, but do not measure overall outcomes of bipolar treatment.

These data indicate that greater efforts are needed to translate clinical research findings into everyday practice. Research has shown that simply providing information, such as evidence-based clinical guidelines, will not increase evidence-based practice.34 Additional data, however, also suggest that specific provider training in combination with a collaborative approach may improve guideline concordance in patients with bipolar disorder.35 In this study, prescription patterns of certain contraindicated types and combinations of medications most likely reflect physician-driven behavior and seem to suggest that many physicians are not aware of or do not agree with treatment guidelines. Specific barriers to guideline concordance particular to this population, however, are not clearly evident and warrant further investigation.

Low rates of guideline concordance may also be explained by physician uncertainty around treating youth with certain medications indicated for bipolar disorder. Physicians may feel more comfortable prescribing stimulants, as there is more research on their efficacy and safety in child and adolescent populations, or antidepressants, which may have been acceptable at the time (2000-2001), prior to the FDA black box warning of 2004.36 A similar study looking at guideline concordance in a privately insured adult population with bipolar disorder showed higher rates of guideline concordance37 compared to this pediatric study population. The study in the adult population found about half as many adults as youth in the current study were receiving contraindicated antidepressant monotherapy (33% vs 14%, respectively). The study in adults also found higher rates of indicated mood-stabilizing medication prescriptions and higher rates of psychotherapy. More uncertainty around diagnosis in youth and better evidence for treatment in adults may explain some of the variance in guideline concordance. These data also indicate that physicians may be aware of general guidelines for the treatment of bipolar disorder, but physicians or parents of youth with bipolar disorder may be reluctant to use certain treatments in children and adolescents due to weak evidence in youth, or they may just be unaware of treatment guidelines for youth with bipolar disorder. Physicians may extrapolate certain guidelines or recommendations from adult literature, however, which they feel are safe for children and adolescents. Interestingly, although limiting antipsychotic treatment solely to second-generation antipsychotics was not included in the child and adolescent guidelines9 until 2005, it was almost perfectly adhered to for this cohort of youth (only 1% of youth ever filled a prescription for a first-generation antipsychotic). This seems to be a practice that was adopted rigorously by clinicians, possibly on the basis of evidence from adults.38

The fact that the majority of the guidelines were associated with a lower use of emergency department visits when controlling for other demographic characteristics sends a strong message for the potential value of the guidelines. The Institute of Medicine defines quality of care as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”39 These data suggest that incorporating clinical guidelines for the treatment of bipolar disorder in children and adolescents may help to ensure a higher standard of quality and reduce clinical uncertainty. The fact that guideline concordance does not always ensure a reduction in emergency department visits or inpatient hospitalizations, however, also suggests that other factors are important in determining patient outcomes and that guidelines may not be equally effective for all youth with bipolar disorder. Therefore, careful consideration of individual circumstances is an important part of the care of youth with bipolar disorder.

Further research regarding specific barriers to guideline concordance for youth with bipolar disorder is warranted. Problems with guideline conformance demonstrated by the data may be influenced by refusal or nonadherence to prescribed treatment by patients or parents. Therefore, a research approach that investigates barriers to guideline concordance from the perspective of the patient and family may benefit this population. In addition to identifying barriers to guideline concordance specific to this population, it is important to collect information that provides more detailed evidence of treatments and services provided to youth diagnosed with bipolar disorder and that may identify subgroups of youth receiving different patterns of care. Further studies that provide more detail around the clinical presentation of the youth and treatments and services provided would help to address this issue.

Although deviation from guidelines may be warranted in some cases due to individual variation and patient complexity or patient and/or family preferences, the evidence base available in conjunction with expert consensus should help to guide care management strategies and improve outcomes and quality of care for the majority of patients. Explicit incorporation of guideline-concordant care is likely to increase the overall quality of care and presage better long-term outcomes for youth diagnosed with bipolar disorder.

Disclosure of off-label usage: The authors have determined that, to the best of their knowledge, no investigational information about pharmaceutical agents that is outside US Food and Drug Administration−approved labeling has been presented in this article.

Author affiliations: Health Services Research Department, Institute of Psychiatry, King’s College London, London, United Kingdom (Dr Evans-Lacko); Division of Child & Adolescent Psychiatry, Department of Psychiatry, The Johns Hopkins University School of Medicine, Baltimore, Maryland (Drs dosReis and Kastelic); Postgraduate Program in Developmental Disorders, Mackenzie Presbyterian University, Sao Paulo, Brazil (Dr Paula); and Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (Dr Steinwachs).

Financial disclosure: Drs Evans-Lacko, dosReis, Kastelic, Paula, and Steinwachs have no personal affiliations or financial relationships with any commercial interest to disclose relative to the article.

Funding/support: The National Institute of Mental Health, Bethesda, Maryland, provided funding support (T32 MH 19545), and Thomson Medstat, Ann Arbor, Michigan, provided data.

Previous presentation: Presented as a poster at the Seventh International Conference on Bipolar Disorder, Pittsburgh, Pennsylvania, June 7-9, 2007.

This CME activity is expired. For more CME activities, visit cme.psychiatrist.com.
Find more articles on this and other psychiatry and CNS topics:
The Journal of Clinical Psychiatry
The Primary Care Companion for CNS Disorders

REFERENCES

1. Post RM, Kowatch RA. The health care crisis of childhood-onset bipolar illness: some recommendations for its amelioration. J Clin Psychiatry. 2006;67(1):115-125. PubMed doi:10.4088/JCP.v67n0118

2. Wennberg JE. Practice variation: implications for our health care system. Manag Care. 2004;13(suppl):3-7. PubMed

3. American Academy of Child and Adolescent Psychiatry. Practice parameters for the assessment and treatment of children and adolescents with bipolar disorder. J Am Acad Child Adolesc Psychiatry. 1997;36(1):138-156.

4. Brumback RA, Weinberg WA. Mania in childhood, 2: therapeutic trial of lithium carbonate and further description of manic-depressive illness in children. Am J Dis Child. 1977;131(10):1122-1126. PubMed

5. DeLong R. Lithium treatment and bipolar disorders in childhood. N C Med J. 1990;51(4):152-154. PubMed

6. Geller B, Cooper TB, Sun K, et al. Double-blind and placebo-controlled study of lithium for adolescent bipolar disorders with secondary substance dependency. J Am Acad Child Adolesc Psychiatry. 1998;37(2):171-178. PubMed doi:10.1097/00004583-199802000-00009

7. Lena B. Lithium in child and adolescent psychiatry. Arch Gen Psychiatry. 1979;36(8 Spec No):854-855. PubMed

8. McKnew DH, Cytryn L, Buchsbaum MS, et al. Lithium in children of lithium-responding parents. Psychiatry Res. 1981;4(2):171-180. PubMed doi:10.1016/0165-1781(81)90020-2

9. Kowatch RA, Fristad M, Birmaher B, et al. Child Psychiatric Workgroup on Bipolar Disorder. Treatment guidelines for children and adolescents with bipolar disorder. J Am Acad Child Adolesc Psychiatry. 2005;44(3):213-235. PubMed doi:10.1097/00004583-200503000-00006

10. Ferrier N, Pilling S, Bazire S. Bipolar Disorder: The Management of Bipolar Disorder in Adults, Children, and Adolescents, in Primary and Secondary Care. Leicester, UK: The British Psychological Society; 2006. http://www.nice.org.uk/page.aspx?o=63354. Accessed Jan 6, 2007.

11. Yatham LN, Kennedy SH, O’ Donovan C, et al. Canadian Network for Mood and Anxiety Treatments. Canadian Network for Mood and Anxiety Treatments (CANMAT) guidelines for the management of patients with bipolar disorder: consensus and controversies. Bipolar Disord. 2005;7(suppl 3):5-69. PubMed

12. Calabrese JR, Keck PE Jr, Bowden CL, et al. A US perspective of the CANMAT bipolar guidelines. Bipolar Disord. 2005;7(suppl 3):70-72. PubMed

13. Owen RR, Thrush CR, Kirchner JE, et al. Performance measurement for schizophrenia: adherence to guidelines for antipsychotic dose. Int J Qual Health Care. 2000;12(6):475-482. PubMed doi:10.1093/intqhc/12.6.475

14. Sohler NL, Walkup J, McAlpine D, et al. Antipsychotic dosage at hospital discharge and outcomes among persons with schizophrenia. Psychiatr Serv. 2003;54(9):1258-1263. PubMed doi:10.1176/appi.ps.54.9.1258

15. Multum Lexicon Database. Denver, CO: Cerner Multum; 2005. http://www.multum.com/Lexicon.htm. Accessed Jan 6, 2007.

16. Lurie N, Popkin M, Dysken M, et al. Accuracy of diagnoses of schizophrenia in Medicaid claims. Hosp Community Psychiatry. 1992;43(1):69-71. PubMed

17. Blow FC, McCarthy JF, Valenstein M, et al. Care for Veterans With Psychosis in the VHA, FY03: 5th Annual National Psychosis Registry Report. Ann Arbor, MI: Veterans Affairs Serious Mental Illness Treatment Research and Evaluation Center; 2004. >

18. Kashner TM. Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs. Med Care. 1998;36(9):1324-1336. PubMed doi:10.1097/00005650-199809000-00005

19. Un×¼tzer J, Simon G, Pabiniak C, et al. The use of administrative data to assess quality of care for bipolar disorder in a large staff model HMO. Gen Hosp Psychiatry. 2000;22(1):1-10. PubMed doi:10.1016/S0163-8343(99)00057-2

20. Weiner JP, Abrams C, Millman A, eds. The Johns Hopkins ACG Case-Mix System: Documentation & Application Manual. Baltimore, MD: The Johns Hopkins University; 2001.

21. Baldwin LM, Klabunde CN, Green P, et al. In search of the perfect comorbidity measure for use with administrative claims data: does it exist? Med Care. 2006;44(8):745-753. PubMed doi:10.1097/01.mlr.0000223475.70440.07

22. Weiner JP, Starfield BH, Steinwachs DM, et al. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care. 1991;29(5):452-472. PubMed doi:10.1097/00005650-199105000-00006

23. Rosen AK, Loveland S, Anderson JJ, et al. Evaluating diagnosis-based case-mix measures: how well do they apply to the VA population? Med Care. 2001;39(7):692-704. PubMed doi:10.1097/00005650-200107000-00006

24. Weiner JP, Starfield BH, Powe NR, et al. Ambulatory care practice variation within a Medicaid program. Health Serv Res. 1996;30(6):751-770. PubMed

25. Carlsson L, Strender LE, Fridh G, et al. Types of morbidity and categories of patients in a Swedish county: applying the Johns Hopkins Adjusted Clinical Groups System to encounter data in primary health care. Scand J Prim Health Care. 2004;22(3):174-179. PubMed doi:10.1080/02813430410006567

26. Orueta JF, Lopez-De-Munain J, Bסez K, et al. Application of the ambulatory care groups in the primary care of a European national health care system: does it work? Med Care. 1999;37(3):238-248. PubMed doi:10.1097/00005650-199903000-00004

27. Broemeling A, Watson D, Black C. Chronic Conditions and Co-Morbidity Among Residents of British Columbia. Vancouver, BC, Canada: Centre for Health Services and Policy Research; February 2005. http://www.chspr.ubc.ca/files/publications/2005/chspr05-08.pdf. Accessed March 7, 2007.

28. US Department of Health and Human Services. Area Resource File (ARF) 2004. Rockville, MD: Health Resources and Services Administration; 2004.

29. Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121-130. PubMed doi:10.2307/2531248

30. Moreno C, Laje G, Blanco C, et al. National trends in the outpatient diagnosis and treatment of bipolar disorder in youth. Arch Gen Psychiatry. 2007;64(9):1032-1039. PubMed doi:10.1001/archpsyc.64.9.1032

31. Young JL, Spitz RT, Hillbrand M, et al. Medication adherence failure in schizophrenia: a forensic review of rates, reasons, treatments, and prospects. J Am Acad Psychiatry Law. 1999;27(3):426-444. PubMed

32. Perlick DA, Rosenheck RA, Clarkin JF, et al. Impact of family burden and affective response on clinical outcome among patients with bipolar disorder. Psychiatr Serv. 2004;55(9):1029-1035. PubMed doi:10.1176/appi.ps.55.9.1029

33. Birmaher B, Axelson D, Strober M, et al. Clinical course of children and adolescents with bipolar spectrum disorders. Arch Gen Psychiatry. 2006;63(2):175-183. PubMed doi:10.1001/archpsyc.63.2.175

34. Shojania KG, Ranji SR, McDonald KM, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA. 2006;296(4):427-440. PubMed doi:10.1001/jama.296.4.427

35. Dennehy EB, Bauer MS, Perlis RH, et al. Concordance with treatment guidelines for bipolar disorder: data from the Systematic Treatment Enhancement Program for Bipolar Disorder. Psychopharmacol Bull. 2007;40(3):72-84. PubMed

36. US Food and Drug Administration. Suicidality in Children and Adolescents Being Treated With Antidepressant Medications. October 15, 2004. http://www.fda.gov/Drugs/DrugSafety/PublicHealthAdvisories/ucm161679.htm. Accessed April 7, 2010.

37. Busch AB, Ling D, Frank RG, et al. Changes in the quality of care for bipolar I disorder during the 1990s. Psychiatr Serv. 2007;58(1):27-33. PubMed doi:10.1176/appi.ps.58.1.27

38. Kahn DA, Sachs GS, Printz DJ, et al. Medication treatment of bipolar disorder 2000: a summary of the expert consensus guidelines. J Psychiatr Pract. 2000;6(4):197-211. PubMed

39. Institute of Medicine. Crossing the Quality Chasm: The IOM Health Care Quality Initiative. 2007. http://www.iom.edu/?id=18795. Accessed January 6, 2007.