Pilot Study of Implementation of an Internet-Based Depression Prevention Intervention (CATCH-IT) for Adolescents in 12 US Primary Care Practices: Clinical and Management/Organizational Behavioral Perspectives

Pilot Study of Implementation of an Internet-Based Depression Prevention Intervention (CATCH-IT) for Adolescents
in 12 US Primary Care Practices: Clinical and
Management/Organizational Behavioral Perspectives

Jeffrey C. Eisen, MD, MBA; Monika Marko-Holguin, MSS; Joshua Fogel, PhD;
Alonso Cardenas, BA; My Bahn, PhD; Nathan Bradford, MD; Blake Fagan, MD;
Peggy Wiedmann, MD; and Benjamin W. Van Voorhees, MD, MPH

ABSTRACT

Objective: To explore the implementation of CATCH-IT (Competent Adulthood Transition with Cognitive-behavioral Humanistic and Interpersonal Training), an Internet-based depression intervention program in 12 primary care sites, occurring as part of a randomized clinical trial comparing 2 versions of the intervention (motivational interview + Internet program versus brief advice + Internet program) in 83 adolescents aged 14 to 21 years recruited from February 1, 2007, to November 31, 2007.

Method: The CATCH-IT intervention model consists of primary care screening to assess risk, a primary care physician interview to encourage participation, and 14 online modules of Internet training to teach adolescents how to reduce behaviors that increase vulnerability to depressive disorders. Specifically, we evaluated this program from both a management/organizational behavioral perspective (provider attitudes and demonstrated competence) and a clinical outcomes perspective (depressed mood scores) using the RE-AIM model (Reach, Efficacy, Adoption, Implementation, and Maintenance of the intervention).

Results: While results varied by clinic, overall, clinics demonstrated satisfactory reach, efficacy, adoption, implementation, and maintenance of the CATCH-IT depression prevention program. Measures of program implementation and management predicted clinical outcomes at practices in exploratory analyses.

Conclusion: Multidisciplinary approaches may be essential to evaluating the impact of complex interventions to prevent depression in community settings. Primary care physicians and nurses can use Internet-based programs to create a feasible and cost-effective model for the prevention of mental disorders in adolescents in primary care settings.

Trial Registration: ClinicalTrials.gov identifiers: NCT00152529 and NCT00145912

Prim Care Companion CNS Disord 2013;15(6):doi:10.4088/PCC.10m01065

© Copyright 2013 Physicians Postgraduate Press, Inc.

Submitted: August 10, 2010; accepted October 2, 2013.

Published online: December 19, 2013.

Corresponding author: Benjamin W. Van Voorhees, MD, MPH, University of Illinois at Chicago, 840 South Wood St M/C 856, Chicago, IL 60612 ([email protected]).

Depression is the most common mental disorder in adolescence, affecting school performance, personal relationships, and future aspirations.1–9 A scarcity of providers dedicated to child and adolescent mental health, limited coverage for mental health services, recent recommendations for screening and treatment in primary care, and success of chronic care10–12 and other model interventions10–14 have reinforced the primary care physician role for intervening in adolescent depressive illness.10–12 However, most adolescents do not seek treatment, and even with full implementation of evidence-based strategies, the burden of this disorder will only be partially reduced.11,12 Also, few such programs have been found to decrease the incidence of depressive disorders in youth.13,15 To date, the most effective programs utilize a group psychotherapy format, which is difficult to implement and is acceptable to only a minority of at-risk adolescents.16,17 Internet-based approaches for treatment of adults with depressed mood and anxiety have demonstrated promise in trials in Europe and Australia, but few interventions have been developed for targeting adolescents.18–25 Similarly, new and lower-cost approaches will be in increasing demand for the emerging patient-centered medical home and accountable care organization models.26

To address the need for a feasible and acceptable behavioral and preventive intervention for depression in the primary care setting, we developed CATCH-IT (Competent Adulthood Transition with Cognitive-behavioral Humanistic and Interpersonal Training) (ClinicalTrials.gov identifiers: NCT00152529 and NCT00145912).27,28 The CATCH-IT intervention model consists of primary care screening to assess risk, a primary care physician interview to encourage participation, and 14 online modules of Internet training to teach adolescents how to reduce behaviors that increase vulnerability for depressive disorders.13,29–31 The rationale for adding the primary care interview was to enhance the adherence and therapeutic aspects of the Internet modules.15,18,23,25,32

In order to determine how much physician contact would be required to engage adolescents with the Internet intervention, we conducted a randomized clinical trial comparing a primary care physician motivational interview to primary care physician brief advice. Both randomization groups substantially engaged the Internet site (motivational interview: 90.7% vs brief advice: 77.5%), and the motivational interview group demonstrated greater overall use. In both groups, 10-item Center for Epidemiologic Studies Depression Scale (CES-D-10) scores declined from baseline to 12 weeks (motivational interview: 24.0 to 17.0, P < .001; brief advice: 25.2 to 15.5, P < .001).33,34 In both groups, the percentage of those with clinically significant depression symptoms based on CES-D-10 scores declined (motivational interview: 52%–12%, P < .001; brief advice: 50%–15%, P < .001). However, the motivational interview group demonstrated fewer clinical depressive episodes at 2-, 6-, and 12-month follow-up.33,34 Furthermore, both groups demonstrated increases in social support and reduction in depression, which were associated with reduced school impairment. While these results proved promising, the movement of evidence-based multicomponent interventions toward community practice implementation also proved to be challenging.35–41 This implementation process for primary care–based depression prevention using an Internet-based model has not been described here.

We believe that careful evaluation of complex interventions early in the development cycle is critical in identifying key factors that may interfere with or promote implementation and the eventual public health impact. Many barriers have been identified including provider attitudes,42,43 intervention complexity,44,45 and practice context and structure.46 Unfortunately, interventions are rarely subjected to rigorous evaluations of their actual potential public health impact. Recognizing this dilemma, Glasgow et al47 developed the RE-AIM model to assess the public health impact of interventions in actual settings. RE-AIM is a systematic method designed to estimate such factors as program Reach, Efficacy, Adoption, Implementation, and Maintenance on public health impact. Although there are limited empirical data on RE-AIM metrics, they extend current methods that identify interventions most likely to impact population health and provide evidence that such programs are worth sustained investment because of their feasibility and sustainability in real-world environments.47

The purpose of this study was to explore the implementation of a first-of-its-kind primary care, Internet-based depression intervention for adolescents. Specifically, we examine practice structure/setting, knowledge and capability to perform the intervention, attitudes of providers toward the intervention, and the potential public health impact of the intervention (RE-AIM framework). We then take the next step to explore the relationship between these potential barriers, structure/setting, knowledge and capability to perform the intervention, and the attitudes of providers toward the intervention, on the 5 RE-AIM outcomes (reach, efficacy, adoption, implementation, maintenance on public health impact). First, we hypothesized that providers would have favorable attitudes toward the CATCH-IT intervention even if they held less favorable ones toward conventional mental health interventions. Second, we hypothesized that physicians could gain basic minimal proficiency in motivational interviewing after just a 1-hour instruction session during lunch. Third, we hypothesized that, with regard to public health impact, clinical practices might vary considerably in their ability to perform the intervention and that this variability would affect the potential public health impact.

METHOD

Overall Study Design

Participants and eligibility. Adolescents and emerging adults from 14 to 21 years of age were recruited from February 1, 2007, to November 31, 2007, by direct primary care screening for risk of disorder (subthreshold depressive symptoms) and by posted advertisements.48 Those adolescents with positive screens provided informed consent (or had their parents provide informed consent and they provided assent) to either study staff or the principal investigator. We followed adolescents to evaluate their clinical outcomes by telephone at 6 and 12 weeks and with a self-administered poststudy questionnaire at 4 to 6 weeks. All protocols were approved by the University of Chicago Institutional Review Board, Chicago, Illinois. A detailed description of the trial methods has been previously reported.34

Intervention. We conducted a randomized controlled trial comparing motivational interview + Internet program (motivational interview group) versus brief advice + Internet program (brief advice group) in 12 US primary care sites (for purposes of this analysis, 2 primary care sites located 1 block from each other were combined for the total of 12 sites described in this study) within 5 different health systems spanning 4 states located in the US Midwest and Southeast. Primary care physicians performed initial and follow-up interviews for each participant, while nurses performed the screening for risk. For the brief advice condition (1–2 minutes), the physician used a “directive” approach to advise the adolescent that he/she was experiencing depressed mood and was at risk for depressive disorder and recommended using the CATCH-IT Internet site. With regard to the motivational interview condition (6–15 minutes), the physician employed an “autonomy-focused” approach to help the adolescent develop a positive personal cost/benefit assessment toward completing the intervention and building resiliency. Both groups received equal and private (secure sign-in) access to the Internet site. The CATCH-IT Internet component consists of 14 modules based on behavioral activation, cognitive-behavioral therapy,30,49 interpersonal psychotherapy,31,50 and a community resiliency concept model, with the overarching goal of providing a sense of mastery over emotions in a range of domains (eg, peer, family, school). Acknowledging that risk factors occur within ecological contexts and across multiple domains, a workbook supporting the development of resiliency in adolescence was provided to the parents of adolescents.51 The development and content of the intervention have been extensively described in prior publications.27,28

Recruitment. Recruitment was accomplished by screening all adolescents visiting the primary care physician for risk of depressive disorder (presence of at least 1 core symptom of depressive disorder for at least 2 weeks)52 as well as through advertisements posted in and around the clinics. Adolescents (aged 14–21 years) who reported any core symptoms of depressive disorder for 2 weeks at 2 separate assessment points were enrolled in the study. Adolescents with borderline symptoms of major depression and directly supervised by their primary care provider were also enrolled in the study. Exclusion criteria were applied to adolescents with major depression, with frequent self-harm thoughts/intent, or with other mental disorders (not including alcohol abuse). Study staff contacted the adolescents by telephone and conducted a full eligibility assessment. Qualified assessors (blinded to group assignment) completed postintervention assessments of adolescents’ clinical status.53

Training of providers. All providers (primary care physicians and nurses/medical assistants) were trained during two 1-hour sessions (lunchtime) at baseline and subsequently after the completion of the intervention by all participants as a booster session. The sessions included 40 minutes of didactic material on the screening, study design, and interview components. This 40-minute didactic session was followed by a 20-minute video demonstration of the brief advice and motivational interview approaches using the scripts provided in the study. This session also included a postvideo discussion. Physicians were provided with a copy of the video as well as a script (1 page) on the day of the interview. Similarly, adolescents receiving the motivational interview were provided a preinterview goal-setting questionnaire to prepare for the physician interview that included the motivational interview questions. Nurses/medical assistants were also provided with a short description script on how to introduce the study to the participants. All interviews between the physicians and the participants were audio recorded. Nurses/medical assistants received $100 for completing training, assisting in the study, and completing a poststudy questionnaire. The physicians received $100 for training and a poststudy questionnaire and $50 for each interview they completed with the participant. All physician providers provided consent for study participation and audio recording of the interviews.

Measures

Survey design and collection of recorded interviews. All health care providers implementing CATCH-IT in their offices were eligible to complete a survey following completion of the program by all participants. The study was administered in 2 phases. In the first phase, health care providers completed a questionnaire after implementing the program in order to assess their attitudes and ability for implementing the online program. The second phase included rating physician skills for executing the 2 interviewing methods of brief advice and motivational interview. All interviews were transcribed and rated by 3 independent reviewers.

Provider characteristics. All providers and office staff completed questionnaires characterizing their professional backgrounds (eg, medical assistants, nurses, and physicians, including specialty and board certification status), practice ownership models (eg, community or hospital not for profit, group practice physician owned, medical school), practice size (intermediate practices [5–10 providers]), small practices [≤ 4 providers]) and large practices (> 10 providers), location (rural, urban, suburban), race/ethnicity, gender, and age.

Adolescent outcome measure. All adolescents completed self-reported questionnaires at baseline and 6- and 12-weeks’ postenrollment. Trained assessors evaluated depressed mood at follow-up at 6- and 12-weeks’ postenrollment. We report outcomes as derived from the CESD-10.54 Cronbach α for the CESD-10 was 0.83 at baseline. We used CESD-10 scores measured at baseline and 12-week follow-up to calculate
pre/post effect sizes for each clinic.

Provider knowledge and attitudes. The survey was organized into 2 categories for evaluation: (1) knowledge (true/false response, eg, “Even 1 episode of major depression can raise a patient’s risk of substance abuse or suicide.”) and (2) attitudes (Likert scale ratings: 1 = strongly disagree to 5 = strongly agree) in several areas that include rating scales for the CATCH-IT intervention, such as feasibility screening (eg, “The screening process was easy to use.”), interview (separate scales for each type of interview, eg, “The script was easy to use.”), and Internet (eg, “The modules were easy to read for my patients.”); barriers (eg, “The screening is difficult for the nursing staff.”); willingness (eg, “I believe that primary care physicians should recognize symptoms of depression.”); capability (eg, I can recognize the symptoms of major depression.”); global appraisal (eg, “My patients benefited from the early intervention process.”); and sustainability (eg, “I would implement the entire program in my office, even if I were not enrolled in the study.”), and primary care interventions for depression in general. The items comprising the knowledge and attitudes can be found in Table 1.

Motivational interview performance. In order to assess clinicians’ motivational interview performance, we also implemented the Motivational Interview Treatment Integrity 3 (MITI3) coding manual using 2 trained raters.55,56 The MITI3 divides the assessment into 2 main parts: global spirit score and behavior counts.

The global spirit score captures raters’ judgment on the relationship between the patient and the physician on the key motivational interview elements of evocation (eliciting motivation), collaboration (power sharing), autonomy (supporting patient perception of choice), direction (focus on behavior change), and empathy (understanding). Each category is rated on a Likert-type scale (1 = low to 5 = high), for which the coder assumes a beginning score of “3” (average knowledge and proficiency in motivational interview performance), moving up for higher knowledge and proficiency or down for lower knowledge and proficiency in motivational interview performance.

Behavior counts measure specific behaviors determined by categorization and decision rules, avoiding having to rely on the coder’s judgment. This component of the MITI3 includes open-ended versus closed-ended questions, simple versus complex reflections, and an adherence versus nonadherence to motivational interview approach. We report these counts as both the absolute mean value and the percentage of the desired behavior from the total in that category, eg, ratio of reflections to questions (simple + complex reflections)/(open-ended questions + closed-ended questions), percent of open-ended questions (number of open-ended questions/(open-ended questions + closed-ended questions), percent of complex reflections (complex reflections/complex + simple reflections), and percent of motivational interview adherent behaviors (total motivational interview adherent behaviors/motivational interview adherent behaviors + motivational interview nonadherent behaviors).

Beginners’ level of competence. The score thresholds for this part of the MITI3 vary depending on the nature of the behavior to achieve beginners’ level of performance. Motivational interview performance is rated at a beginners’ level of competence if global spirit score > 5.0, reflections to questions ratio ≥ 1.0, percent of open-ended questions ≥ 50%, percent of reflections that are complex ≥ 40%, and behaviors categorized as motivational interview adherent > 90%. Further information about MITI3 score thresholds have been previously published.58 Two raters, trained in the method by a certified motivational interview trainer and rater, independently rated all of the tapes and then reconciled any differences by consensus.58

Provider comments. Comments were collected during the study and also at poststudy meetings with providers. They were asked to provide general comments about the study and intervention as well as any barriers. The site principal investigators provided a summary consensus.

Statistical Analysis

We present descriptive statistics and compare mean differences between physicians and nurses/medical assistants with analysis of variance. We also conducted exploratory bivariate linear regression analysis to examine the relationship between predictors (eg, practice structure/context, provider attitudes, and provider motivational interview performance) on RE-AIM model outcomes. Stata 11 (StataCorp LP, College Station, Texas) was used to conduct the analysis.

RE-AIM Public Health Impact Analysis

RE-AIM is a 5-dimension model based on the factors of reach, efficacy, adoption, implementation, and maintenance.47 Reach was calculated as the proportion of at-risk adolescents identified in each primary care setting. This calculation was determined by dividing the number who screened positive at the clinic by the number of “probable” positive screens based on the performance of 1 “gold standard” clinic that was thought to have achieved nearly complete implementation of screening. The specific reach formula used was as follows: number of months in study × number of physicians × specialty factor (eg, pediatrics = 1 vs family medicine or medicine pediatrics = 1/3 because of smaller number of adolescent visits) × observed proportion of positive screens per physician/month at a defined gold standard clinic. Efficacy was calculated as the mean effect size of the CESD-10 score change from baseline to 12 weeks for adolescents enrolled from that clinic. Adoption was calculated as the proportion of physicians approached willing to participate. Implementation was calculated as the proportion of adolescents completing the intervention. Maintenance was calculated as the proportion of primary care sites agreeing to continue the intervention following the pilot program. Finally, the factors were multiplied to calculate the public health impact (reach × efficacy × adoption ×
implementation × maintenance = public health impact).

RESULTS

Sample Characteristics

Provider characteristics. Participants included 63 nurses/medical assistants and physicians, of which 36 were identified as nurses/medical assistants (n = 23/36, 64% response) and 27 as primary care physicians (n = 19/27, 70% response). The responding nurses/medical assistants (mean age = 38.3 years, SD = 12.8, 30.4% [n = 7/23] ethnic minority) comprised the following: 60.9% (14/23) medical assistant, 26.1% (6/23) registered nurse, 8.7% (2/23) licensed practical nurse, and 4.4% (1/23) “other.” Of the responding physicians (mean age = 42.6 years, SD = 11.8, 38% [n = 7/19] ethnic minority), 57.9% (11/19) were board certified in pediatrics and 42.1% (8/19) were board certified in internal medicine. Practice ownership models included 57.9% (11/19) community or hospital not for profit, 15.8% (3/19) group practice physician owned, and 26.3% (5/19) medical school. With regard to practice size, 44.4% (8/18) of physicians reported that they were in intermediate practices (5–10 providers), 38.9% (7/18) in small practices (≤ 4 providers), and 16.7% (3/18) in large practices (> 10 providers). With regard to setting, 52.6% (10/19) reported urban location, 21.1% (4/19) suburban location, and 26.3% (5/19) rural location. Representation of nurses and medical assistants in terms of practice size and location was similar to that of the physicians (data not shown).

Adolescent characteristics. The participants included 83 individuals with a mean age of 17.5 years (SD = 2.04). More than half were female (n = 47, 56.2%), with whites comprising the majority of the sample (n = 49, 59.0%), followed by blacks (n = 19, 22.9%), Asians (n = 5, 6.0%), Hispanics (n = 4, 5.0%), other (n = 3, 3.6%), and 3 participants (n = 3, 3.6%) who chose not to disclose their race/ethnicity. More than half (n = 56/81, 69.1%) lived at home with both parents. An almost equal number of adolescents reported parents being divorced (n = 16/79, 20.3%) or never married (n = 17/79, 21.5%). The majority came from families in which both parents had completed college (n = 35/74, 47.3% of fathers and n = 30/76, 39.5% of mothers). Most reported having completed at least 2 years of high school (n = 43/74, 58.1%), with 20 (n = 20/74, 27.0%) having completed at least 2 years of college. There was a mean baseline CESD-10 score of 24.4 (SD = 12.6), with the majority reporting no current depressive disorders (n = 64/83, 77.1%), although 18 (n = 18/83, 21.7%) reported having received prior counseling, 13 (n = 13/83, 15.7%) reported having been treated with medication, and 23 (n = 23/83, 27.7%) reported a family history of depression. Almost half (n = 37/83, 44.6%) reported at least 1 core symptom of generalized anxiety disorder and almost half (n = 40/83, 48.2%) reported less than 1 core symptom for panic disorder. Almost one-fifth (n = 15/83, 18.1%) disclosed at least 1 core symptom to meet eligibility for substance abuse.

Provider Attitudes and Comments

Both types of providers generally rated intervention components in the neutral/agree range (Table 2). This pattern was seen for screening, the motivational interview, the brief advice interview, the Internet component, global appraisal screening, and sustainability. There were some variations from this general pattern. Barriers and capability had ratings that were in the “disagree/neutral” range. For willingness, ratings were in the “strongly disagree/disagree” range. Nurses/medical assistants had significantly higher ratings than physicians only for capability (P value = .02). With regard to comments, providers expressed interest and enthusiasm for a preventive model that might address the issue of depression, a presentation often seen in primary care settings. Providers identified several concerns as follows: (1) lack of local mental health specialists to treat cases of major depression identified by screening, (2) difficulty implementing screening within a busy practice schedule, (3) difficulty incorporating the shared decision-making style of motivational interviewing into their counseling approach, and (4) limited practice time to manage follow-ups with other competing priorities (data not shown).

Knowledge and Performance

The mean percent correct for knowledge of the intervention components was 39.13% for nurses/medical assistants and 63.16% for physicians (Table 3). Most physicians scored above the cutoff for demonstrating beginners’ level of competency on the direction and empathy scales (mean = 4.69, SD = 0.54 and mean = 4.03, SD = 0.91, respectively). In terms of demonstrating behavior that exemplifies appropriate use of motivational interview, physicians demonstrated at least 1 behavior that was motivational interview adherent and 1 behavior that was motivational interview nonadherent in each interview (mean = 1.57, SD = 1.81 and mean = 1.25, SD = 1.10, respectively). Physicians employed more open-ended questions during interviews (mean = 7.00, SD = 4.21) than closed-ended questions (mean = 3.71, SD = 2.15). Physicians averaged approximately one and a half complex reflection questions (mean = 1.51, SD = 1.31) and 2 simple reflection questions (mean = 2.14, SD = 1.48). In terms of achieving standards of beginners’ competency, there was excellent competency of 100% for “> 50% open-ended questions” and strong (64.29%) competency for “percent of interviews with > 40% complex reflections.” However there was weaker competency for the other areas of the global spirit score, such as the “reflections to questions ratio > 1,” which was not reached in any of the interviews performed. Similarly, the percent of motivational interview adherent behaviors > 90% was achieved by only 4 of the 28 physicians who completed such interviews.

Practice Structure and Setting and RE-AIM Outcomes

RE-AIM analysis revealed a range of values for the different practice locations for total public health impact ranging from 0.00 to 0.59 (Table 4). The adoption statistic ranged from 0.50 to 1.00. Reach (proportion of adolescents at risk actually identified) varied from 0.30 at a federally qualified community health center to 1.00 at a suburban, multispecialty group. Efficacy (CESD-10 score change from baseline sustained to 3 months) varied from 0.19 at an urban physician hospital organization to 1.86 at a health maintenance organization model clinic. Implementation ranged from 0.00 in 2 Midwest, urban, physician hospital organizations to 1.00 at a Midwest, suburban, multispecialty physician group. With regard to maintenance, overall, 85% of all primary care sites agreed to continue the intervention. Interestingly, a federally qualified health center (practice F) had the third greatest efficacy statistic at 3 months (1.18), as well as 100% adoption, very high implementation, and 100% maintenance values, but due to lower reach values, had an overall public health impact of 0.26, only the fifth highest value of the participating clinics.

Predictors of RE-AIM Outcomes

The exploratory regression analyses identified several significant predictors in terms of both practice context and structure, as well as in provider attitudes (Table 5). With regard to practice context and structure, lower median household income was significantly associated with higher efficacy and higher implementation, while higher median household income was significantly associated with higher maintenance and higher public health impact. Also, a lower percentage of ethnic minority patient participants was significantly associated with higher adoption and higher reach. With regard to provider attitudes, feasibility screening was significantly associated with higher maintenance and higher public health impact. Also, feasibility of the Internet component of the intervention was significantly associated with higher efficacy and higher implementation. The provider knowledge performance measure was not significantly associated with any RE-AIM outcome.

DISCUSSION

Providers were able to implement the intervention with adequate competence and with potential for public health impact. Providers had neutral/agree ratings for many of the attitude measures of screening, the motivational interview, the brief advice interview, the Internet component, global appraisal screening, and sustainability. This finding lends partial support to our first assumption stating that providers would have favorable attitudes toward the CATCH-IT intervention even if they held less favorable ones toward conventional mental health interventions. The motivational interview performance scores marginally support our second assumption that physicians could gain basic minimal proficiency in motivational interviewing after just a 1-hour instruction session during lunch, as we found that basic minimal competence in the intervention could be obtained in 1 brief training session for open-ended questions and complex reflections but not for other components of the global spirit score (reflections to questions ratio > 1 and motivational interview adherent behaviors > 90%). Our last assumption that clinical practices might vary considerably in their ability to perform the intervention and that this variability would affect the potential public health impact was supported by evidence of positive impact. Also, we found that certain aspects of both practice context/structure and provider attitudes may influence public health impact.

The finding that primary care physicians and nurses/medical assistants find an Internet depression prevention intervention model valuable and sustainable is a new finding. Despite studies and recommendations for primary care–based studies of Internet-based interventions for depression, little is known about how primary care physicians perceive them.20,22 However, the relative indifference with regard to feasibility and the slightly negative approach to barriers are consistent with reports of primary care physician discomfort diagnosing and treating major depression among adolescents.42,43,58,59 Also, concerns about challenges in treating depressive disorders and difficulties in implementing components are consistent with prior qualitative studies of depression in primary care.43

The challenge of using brief training methods for primary care physicians to attain sufficient competence to practice a complex, if short, psychosocial intervention is apparent. Most reports focus on training mental health providers in motivational interviewing techniques using much longer training programs than our training session.60 Achieving some level of competence by primary care physicians (most aptly described as “novice”) in motivational interviewing using a brief session of only 60 minutes has not been reported. We found, similar to previous studies, that physicians can be trained in motivational interviewing techniques.61 However, lack of formal training in counseling approaches for primary care physicians62 may relate to a number of areas for which we did not find high levels of competency. Studies of training psychologists in motivational interview show that optimal performance may be achieved after several days of training over several months.60 Similarly, the brevity and simplicity of the training to implement this intervention stand in contrast to those required to implement complex interventions for depression using collaborative44 or integrated care models.63

Application of the RE-AIM framework to a primary care–based intervention for depression is a new approach. Our findings are consistent with prior RE-AIM studies that demonstrate moderate uptake by primary care practices and schools for physical health promotion interventions and complement the case-based approaches used for depressive illness interventions in primary care.41,64,65 With regard to our exploratory analyses, our findings are consistent with prior empirical and theoretical work implicating aspects of practice context/structure and provider attitudes as promoting care for depressive illness or conversely serving as barriers.41,43,63 However, this relationship has not been demonstrated previously for a preventive intervention for mental disorders in a primary care setting. Further, our findings for the association of patient median household income and race/ethnicity in the assessment of public health impact have not been previously examined.

The major strength of this study is the many factors that influence implementation of a novel preventive intervention in a wide range of clinics, including suburban practices and community health centers across the Southeastern and Midwestern United States. There are several limitations. First, selection bias (physicians who are more favorably disposed to the intervention enrolled in the study) and response bias (favorable responses to please the investigator) may have resulted in more positive appraisals for intervention implementation than would be found in a representative sample of providers. Second, the RE-AIM model is limited in that it assumes that each of the 5 factors interact multiplicatively and that all 5 dimensions are of equal importance and given equal weight.47 Third, the regression models are limited by the small sample size and make our conclusions exploratory. Fourth, we did not include any measure of the presence or absence of the medical home model at the sites.26

Future Directions

We are currently conducting a 5-year, multisite clinical trial (supported by the National Institutes of Health) implemented at 11 primary care practices in 4 major US health systems. This active study seeks to include a representative sample of adolescents and their parents/legal guardians and primary care providers (physicians, nurses, medical assistants, and administrative staff). Preliminary results suggest implementation of an Internet depression prevention program in primary care settings continues to be a complex process. Barriers such as provider attitudes, intervention complexity, and practice context/structure continue to be prevalent in the most practices. Among providers’ attitudes, screening for depression continues to be a major barrier, especially in practices that serve adolescents from hard-to-reach populations. These practices have also noted factors such as scheduling for motivational interviews and time to accomplish screening to be major impediments, primarily due to a heavy work flow. Regardless of the practice context/structure, common elements such as Health Insurance Portability and Accountability Act compliance requirements and insurance compensation are major challenges in implementing an Internet depression prevention program.

In conclusion, a relatively simple training and implementation approach can successfully introduce a detailed Internet-based preventive intervention for depression in primary care that demonstrates the potential for significant public health impact. By shifting the focus to the patient (Internet-based self-management) and minimizing changes in practice work flow, this approach may represent a “reengineering” around barriers to implementation of interventions for depressive disorders in primary care. Such a model could be an important addition to the patient-centered medical home/accountable care organization, which depends on patient empowerment and use of lower-cost approaches to modify and preempt disease course.26 This study represents the first step in adding a new technology-based armamentarium to the therapeutic kit of the primary care physician and nurse to create a feasible and cost-effective model for the prevention of mental disorders in adolescents in primary care settings.

Author affiliations: Department of Pediatrics, University of Illinois at Chicago, Chicago, Illinois (Ms Marko-Holguin and Dr Van Voorhees); Department of Psychiatry, Harvard Medical School, Cambridge Health Alliance, Cambridge, Massachusetts (Dr Eisen); Department of Finance and Business Management, Brooklyn College of the City University of New York, Brooklyn (Dr Fogel); Department of Medicine, University of Chicago, Chicago, Illinois (Mr Cardenas and Dr Bahn); Department of Family Medicine, Anderson Area Medical Center, Anderson, South Carolina (Dr Bradford); Department of Family Medicine, University of North Carolina-Chapel Hill at the Mountain Area Health Education Center, Asheville, North Carolina (Dr Fagan); and Advocate Healthcare, Chicago, Illinois (Dr Wiedmann).

Potential conflicts of interest: Dr Van Voorhees has served as a consultant to Prevail Health Solutions, Mevident, Social Kinetics, and Hong Kong University to develop Internet-based interventions. To facilitate dissemination, the University of Chicago agreed to grant a no-cost license to Mevident Incorporated (3/5/2010) to develop a school-based version. Neither Dr Van Voorhees nor the university will receive any royalties or equity. Dr Van Voorhees has agreed to assist the company in adapting the intervention at the rate of $1,000/day for 5.5 days. Drs Eisen, Fogel, Bahn, Bradford, Fagan, and Wiedmann and Ms Marko-Holguin and Mr Cardenas report no conflicts of interest related to the subject of this article.

Funding/support: Supported by a NARSAD Young Investigator Award, Robert Wood Johnson Foundation Depression in Primary Care Value Grant, and a career development award from the National Institute of Mental Health (NIMH K-08 MH 072918-01A2).

Previous presentation: Presented at the 2nd Annual National Institutes of Health Conference on the Science of Dissemination and Implementation; January 29, 2009; Bethesda, Maryland.

Role of sponsor: The funding organization had no role in the design of the study; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the paper for publication.

REFERENCES

1. Reinherz HZ, Giaconia RM, Hauf AM, et al. General and specific childhood risk factors for depression and drug disorders by early adulthood. J Am Acad Child Adolesc Psychiatry. 2000;39(2):223–231. doi:10.1097/00004583-200002000-00023 PubMed

2. Horwitz AV, White HR. Becoming married, depression, and alcohol problems among young adults. J Health Soc Behav. 1991;32(3):221–237. doi:10.2307/2136805 PubMed

3. Ernst C, Földényi M, Angst J. The Zurich Study: XXI. Sexual dysfunctions and disturbances in young adults: data of a longitudinal epidemiological study. Eur Arch Psychiatry Clin Neurosci. 1993;243(3–4):179–188. doi:10.1007/BF02190725 PubMed

4. Binder J, Angst J. Social consequences of psychic disturbances in the population: a field study on young adults (author’s transl) [Article in German]. Arch Psychiatr Nervenkr. 1981;229(4):355–370. doi:10.1007/BF01833163 PubMed

5. Skodol AE, Schwartz S, Dohrenwend BP, et al. Minor depression in a cohort of young adults in Israel. Arch Gen Psychiatry. 1994;51(7):542–551. doi:10.1001/archpsyc.1994.03950070034008 PubMed

6. Runeson B. Mental disorder in youth suicide: DSM-III-R Axes I and II. Acta Psychiatr Scand. 1989;79(5):490–497. doi:10.1111/j.1600-0447.1989.tb10292.x PubMed

7. Breslau N, Kilbey MM, Andreski P. DSM-III-R nicotine dependence in young adults: prevalence, correlates and associated psychiatric disorders. Addiction. 1994;89(6):743–754. doi:10.1111/j.1360-0443.1994.tb00960.x PubMed

8. Christie KA, Burke JD Jr, Regier DA, et al. Epidemiologic evidence for early onset of mental disorders and higher risk of drug abuse in young adults. Am J Psychiatry. 1988;145(8):971–975. PubMed

9. Kessler RC, Walters EE. Epidemiology of DSM-III-R major depression and minor depression among adolescents and young adults in the National Comorbidity Survey. Depress Anxiety. 1998;7(1):3–14. doi:10.1002/(SICI)1520-6394(1998)7:1<3::AID-DA2>3.0.CO;2-F PubMed

10. Nutting PA, Rost K, Dickinson M, et al. Barriers to initiating depression treatment in primary care practice. J Gen Intern Med. 2002;17(2):103–111. doi:10.1046/j.1525-1497.2002.10128.x PubMed

11. Gilbody S, Whitty P, Grimshaw J, et al. Educational and organizational interventions to improve the management of depression in primary care: a systematic review. JAMA. 2003;289(23):3145–3151. doi:10.1001/jama.289.23.3145 PubMed

12. Stein RE, Zitner LE, Jensen PS. Interventions for adolescent depression in primary care. Pediatrics. 2006;118(2):669–682. doi:10.1542/peds.2005-2086 PubMed

13. Clarke GN, Hornbrook M, Lynch F, et al. A randomized trial of a group cognitive intervention for preventing depression in adolescent offspring of depressed parents. Arch Gen Psychiatry. 2001;58(12):1127–1134. doi:10.1001/archpsyc.58.12.1127 PubMed

14. Garber J, Clarke GN, Weersing VR, et al. Prevention of depression in at-risk adolescents: a randomized controlled trial. JAMA. 2009;301(21):2215–2224. doi:10.1001/jama.2009.788 PubMed

15. Gillham JE, Shatte AJ, Freres DR. Preventing depression: a review of cognitive-behavioral and family interventions. Appl Prev Psychol. 2000;9(2):63–88. doi:10.1016/S0962-1849(00)80007-4

16. Van Voorhees BW, Cooper LA, Rost KM, et al. Primary care patients with depression are less accepting of treatment than those seen by mental health specialists. J Gen Intern Med. 2003;18(12):991–1000. doi:10.1111/j.1525-1497.2003.21060.x PubMed

17. Van Voorhees BW, Fogel J, Houston TK, et al. Attitudes and illness factors associated with low perceived need for depression treatment among young adults. Soc Psychiatry Psychiatr Epidemiol. 2006;41(9):746–754. doi:10.1007/s00127-006-0091-x PubMed

18. Kaltenthaler E, Shackley P, Stevens K, et al. A systematic review and economic evaluation of computerised cognitive behaviour therapy for depression and anxiety. Health Technol Assess. 2002;6(22):1–89. PubMed

19. Spek V, Cuijpers P, Nyklícek I, et al. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37(3):319–328. doi:10.1017/S0033291706008944 PubMed

20. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the Internet: randomized controlled trial. BMJ. 2004;328(7434):265. doi:10.1136/bmj.37945.566632.EE PubMed

21. Cavanagh K, Shapiro DA, Van Den Berg S, et al. The effectiveness of computerized cognitive behavioral therapy in routine care. Br J Clin Psychol. 2006;45(pt 4):499–514. doi:10.1348/014466505X84782 PubMed

22. Tylee A. Identifying and managing depression in primary care in the United kingdom. J Clin Psychiatry. 2006;67(suppl 6):41–45. PubMed

23. Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res. 2002;4(1):e3. doi:10.2196/jmir.4.1.e3 PubMed

24. Santor DA, Poulin C, LeBlanc JC, et al. Online health promotion, early identification of difficulties, and help seeking in young people. J Am Acad Child Adolesc Psychiatry. 2007;46(1):50–59. doi:10.1097/01.chi.0000242247.45915.ee PubMed

25. Clarke GN. Adolescent Use of Web-Based Depression Treatment Programs. Portland, OR: Kaiser Permanente Center for Health Research; 2002.

26. Longworth DL. Accountable care organizations, the patient-centered medical home, and health care reform: what does it all mean? Cleve Clin J Med. 2011;78(9):571–582. doi:10.3949/ccjm.78gr.11003 PubMed

27. Landback J, Prochaska M, Ellis J, et al. From prototype to product: development of a primary care/Internet–based depression prevention intervention for adolescents (CATCH-IT). Community Ment Health J. 2009;45(5):349–354. doi:10.1007/s10597-009-9226-3 PubMed

28. Van Voorhees BW, Ellis JM, Gollan JK, et al. Development and process evaluation of a primary care Internet-based intervention to prevent depression in emerging adults. Prim Care Companion J Clin Psychiatry. 2007;9(5):346–355. doi:10.4088/PCC.v09n0503 PubMed

29. Bell CC. Cultivating resiliency in youth. J Adolesc Health. 2001;29(5):375–381. doi:10.1016/S1054-139X(01)00306-8 PubMed

30. Jacobson NS, Martell CR, Dimidjian S. Behavioral activation treatment for depression: returning to contextual roots. Clin Psychol Sci Pract. 2001;8(3):255–270. doi:10.1093/clipsy.8.3.255

31. Stuart S, Robertson M. Interpersonal Psychotherapy: A Clinicians Guide. New York, NY: Oxford University Press; 2003:315.

32. Merry S, McDowell H, Hetrick S, et al. Psychological and/or educational interventions for the prevention of depression in children and adolescents. Cochrane Database Syst Rev. 2004;1(1):CD003380. PubMed

33. Van Voorhees B. 6-Month outcomes of a randomized trial of a primary care–based adolescent depression prevention intervention. J Gen Intern Med. 2008;23(suppl 2):232.

34. Van Voorhees BW, Fogel J, Reinecke MA, et al. Randomized clinical trial of an Internet-based depression prevention program for adolescents (Project CATCH-IT) in primary care: 12-week outcomes. J Dev Behav Pediatr. 2009;30(1):23–37. doi:10.1097/DBP.0b013e3181966c2a PubMed

35. Van Voorhees BW, Walters AE, Prochaska M, et al. Reducing health disparities in depressive disorders outcomes between non-Hispanic whites and ethnic minorities: a call for pragmatic strategies over the life course. Med Care Res Rev. 2007;64(suppl 5):157S–194S. doi:10.1177/1077558707305424 PubMed

36. Zuckerbrot RA, Cheung AH, Jensen PS, et al; GLAD-PC Steering Group. Guidelines for Adolescent Depression in Primary Care (GLAD-PC), 1: identification, assessment, and initial management. Pediatrics. 2007;120(5):e1299–e1312. doi:10.1542/peds.2007-1144 PubMed

37. Zuckerbrot RA, Maxon L, Pagar D, et al. Adolescent depression screening in primary care: feasibility and acceptability. Pediatrics. 2007;119(1):101–108. doi:10.1542/peds.2005-2965 PubMed

38. Asarnow JR, Jaycox LH, Duan N, et al. Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: a randomized controlled trial. JAMA. 2005;293(3):311–319. doi:10.1001/jama.293.3.311 PubMed

39. Gask L, Dowrick C, Dixon C, et al. A pragmatic cluster randomized controlled trial of an educational intervention for GPs in the assessment and management of depression. Psychol Med. 2004;34(1):63–72. doi:10.1017/S0033291703001065 PubMed

40. Asarnow JR, Jaycox LH, Anderson M. Depression among youth in primary care models for delivering mental health services. Child Adolesc Psychiatr Clin N Am. 2002;11(3):477–497, viii [viii.]. doi:10.1016/S1056-4993(02)00006-8 PubMed

41. Reiss-Brennan B, Briot P, Daumit G, et al. Evaluation of “depression in primary care” innovations. Adm Policy Ment Health. 2006;33(1):86–91. doi:10.1007/s10488-005-4239-x PubMed

42. Fremont WP, Nastasi R, Newman N, et al. Comfort level of pediatricians and family medicine physicians diagnosing and treating child and adolescent psychiatric disorders. Int J Psychiatry Med. 2008;38(2):153–168. doi:10.2190/PM.38.2.c PubMed

43. Richardson LP, Lewis CW, Casey-Goldstein M, et al. Pediatric primary care providers and adolescent depression: a qualitative study of barriers to treatment and the effect of the black box warning. J Adolesc Health. 2007;40(5):433–439. doi:10.1016/j.jadohealth.2006.12.006 PubMed

44. Fortney JC, Pyne JM, Smith JL, et al. Steps for implementing collaborative care programs for depression. Popul Health Manag. 2009;12(2):69–79. doi:10.1089/pop.2008.0023 PubMed

45. Gilbody S, Bower P, Whitty P. Costs and consequences of enhanced primary care for depression: systematic review of randomized economic evaluations. Br J Psychiatry. 2006;189(4):297–308. doi:10.1192/bjp.bp.105.016006 PubMed

46. Watkins C, Timm A, Gooberman-Hill R, et al. Factors affecting feasibility and acceptability of a practice-based educational intervention to support evidence-based prescribing: a qualitative study. Fam Pract. 2004;21(6):661–669. doi:10.1093/fampra/cmh614 PubMed

47. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. doi:10.2105/AJPH.89.9.1322 PubMed

48. Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41(11):1284–1292. doi:10.1097/01.MLR.0000093487.78664.3C PubMed

49. Clarke GN. The Coping With Stress Course Adolescent: Workbook. Portland, OR: Kaiser Permanente Center for Health Research; 1994.

50. Mufson L, et al. Interpersonal Psychotherapy for Depressed Adolescents. New York, NY: Guilford Press; 2004.

51. Beardslee WR, Gladstone TR, Wright EJ, et al. A family-based approach to the prevention of depressive symptoms in children at risk: evidence of parental and child change. Pediatrics. 2003;112(2):e119–e131. doi:10.1542/peds.112.2.e119 PubMed

52. Johnson JG, Harris ES, Spitzer RL, et al. The Patient Health Questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. 2002;30(3):196–204. doi:10.1016/S1054-139X(01)00333-0 PubMed

53. Van Voorhees BW, Fogel J, Pomper BE, et al. Adolescent dose and ratings of an Internet-based depression prevention program: a randomized trial of primary care physician brief advice versus a motivational interview. J Cogn Behav Psychother. 2009;9(1):1–19. PubMed

54. Radloff LS. The use of the Center for Epidemiologic Studies Depression Scale in adolescents and young adults. J Youth Adolesc. 1991;20(2):149–166. doi:10.1007/BF01537606

55. Pierson HM, Hayes SC, Gifford EV, et al. An examination of the Motivational Interviewing Treatment Integrity code. J Subst Abuse Treat. 2007;32(1):11–17. doi:10.1016/j.jsat.2006.07.001 PubMed

56. Moyers TB, Martin T, Manuel JK, et al. Assessing competence in the use of motivational interviewing. J Subst Abuse Treat. 2005;28(1):19–26. doi:10.1016/j.jsat.2004.11.001 PubMed

57. Bennett GA, Roberts HA, Vaughan TE, et al. Evaluating a method of assessing competence in motivational interviewing: a study using simulated patients in the United Kingdom. Addict Behav. 2007;32(1):69–79. doi:10.1016/j.addbeh.2006.03.022 PubMed

58. Liu SI, Lu RB, Lee MB. Non-psychiatric physicians’ knowledge, attitudes and behavior toward depression. J Formos Med Assoc. 2008;107(12):921–931. doi:10.1016/S0929-6646(09)60015-2 PubMed

59. Richards JC, Ryan P, McCabe MP, et al. Barriers to the effective management of depression in general practice. Aust N Z J Psychiatry. 2004;38(10):795–803. doi:10.1080/j.1440-1614.2004.01464.x PubMed

60. Schoener EP, Madeja CL, Henderson MJ, et al. Effects of motivational interviewing training on mental health therapist behavior. Drug Alcohol Depend. 2006;82(3):269–275. doi:10.1016/j.drugalcdep.2005.10.003 PubMed

61. Erol S, Erdogan S. Application of a stage-based motivational interviewing approach to adolescent smoking cessation: the Transtheoretical Model-based study. Patient Educ Couns. 2008;72(1):42–48. doi:10.1016/j.pec.2008.01.011 PubMed

62. Emmons KM, Rollnick S. Motivational interviewing in health care settings: opportunities and limitations. Am J Prev Med. 2001;20(1):68–74. doi:10.1016/S0749-3797(00)00254-3 PubMed

63. Kilbourne AM, Irmiter C, Capobianco J, et al. Improving integrated general medical and mental health services in community-based practices. Adm Policy Ment Health. 2008;35(5):337–345. doi:10.1007/s10488-008-0177-8 PubMed

64. Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help? J Am Med Inform Assoc. 2009;16(6):889–897. doi:10.1197/jamia.M3085 PubMed

65. Eakin EG, Brown WJ, Marshall AL, et al. Physical activity promotion in primary care: bridging the gap between research and practice. Am J Prev Med. 2004;27(4):297–303. PubMed