Computer-Assisted Cognitive-Behavior Therapy for Depression: A Systematic Review and Meta-Analysis

Computer-Assisted Cognitive-Behavior Therapy for Depression:

A Systematic Review and Meta-Analysis

Computer-assisted cognitive-behavior therapy (CCBT) was developed so evidence-based psychotherapy could be delivered to larger numbers of people and broader populations than may be possible with traditional face-to-face therapy.1-10 Although standard cognitive-behavior therapy (CBT) has been shown to be an effective treatment for depression in many studies,11-13 there are significant barriers to providing CBT for many people who could benefit from treatment. One of the problems that has limited broader dissemination of CBT is an insufficient number of trained therapists.8 Other constraints may include the costs of treatment, the usual requirement for 12-20 hours of therapist time for standard CBT,11-13 lack of access to therapy in rural areas or in other underserved populations, and reticence of people with depression to seek out and accept treatment in clinical settings.1-4,14,15

From the beginning of research on CCBT for depression in the late 1980s, reducing cost and improving availability of treatment have been overarching goals.1-10,16 Other potential contributions of CCBT may be the ability to deliver a consistent therapeutic program on the basics of CBT to each user, enhancement of the therapy experience by offering multimedia learning opportunities, access to therapy at any time or location users may choose, use of interactive learning exercises that build CBT skills, provision of regular feedback to users, promotion of CBT homework completion, and data recording and management.1,3,5,17

The majority of studies of CCBT have employed hybrid treatment delivery methods that combine computerized delivery of skill building modules for CBT with clinician support (usually in the range of 1-5 hours).3,18-39 A smaller number of investigations have used CCBT as a stand-alone treatment method with no clinician support40-53 or have not provided information on involvement of therapists or other helping persons in treatment.54-56

Previous meta-analyses of CCBT5-8,10 have found evidence for the efficacy of CCBT. However, these reports have not included many recent trials assessed in the current analysis and often have utilized studies with mixed diagnostic groups. With a greater number of studies now available, it is now possible to more fully assess the importance of human support and other potential moderators of outcomes in CCBT. The aims of the present meta-analysis are to assess the efficacy of CCBT for depression in randomized controlled trials, examine the association between clinician or other helping person involvement in CCBT and outcome, explore other possible treatment moderators (eg, completion rate, study setting, pretreatment severity), and evaluate progress and limitations in CCBT research that may influence adoption of CCBT to address treatment delivery problems.

METHODS

Selection of Randomized Controlled Trials

A computerized search for studies meeting the criteria below was conducted using Ovid MEDLINE, PsycINFO, PubMed, and Scopus from their beginnings to July 18, 2016. The computerized search used the keywords “randomized, controlled trials of computer-assisted cognitive-behavior therapy for depression” and “randomized, controlled trials of mobile apps for cognitive-behavior therapy of depression.” A manual search using other meta-analyses and published reports of CCBT5-10 also was performed.

The criteria for including studies in the meta-analysis were as follows: (1) the study was a randomized controlled trial (RCT) with control group (ie, wait list, attention control, or treatment as usual) other than standard face-to-face CBT; (2) subjects were depressed as measured by depression rating scales; (3) inclusion criteria specified for depression (ie, clinical diagnosis of depression, diagnosis with standardized assessment [eg, DSM-IV,57 SCID,58 MINI59; see Table 1 for listing and full names of all diagnostic instruments and measures], or assessment with validated measure for depressive symptoms and appropriate cutoff score [eg, PHQ-9,60 BDI,61 HDRS,62 CES-D63); (4) participants were 16 years of age or older; (5) the study involved use of a computer program or mobile app that covers core methods of CBT to deliver all or part of the treatment; and (6) the study reported posttreatment mean scores with standard deviation using a psychometrically valid depression rating scale (eg, PHQ-9, BDI, HDRS, CES-D).

• Computer-assisted cognitive-behavior therapy (CCBT) offers clinicians the opportunity to leverage their time with patients. By using technology to increase the efficiency of their work, they can deliver evidence-based care to more patients than would be possible with standard cognitive-behavior therapy (CBT).
• Because CCBT programs are designed to offer interactive learning experiences, clinicians can use these programs to help patients build CBT skills for depression.
• By promoting self-help between sessions, CCBT has potential for augmenting the learning process and encouraging application of CBT principles in daily life.

Data Extraction

Data were extracted independently by 2 authors (J.H.W., D.R.) and then finalized by discussion with an additional author (M.E.T.) leading to consensus. Extracted data included number of subjects; subject recruitment method (eg, clinical population with advertisements, online only, nonclinical population with advertisements/announcements); type of control group (eg, wait list, attention control, psychoeducational website, treatment as usual); control of other treatments (eg, not controlled, drug-free, no other CBT or psychotherapy allowed); inclusion criteria for depression (eg, DSM-IV diagnosis on SCID58 or other clinical interview, cutoff score on self-report depression scale); type of CCBT program (eg, text, multimedia); pre- and posttreatment means and standard deviations on standard depression rating scale; time spent by clinician or other helping person in treatment/support of CCBT; type of support provided (eg, face-to-face, e-mail or other online asynchronous communication, telephone); treatment completion rate; treatment setting (primary care or non-primary care); and weeks of follow-up data after study completion. When data were not available (eg, pre- and posttreatment means and standard deviations on standard depression rating scale, time spent by clinician or other helping person, intent-to-treat data), corresponding authors of studies were contacted to request these data.

Assessment of Study Integrity/Bias

To assess study quality, we employed the CLEAR NPT,69 a checklist developed to evaluate reports of nonpharmacological, randomized clinical trials. The CLEAR NPT contains 10 questions, most of which are answered “yes,” “no,” or “unclear.” Question items address the adequacy of the randomization process; how well details of the interventions were described; the degree of care provider experience or skill; whether and how well adherence to treatment protocols was measured; whether caretakers, participants undergoing treatments, and outcome evaluators were or could be blinded and provisions taken if not; the consistency across treatment conditions of follow-up assessments; and whether the intent-to-treat (ITT) principle was followed in conducting analyses. Two of the study authors (T.D.E. and G.K.B.) independently evaluated each study using the CLEAR NPT criteria. Ratings were subsequently compared, and differences were reconciled through discussion leading to consensus.

Statistical Methods

To determine the efficacy of CCBT versus control conditions, we used the DerSimonian-Laird method for the random effect models. We calculated the effect sizes with Hedge’s g,70 which is the difference in means at posttreatment or follow-up divided by the pooled standard deviation of both conditions as the estimate of variance. The primary measure of depression was used for these calculations. For some studies (see footnotes in Figure 1), there were multiple comparisons (eg, multiple versions of CCBT or methods of delivery of CCBT vs control conditions); thus, a total of 45 comparisons were made. For the follow-up analysis, we first examined the initial follow-up assessment time point, regardless of timing. The range of time elapsed before the first follow-up assessment was 4 weeks to 6 months, and the longest period of follow-up was 15 months. In addition, we aggregated all of the follow-up assessments per study in order to capture the overall effects of CCBT over time. We used random effects estimates to better generalize beyond the participants in these studies. The heterogeneity of the effects was examined with Q tests and I2 statistics. We also examined funnel plots to assess symmetry in the findings across studies, and we conducted trim-and-fill analyses to further examine the robustness of the findings.70

We tested several potential moderators: (a) supported CCBT versus self-guided CCBT; (b) completion rates; (c) primary care studies versus non-primary care studies; and (d) pretreatment severity. For the pretreatment severity moderator analysis, we converted depression rating scales scores to z-scores based on norming information for the various measures. The moderator analysis of pretreatment depressive symptom severity required use of a transformation because several different scales were used across the studies. As these highly correlated dependent measures are not usually markedly skewed at pretreatment, we transformed raw scores to z-scores instead of a logarithmic transformation.71

RESULTS

The electronic search yielded 208 publications; an additional 15 articles were identified from a manual search. Of these 223 publications, 183 were excluded (See PRISMA diagram in Supplementary Figure 1). Principal reasons for exclusion were not an RCT = 60; RCT with active therapy (eg, standard CBT or another psychotherapy) as only control = 9; participants were not depressed = 31; no inclusion criteria for depression = 11; mixed sample including stress and anxiety = 2; participants were less than 16 years of age = 13; did not use computer-delivered therapy = 11; did not use CBT = 4; posttreatment means/SDs with standard depression rating scale were not reported or could not be obtained via author contact = 11; no ITT analysis = 9; duplicate (follow-up or other report on principal study found elsewhere in search) = 22. A total of 40 reports were eligible for inclusion in the meta-analysis. Key characteristics of these studies are shown in Table 1.

A forest plot for the posttreatment effects and 95% confidence intervals, along with numerical effect sizes for each comparison, is displayed in Figure 1. The random effects weighted mean effect size for CCBT versus controls at posttreatment was g = 0.502 (SE = 0.057; 95% CI, 0.390 to 0.614; P < .001). This effect is moderate to large; however, there was significant heterogeneity in the effects (Q44 = 222.53, P < .001, I2 = 80.23). Examination of the funnel plot (see Appendix 1) of standard errors for posttreatment effects revealed inconsistencies, and the Egger test of asymmetry was significant (intercept = 3.50, SE = 0.615, P < .001). A Duval and Tweedie trim-and-fill analysis yielded 13 studies that could have affected the overall g by 0.16. The collective results of these bias tests suggest heterogeneity of studies and the possibility of clinically meaningful moderators of treatment effect.

Follow-Up Analysis

Data were reported for follow-up assessments for CCBT and control conditions (other than standard CBT or other specific active treatment) in 19 of 45 comparisons. The majority of follow-up assessments were performed in comparisons of unsupported CBT (13 of 18). Only 6 of 24 comparisons of supported CBT versus control conditions provided follow-up measures, most likely due to a preponderance of wait-list control groups in these studies. We first examined mean effects based on the initial follow-up assessment time point, regardless of the timing of the follow-up (see Supplementary Figure 2). The random-effects weighted mean effect size for CCBT versus controls at the first follow-up assessment points was g = 0.386 (SE = 0.091; 95% CI, 0.207 to 0.565; P < .001). There was significant heterogeneity in the effects (Q19 = 124.89, P < .001, I2 = 84.79). We also aggregated the follow-up assessment time points, and the results were similar. The random-effects weighted mean effect size for CCBT versus controls at the aggregated follow-up assessment points was g = 0.354 (SE = 0.084; 95% CI, 0.190 to 0.518; P < .001).

Moderators

Support. We tested whether studies with support from a clinician or other helping person differed from those without any significant support (Figure 2). Among the studies listed in Figure 2, a few provided such minimal support (ie, Gilbody et al50 reported a mean of less than 7 minutes of technical support via telephone for the entire course of treatment, and Montero-Mar×­n et al52 sent a total of 17 e-mail messages to 13 of 296 patients) that we considered them to be self-guided CCBT. Also, we found 3 studies21,24,39 that either compared clinician with technician support or used nonclinicians to provide support. These were included in the category of supported studies. The random-effects weighted mean effect size for clinician or other helping person supported studies was g = 0.673 (SE = 0.065; 95% CI, 0.546 to 0.801; P < .001). By contrast, CCBT studies that did not include support had a random-effects weighted mean effect size at posttreatment of g = 0.239 (SE = 0.063; 95% CI, 0.115 to 0.364; P < .001). Thus, studies that had clinician or other helping person support outperformed those without such support by a large margin. An effect size difference of 0.43 on the Hamilton Depression Rating Scale, for example, would correspond to about a 15% difference in response or remission rates.72

We also compared the type of clinician support: e-mail (without or with text messaging or other asynchronous online support) versus telephone (without or with e-mail, text messaging, or other asynchronous online support) versus in person (See Supplementary Figure 3). The random-effects weighted mean effect size at posttreatment was lowest for e-mail (without or with text messaging or other asynchronous online support) (n = 9; g = 0.562; SE = 0.107; 95% CI, 0.353 to 0.938; P < .001), intermediate for telephone (without or with e-mail, text messaging, or other asynchronous online support) (n = 9; g = 0.782; SE = 0.114; 95% CI, 0.558 to 1.006; P < .001), and largest when face-to-face person support was provided (n = 3; g = 0.833 SE = 0.207; 95% CI, 0.427 to 1.24; P < .001).

Completion rate. We separated completion rates into quartiles for simplicity and also calculated mean effect sizes for studies that did not report the percentage of patients who completed treatment (Figure 3). The lowest random-effects weighted mean effect size (g = 0.293; SE = 0.101; 95% CI, 0.095 to 0.492; P = .004) was found for investigations that did not report completion rate. Intermediate mean effect sizes were found for studies with less than 25% completion (g = 0.410; SE = 0.155; 95% CI, 0.107 to 0.714; P = .008) and for studies with 26%-50% completion rate at posttreatment (g = 0.451; SE = 0.122; 95% CI, 0.213 to 0.689; P < .001). For studies with 51%-75% completion rate, the random-effects weighted mean effect size at posttreatment was g = 0.653 (SE = 0.115; 95% CI, 0.428 to 0.878; P < .001). Lastly, the random effects weighted mean effect size for studies with 76%-100% completion rate at posttreatment was g = 0.818 (SE = 0.146; 95% CI, 0.532 to 1.103; P < .001). Thus, higher completion rates were associated with larger mean effect sizes.

Primary care. We tested whether the outcomes of studies of CCBT in primary care differed from the other studies. An earlier report73 had found a random-effects weighted mean effect size for CCBT for depression in primary care that appeared to be lower than reported effect sizes in broader populations. The random-effects weighted mean effect size for the primary care studies at posttreatment was g = 0.224 (SE = 0.123; 95% CI, −0.012 to 0.464; P = .068). In contrast, studies that were not conducted in primary care sites had a significantly higher random-effects weighted mean effect size at posttreatment of g = 0.565 (SE = 0.061; 95% CI, 0.445 to 0.685; P < .001). See Supplementary Figure 4 for a forest plot of these mean effect sizes.

Pretreatment severity. The random-effects weighted mean effect size for studies with higher pretreatment severity (z-scores from 0.50 to 1.0, indicating higher severity) had a mean posttreatment effect size of g = 0.865 (SE = 0.160; 95% CI, 0.551 to 1.180; P < .001). The effect sizes for studies with lower severe pretreatment severity scores (z-scores less than 0.50) were generally in the moderate range (g = 0.400 to 0.481). See Supplementary Figure 5 for a forest plot of these data.

Assessment of Study Quality

As summarized in Figure 4 and detailed in Supplementary Table 1 (a display of quality ratings on all questions for each of the 40 studies in the meta-analysis), the methodological quality ratings were good in general. Ninety percent or more followed adequate randomization processes, described the interventions in sufficient detail, and assessed adherence quantitatively. Consistent with our meta-analysis inclusion criteria, all data were analyzed according to the intent-to-treat principle. Eighty percent adhered to the same follow-up schedule across treatment conditions. Given the nature of the intervention and the reliance on self-report outcome measures, the posttreatment assessments often could not be blinded. Specifically, half either did not conceal treatment assignments such that investigators could not foresee them or did not provide enough information for this question to be evaluated. Several articles did not report whether the proportions of participants receiving other nonstudy treatments (eg, pharmacologic) were the same in each randomized group, and a few did not report the qualifications or training of the care providers for the study interventions. Also, we found that dropout rates among studies or between study conditions varied considerably. Despite these shortcomings, by systematically selecting a group of studies using randomization, minimal contact control conditions, and validated depression scales to measure outcomes, it seems likely that the results of the meta-analysis were not excessively biased by methodological flaws.

DISCUSSION

We confirmed that CCBT has antidepressant effects, as measured by larger mean reductions of scores on validated symptom measures than observed in control conditions. Overall, the magnitude of this effect for supported CCBT, as measured by Hedges g, is moderate to large and is comparable to the effects reported in meta-analyses74-76 of standard treatments of depression, including antidepressant pharmacotherapy and studies of individual psychotherapy. Our meta-analysis included substantially more studies than earlier meta-analyses5-10,77 and explored a variety of possible moderators of treatment outcome. However, results were largely consistent with previous reports except as noted in the discussion of effects of clinician support that follows. Because available methods of CCBT can substantially reduce clinician time, be more convenient than standard psychotherapy for patients, and offer significant cost savings,16,77-79 broader implementation of CCBT could have a significant, favorable impact on public health.

The effects of CCBT in our meta-analysis were strongly influenced by support from a clinician or other helping person. Although a few studies of unguided CCBT had moderate to large effects,23,24,45,53 these were in the minority (4 of 18 comparisons of unguided CCBT vs control conditions), and many investigations of unguided CCBT had very small effect sizes. We conclude that with current methods, CCBT usually is more effective when supported by a clinician or other helping person. It is not surprising that clinical support can improve the impact of CCBT for depression—a condition that is characterized by fatigue, diminished motivation, pessimism, and difficulties with concentration, memory, and effortful cognition. Other than 1 recent report,77 other meta-analyses6-8 also found that effects of CCBT are larger when supported by clinicians or other helping professionals. Ahern and coworkers77 noted that their meta-analysis of 29 studies of CCBT of depression may not have found differences between supported and unsupported CCBT because of the large heterogeneity of studies they categorized as supported (including trials with negligible amounts of support). In comparing our analysis with that of Ahern and coworkers,77 we noted that they analyzed only 6 investigations that were listed as providing no support, while our report included 18 such studies.

Most studies that offered support for CCBT utilized clinicians with experience in mental health treatment. However, a few explored the possibility that effective support can be delivered by technicians or trained volunteers.21,24,39 Their role has been described as providing encouragement, support, and feedback to users or helping participants to use the online intervention. In 1 study21 that directly compared clinician support with technician support, no statistically significant differences in effectiveness were found. However, more research is needed on provision of support by nonclinicians and the possibility that such methods could be implemented widely, thus influencing the future development of psychotherapy.

At this juncture, it appears that support often can be delivered effectively by telephone or e-mail and other asynchronous online methods. However, the results of our meta-analysis suggest that the mode of delivery can influence outcome. Of the studies reporting a significant amount of support (about 1 hour or more), most (18 of 21) used telephone and/or e-mail or other asynchronous online support with no face-to-face contact. The lowest mean effect sizes were observed in investigations that used only e-mail, text messages, or other online asynchronous support. When telephone or face-to-face support was used, the mean effect sizes were significantly larger. No studies used telemedicine for support—a method that may fit well with delivery of CCBT.

Although we extracted data on the amount of support time in minutes or hours, there was considerable variability in the level of precision of reports. Some studies recorded actual support time, while others noted only the amount of time planned for treatment support or a range of support time that may have been provided. Because of this common lack of precision and other sources of variability noted in this discussion, we did not attempt to assess the relationship between reported amounts of support (except to compare any meaningful support vs no significant support) and mean effect sizes.

Other variations that make it difficult to assess relationships between the amount or type of support and outcome include (1) differences in patient recruitment methods (eg, from mental health or primary care clinical practices, online, advertisements, or combinations of these) and (2) differences in baseline severity, illness complexity, and use of other treatments (eg, mildly depressed persons recruited solely via the Internet; more severely depressed patients who are more representative of those seen in typical clinical settings; drug-free samples in which the only treatments are CCBT vs controls). It is possible that highly motivated persons who seek treatment online may require less support than those who have more severe or chronic depression and seek treatment at providers’ offices. The amount and type of support could be a fruitful topic for further study; for example, studies could systematically vary the amount or type of clinical contact (eg, 1 hour vs 4 hours), mode of delivering support, and the level of training of the persons providing support (eg, technician, volunteer, case manager, peer counselor, or psychotherapist) for persons with varied complexity and severity of illness.

Treatment completion rate was found to be an important moderator of treatment effect with the highest mean effect sizes being observed when completion rate was in the top quartile, intermediate mean effect sizes in the next to highest quartile, smaller effects in the lower 2 quartiles, and the lowest mean effect sizes in studies that did not report completion rate. It is likely that completion rate is a complex phenomenon that is affected by a large variety of influences including clinical, sociodemographic, and technical variables. For example, completion rate reported as number of modules accessed does not necessarily account for time spent on modules, which may be a more accurate indicator of completion and adherence. We had suspected that provision of support from a clinician or other helping person would enhance completion rates but found no significant differences in completion rates between supported and nonsupported studies. Potential reasons for difficulties in completing treatment that may be especially relevant in delivery of CCBT could include lack of engagement in computer programs (for example, patients experience the program as too heavily laden with text, onerous to use, not responsive to their specific problems and needs, giving feedback that is off target, not enjoyable or helpful), limited customization or flexibility in treatment, technical problems with reliable access to online programs, and lack of experience in using online resources. Although patients’ subjective responses to CCBT have been reported to be positive in aggregate,80,81 there is room for improvement in the levels of engagement, interactivity, and customization of computer programs used in treatment of depression.

One issue that has received little attention is the use of CCBT in underserved or disadvantaged populations in which educational levels, access to the Internet, or minimal or no prior experience in using computers may present barriers to implementation of a technology-based treatment. Many of the studies in our meta-analysis used subject recruitment strategies that prevented or reduced the likelihood that such persons would participate (eg, cutoffs on educational level and reading proficiency, recruitment primarily via Internet, requirement to have online access), thus yielding samples of predominately well-educated patients with experience in using online resources. One notable exception is a study with mothers of children enrolled in Head Start in which computer access was provided as a component of study participation.29 More work needs to be done on designing CCBT methods to overcome obstacles to implementation in patients from disadvantaged populations.

Treatment setting was another potential moderator explored in our meta-analysis. Specifically, we tested whether studies conducted with patients drawn from primary care practices might have smaller effects than those performed elsewhere. Our finding of lower mean effect sizes in studies conducted in primary care settings is consistent with a recent report by Wells and associates.73 Because there have been a limited number of investigations in primary care settings, it is premature to conclude that CCBT is less effective for such patients. However, there are several possible explanations for the observed difference in treatment outcomes. Recruitment methods that select patients from actual clinical practices may tap a population with more severe or complicated conditions, including comorbidities, which may be less common in Internet-recruited samples. Also, less is known about how to implement CCBT in primary care, and it is possible that further development of methods of delivery, including ways to provide support and overcome barriers to participation, could improve outcomes.73 Development of service pathways in primary care, underserved populations, and health care in general are among the implementation science and practice issues that are challenges for CCBT. Future research that utilizes implementation science82 could increase the dissemination of CCBT into routine care.

A final clinically relevant moderator of study outcomes was pretreatment severity. If one presumes that CCBT is not a robust treatment and should be used primarily for milder forms of depression, it would be expected that studies that enrolled more severely depressed persons would observe poorer outcomes than those that enrolled persons with less severe illness. To the contrary, we observed the largest mean effect sizes in studies with the highest mean pretreatment severity scores. In this regard, CCBT is similar to other active treatments of depression, including pharmacotherapy and individual psychotherapy.83 Evidence that CCBT is as effective as a standard treatment for severe depression was reported recently in a direct comparison of clinician-supported CCBT with a full course of up to 20 sessions of standard CBT in drug-free patients with major depressive disorder.84 These findings should lay to rest the notion that CCBT should be limited to patients with milder depressions.

There are several limitations to our review and meta-analysis. First, less than half of the comparisons of CCBT versus a control condition performed follow-up assessments, and studies that did include follow-up measurements were heavily weighted toward unsupported CCBT. Because many supported studies utilized wait-list control groups and did not compare CCBT with other active treatments, their research designs precluded meaningful longer-term evaluations. The single study3 that compared supported CCBT to standard, individual CBT found that both active treatments had sustained effects across 6 months of follow-up. Also, a recent investigation84 of CCBT versus standard CBT showed no loss of effect 6 months after completion of treatment. However, further research is needed to assess the durability of CCBT in comparison to other effective treatments for depression. Second, we were not able to determine if the effectiveness of CCBT is influenced by recruitment method and the education and previous computer experience of study participants. We think that such influences are likely. Third, many studies were performed with persons from nonclinical samples, thus raising questions about generalizability for treatment of depression in clinical practice.

Fourth, many studies reported low completion rates for persons who participated in CCBT. Although some studies had completion rates of 80% or higher, the factors affecting retention of participants have not been adequately investigated. Fifth, we did not assess the quality of the computer programs used and are not aware of any rating system for doing so. However, it could be worthwhile to determine if the programs with multimedia formats and higher production values are associated with better outcomes than more bare-bones programs that rely primarily on reading text for delivery of program content. We suspect that advances in program quality (eg, engagement, interactivity, mobile components, integration of clinician support with computer delivery) could improve adherence and effectiveness of CCBT. Sixth, there has been less study of CCBT in comparison to conventional individual therapy than wait lists or treatment as usual. Furthermore, the benefit of CCBT has not been compared to well-monitored, guideline-concordant pharmacotherapy for depression. Only 23,54 of the 40 studies in our meta-analysis compared CCBT with standard CBT in addition to a wait list or other nonstandard CBT control condition, and only 1 of these3 was in drug-free patients with documented major depressive disorder. Nevertheless, there have been a growing number of studies that have directly examined the effects of CCBT versus standard CBT, and such studies have not found significant differences in treatment outcome.85

Seventh, without access to individual-level data from participants, we could not determine if the effects of CCBT were comparable for patients with more extensive treatment histories, such as those who had not obtained adequate benefit from at least 1 trial of antidepressant medication or had comorbid conditions. Eighth, a variety of rating scales were used to measure symptom severity. Although we used z-scores to account for this variability, greater consistency in use of rating scales would promote higher confidence in analyses of the influence of treatment severity on outcome. Finally, the majority of CCBT studies reviewed here used desktop, laptop, or notebook computers for treatment delivery instead of fully mobile applications. Smartphones would appear to have considerable potential for CCBT because of their convenience, accessibility, and widespread use. However, most mobile apps developed to date have been geared toward specific self-help methods such as relaxation or breathing training, increased activity levels, or spotting negative thoughts instead of the comprehensive CBT programs employed in CCBT.86-89 There have been a limited number of RCTs of these methods for treating patients with well-documented depression, and concerns have been raised about the quality, reliability, and security of mobile apps.89 Also, it is not known whether CCBT will be more effective if delivered with skill-building modules that involve sustained effort for time periods (eg, 20-40 minutes) that approximate traditional therapy, as in most studies reported here; with smartphones in shorter bursts throughout the day; or with a combination of these methods.

A sufficient number of studies of CCBT for depression have been conducted to conclude that this method, when combined with modest amounts of clinician support, offers potential for delivery of evidence-based treatment at greater efficiency and lower cost than standard CBT. Future development of CCBT for depression should address issues such as refining and improving methods for integrating clinician and computer elements of therapy; enhancing program customization, engagement, and interactivity; increasing completion rates; overcoming barriers to implementation, especially in disadvantaged populations and in primary care settings; and taking advantage of newer technologies as they become available.

Submitted: February 15, 2018; accepted July 24, 2018.

Published online: March 19, 2019.

Potential conflicts of interest: Dr Wright is an author of the Good Days Ahead program used in one study in this systematic review and meta-analysis and has an equity interest in Empower Interactive and Mindstreet, developers and distributors of Good Days Ahead. He receives no royalties or other payments from sales of this program. His conflict of interest is managed with an agreement with the University of Louisville. He receives book royalties from American Psychiatric Publishing, Inc., Guilford Press, and Simon and Schuster. Dr Wright receives grant support from the National Institutes of Health (Agency for Health Care Research and Quality) and the Oticon Foundation. Dr Owen receives grant support from the National Institutes of Health (Agency for Health Care Research and Quality). Dr Richards is an employee of SilverCloud Health, developers of computerized psychological interventions for depression, anxiety, stress and comorbid long-term conditions. Dr Richardson is a consultant to SilverCloud Health and may receive royalties from a CBT oriented computer program developed by this company. Dr Eells receives grant support from the National Institutes of Health (Agency for Health Care Research and Quality) and book royalties from Guilford Press and the American Psychological Association. Dr Brown receives book royalties from American Psychiatric Publishing, Inc. Dr Thase reports the following relationships. He is an advisory/consultant to Acadia, Alkermes, Allergan (Forest, Naurex), AstraZeneca, Cerecor, Eli Lilly, Johnson & Johnson (Janssen, Ortho-McNeil), Lundbeck, MedAvante, Merck, Mocksha8, Nestlé (PamLab), Neuronetics, Novartis, Otsuka, Pfizer, Shire, Sunovion, Takeda. In addition to the National Institute of Mental Health, he received grant support from the Acadia, Agency for Healthcare Research and Quality, Alkermes, Assurex, Avanir, Axome, Forest Pharmaceuticals, Johnson & Johnson, Otsuka Pharmaceuticals, and Takeda. Dr Thase received royalties from the American Psychiatric Press, Guilford Publications, Herald House and W.W. Norton & Company, Inc. Dr Thase’s spouse, Dr Diane Sloan, works for Peloton Advantage, which did business with Pfizer and AstraZeneca. Drs Barrett and Rasku and Ms Polser have no disclosures.

Funding/support: None.

Acknowledgments: The authors thank Carol Wahl (University of Louisville), who provided assistance in manuscript presentation, and Ann Schaap, MLS; Deborah Dobiecz; Leslie Pancratz, MLS; and Beth Johnson, MLS, librarians at Norton HealthCare in Louisville, who helped with the literature search. The acknowledged persons have no potential conflicts of interest.

Supplementary material: Available at PSYCHIATRIST.COM.

REFERENCES

1. Greist JH. Computer interviews for depression management. J Clin Psychiatry. 1998;59(suppl 16):20-24, discussion 40-42. PubMed

2. Wright JH, Wright AS. Computer-assisted psychotherapy. J Psychother Pract Res. 1997;6(4):315-329. PubMed

3. Wright JH, Wright AS, Albano AM, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005;162(6):1158-1164. PubMed CrossRef

4. Marks IM, Cavanagh K, Gega L. Computer-aided psychotherapy: revolution or bubble? Br J Psychiatry. 2007;191(6):471-473. PubMed CrossRef

5. Spek V, Cuijpers P, Nykl×­cek I, et al. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37(3):319-328. PubMed CrossRef

6. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196-205. PubMed CrossRef

7. Andrews G, Cuijpers P, Craske MG, et al. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10):e13196. PubMed CrossRef

8. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329-342. PubMed CrossRef

9. Foroushani PS, Schneider J, Assareh N. Meta-review of the effectiveness of computerised CBT in treating depression. BMC Psychiatry. 2011;11(1):131. PubMed CrossRef

10. So M, Yamaguchi S, Hashimoto S, et al. Is computerised CBT really helpful for adult depression? a meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry. 2013;13(1):113. PubMed CrossRef

11. Dobson KS. A meta-analysis of the efficacy of cognitive therapy for depression. J Consult Clin Psychol. 1989;57(3):414-419. PubMed CrossRef

12. Robinson LA, Berman JS, Neimeyer RA. Psychotherapy for the treatment of depression: a comprehensive review of controlled outcome research. Psychol Bull. 1990;108(1):30-49. PubMed CrossRef

13. Butler AC, Chapman JE, Forman EM, et al. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006;26(1):17-31. PubMed CrossRef

14. Mohr DC, Ho J, Duffecy J, et al. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol. 2010;66(4):394-409. PubMed

15. Mohr DC, Hart SL, Howard I, et al. Barriers to psychotherapy among depressed and nondepressed primary care patients. Ann Behav Med. 2006;32(3):254-258. PubMed CrossRef

16. McCrone P, Knapp M, Proudfoot J, et al. Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry. 2004;185(01):55-62. PubMed CrossRef

17. Spurgeon JA, Wright JH. Computer-assisted cognitive-behavioral therapy. Curr Psychiatry Rep. 2010;12(6):547-552. PubMed CrossRef

18. Warmerdam L, van Straten A, Twisk J, et al. Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res. 2008;10(4):e44. PubMed CrossRef

19. Perini S, Titov N, Andrews G. Clinician-assisted internet-based treatment is effective for depression: randomized controlled trial. Aust N Z J Psychiatry. 2009;43(6):571-578. PubMed CrossRef

20. Ruwaard J, Schrieken B, Schrijver M, et al. Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cogn Behav Ther. 2009;38(4):206-221. PubMed CrossRef

21. Titov N, Andrews G, Davies M, et al. Internet treatment for depression: a randomized controlled trial comparing clinician vs technician assistance. PLoS One. 2010;5(6):e10939. PubMed CrossRef

22. Vernmark K, Lenndin J, Bjפrehed J, et al. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. 2010;48(5):368-376. PubMed CrossRef

23. Berger T, Hפmmerli K, Gubser N, et al. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther. 2011;40(4):251-266. PubMed CrossRef

24. Farrer L, Christensen H, Griffiths KM, et al. Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial. PLoS One. 2011;6(11):e28099. PubMed CrossRef

25. Hollפndare F, Johnsson S, Randestad M, et al. Randomized trial of Internet-based relapse prevention for partially remitted depression. Acta Psychiatr Scand. 2011;124(4):285-294. PubMed CrossRef

26. Titov N, Dear BF, Schwencke G, et al. Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav Res Ther. 2011;49(8):441-452. PubMed CrossRef

27. van Bastelaar KMP, Pouwer F, Cuijpers P, et al. Web-based depression treatment for type 1 and type 2 diabetic patients: a randomized, controlled trial. Diabetes Care. 2011;34(2):320-325. PubMed CrossRef

28. Choi I, Zou J, Titov N, et al. Culturally attuned Internet treatment for depression amongst Chinese Australians: a randomised controlled trial. J Affect Disord. 2012;136(3):459-468. PubMed CrossRef

29. Sheeber LB, Seeley JR, Feil EG, et al. Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression. J Consult Clin Psychol. 2012;80(5):739-749. PubMed CrossRef

30. van der Zanden R, Kramer J, Gerrits R, et al. Effectiveness of an online group course for depression in adolescents and young adults: a randomized trial. J Med Internet Res. 2012;14(3):e86. PubMed CrossRef

31. Carlbring P, Hפgglund M, Luthström A, et al. Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. J Affect Disord. 2013;148(2-3):331-337. PubMed CrossRef

32. H׸if׸dt RS, Lillevoll KR, Griffiths KM, et al. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res. 2013;15(8):e153. PubMed CrossRef

33. Mohr DC, Duffecy J, Ho J, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. PubMed CrossRef

34. Buhrman M, Syk M, Burvall O, et al. Individualized guided internet-delivered cognitive-behavior therapy for chronic pain patients with comorbid depression and anxiety: a randomized controlled trial. Clin J Pain. 2015;31(6):504-516. PubMed CrossRef

35. Kivi M, Eriksson MCM, Hange D, et al. Internet-based therapy for mild to moderate depression in Swedish primary care: short term results from the PRIM-NET randomized controlled trial. Cogn Behav Ther. 2014;43(4):289-298. PubMed CrossRef

36. Buntrock C, Ebert D, Lehr D, et al. Effectiveness of a web-based cognitive behavioural intervention for subthreshold depression: pragmatic randomised controlled trial. Psychother Psychosom. 2015;84(6):348-358. PubMed CrossRef

37. Milgrom J, Danaher BG, Gemmill AW, et al. Internet cognitive behavioral therapy for women with postnatal depression: a randomized controlled trial of MumMoodBooster. J Med Internet Res. 2016;18(3):e54. PubMed CrossRef

38. Pugh NE, Hadjistavropoulos HD, Dirkse D. A randomized controlled trial of therapist-assisted, internet-delivered cognitive behavior therapy for women with maternal depression. PLoS One. 2016;11(3):e0149186. PubMed CrossRef

39. Richards D, Timulak L, O’ Brien E, et al. A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav Res Ther. 2015;75:20-31. PubMed CrossRef

40. Clarke G, Reid E, Eubanks D, et al. Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an internet depression skills intervention program. J Med Internet Res. 2002;4(3):E14. PubMed CrossRef

41. Clarke G, Eubanks D, Reid E, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res. 2005;7(2):e16. PubMed CrossRef

42. Spek V, Nykl׭ček I, Smits N, et al. Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med. 2007;37(12):1797-1806. PubMed CrossRef

43. Clarke G, Kelleher C, Hornbrook M, et al. Randomized effectiveness trial of an internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther. 2009;38(4):222-234. PubMed CrossRef

44. Cooper CL, Hind D, Parry GD, et al. Computerised cognitive behavioural therapy for the treatment of depression in people with multiple sclerosis: external pilot trial. Trials. 2011;12(1):259. PubMed CrossRef

45. Glozier N, Christensen H, Naismith S, et al. Internet-delivered cognitive behavioural therapy for adults with mild to moderate depression and high cardiovascular disease risks: a randomised attention-controlled trial. PLoS One. 2013;8(3):e59139. PubMed CrossRef

46. Lintvedt OK, Griffiths KM, S׸rensen K, et al. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother. 2013;20(1):10-27. PubMed CrossRef

47. Meyer B, Bierbrodt J, Schröder J, et al. Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interv. 2015;2(1):48-59. CrossRef

48. Phillips R, Schneider J, Molosankwe I, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med. 2014;44(4):741-752. PubMed CrossRef

49. Birney AJ, Gunn R, Russell JK, et al. MoodHacker mobile web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth. 2016;4(1):e8. PubMed CrossRef

50. Gilbody S, Littlewood E, Hewitt C, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ. 2016;352:i195. PubMed

51. Hobfoll SE, Blais RK, Stevens NR, et al. Vets prevail online intervention reduces PTSD and depression in veterans with mild-to-moderate symptoms. J Consult Clin Psychol. 2016;84(1):31-42. PubMed CrossRef

52. Montero-Mar×­n J, Araya R, Pérez-Yus MC, et al. An internet-based intervention for depression in primary care in Spain: a randomized controlled trial. J Med Internet Res. 2016;18(8):e231. PubMed CrossRef

53. Roepke AM, Jaffee SR, Riffle OM, et al. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J. 2015;4(3):235-246. PubMed CrossRef

54. Selmi PM, Klein MH, Greist JH, et al. Computer-administered cognitive-behavioral therapy for depression. Am J Psychiatry. 1990;147(1):51-56. PubMed CrossRef

55. Proudfoot J, Ryden C, Everitt B, et al. Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry. 2004;185(1):46-54. PubMed CrossRef

56. Williams AD, Andrews G. The effectiveness of internet cognitive behavioural therapy (iCBT) for depression in primary care: a quality assurance study. PLoS One. 2013;8(2):e57447. PubMed CrossRef

57. American Psychiatric Association. Diagnostic and Statistical Manual for Mental Disorders. Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association; 2000.

58. First MB, Spitzer RL, Gibbon M, et al. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Patient Edition (SCID-I/P). New York, NY: Biometrics Research, New York State Psychiatric Institute; 2002.

59. Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(suppl 20):22-33. PubMed

60. Spitzer RL, Kroenke K, Williams JBW. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. JAMA. 1999;282(18):1737-1744. PubMed CrossRef

61. Beck AT, Steer RA, Ball R, et al. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. J Pers Assess. 1996;67(3):588-597. PubMed CrossRef

62. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23(1):56-62. PubMed CrossRef

63. Radloff LS. The Center to Epidemilogic Depression Scale (CES-D) Scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1(3):385-401. CrossRef

64. Subramaniam K, Krishnaswamy S, Jemain AA, et al. The Clinical Interview Schedule-Revised (CIS-R)-Malay Version, Clinical Validation. Malays J Med Sci. 2006;13(1):58-62. PubMed

65. Cox JL, Holden JM, Sagovsky R. Detection of postnatal depression: development of the 10-item Edinburgh Postnatal Depression Scale. Br J Psychiatry. 1987;150(6):782-786. PubMed CrossRef

66. Kessler RC, Andrews G, Colpe LJ, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32(6):959-976. PubMed CrossRef

67. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134(4):382-389. PubMed CrossRef

68. Rush AJ, Trivedi MH, Ibrahim HM, et al. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54(5):573-583. PubMed CrossRef

69. Boutron I, Moher D, Tugwell P, et al. A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol. 2005;58(12):1233-1240. PubMed CrossRef

70. Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. Orlando, FL: Academic Press; 1985.

71. Higgins JPT, White IR, Anzures-Cabrera J. Meta-analysis of skewed data: combining results reported on log-transformed or raw scales. Stat Med. 2008;27(29):6072-6092. PubMed CrossRef

72. Thase ME. The small specific effects of antidepressants in clinical trials: what do they mean to psychiatrists? Curr Psychiatry Rep. 2011;13(6):476-482. PubMed CrossRef

73. Wells MJ, Owen JJ, McCray LW, et al. Computer-assisted cognitive-behavior therapy for depression in primary care: systematic review and meta-analysis. Prim Care Companion CNS Disord. 2018;20(2):17r02196. PubMed

74. Gelenberg AJ, Freeman MP, Markowitz JC, et al; Work Group on Major Depressive Disorder. Practice Guideline for the Treatment of Patients With Major Depressive Disorder. 3rd ed. Arlington, VA: American Psychiatric Association; 2010. {ED: updated to the “book reference” and remove URL and access dates; please review/verify. 10-31-18/rhb

75. Thase ME, Larsen KG, Kennedy SH. Assessing the ‘ true’ effect of active antidepressant therapy v. placebo in major depressive disorder: use of a mixture model. Br J Psychiatry. 2011;199(6):501-507. PubMed CrossRef

76. Cuijpers P, Cristea IA, Karyotaki E, et al. How effective are cognitive behavior therapies for major depression and anxiety disorders? a meta-analytic update of the evidence. World Psychiatry. 2016;15(3):245-258. PubMed CrossRef

77. Ahern E, Kinsella S, Semkovska M. Clinical efficacy and economic evaluation of online cognitive behavioral therapy for major depressive disorder: a systematic review and meta-analysis. Expert Rev Pharmacoecon Outcomes Res. 2018;18(1):25-41. PubMed CrossRef

78. Gerhards SAH, de Graaf LE, Jacobs LE, et al. Economic evaluation of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry. 2010;196(4):310-318. PubMed CrossRef

79. Titov N, Dear BF, Ali S, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behav Ther. 2015;46(2):193-205. PubMed CrossRef

80. Rost T, Stein J, Löbner M, et al. User acceptance of computerized cognitive behavioral therapy for depression: systematic review. J Med Internet Res. 2017;19(9):e309. PubMed CrossRef

81. Titov N, Andrews G, Sachdev P. Computer-delivered cognitive behavioural therapy: effective and getting ready for dissemination. F1000 Med Rep. 2010;2:49. PubMed

82. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1(1):1. CrossRef

83. Hansen NB, Lambert MJ, Forman EM. The psychotherapy dose-response effect and its implications for treatment delivery services. Clin Psychol. 2002;9(3):329-343.

84. Thase ME, Wright JH, Eells TD, et al. Improving the efficiency of psychotherapy for depression: computer-assisted versus standard CBT. Am J Psychiatry. 2018;175(3):242-250. PubMed CrossRef

85. Carlbring P, Andersson G, Cuijpers P, et al. Internet-based vs face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1-18. PubMed CrossRef

86. Aguilera A, Muench F. There’s an app for that: information technology applications for cognitive behavioral practitioners. Behav Ther (N Y N Y). 2012;35(4):65-73. PubMed

87. Shen N, Levitan M-J, Johnson A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth. 2015;3(1):e16. PubMed CrossRef

88. Van Singer M, Chatton A, Khazaal Y. Quality of smartphone apps related to panic disorder. Front Psychiatry. 2015;6:96. PubMed CrossRef

89. Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017;16(3):287-298. PubMed CrossRef