Telehealth Treatment of Patients in an Intensive Acute Care Psychiatric Setting During the COVID-19 Pandemic: Comparative Safety and Effectiveness to In-Person Treatment

J Clin Psychiatry 2021;82(2):20m13815

To cite: Zimmerman M, Terrill D, D’Avanzato C, et al. Telehealth treatment of patients in an intensive acute care psychiatric setting during the COVID-19 pandemic: comparative safety and effectiveness to in-person treatment. J Clin Psychiatry. 2021;82(2):20m13815.
To share: https://doi.org/10.4088/JCP.20m13815
© Copyright 2021 Physicians Postgraduate Press, Inc.

^aDepartment of Psychiatry and Human Behavior, Brown Medical School, and Department of Psychiatry, Rhode Island Hospital, Providence, Rhode Island
*Corresponding author: Mark Zimmerman, MD, 146 West River St, Providence, RI 02904 ([email protected]).

Telehealth services for mental health treatment have gained increasing attention as a cost-effective way to increase accessibility to evidence-based care.^1,2 Reviews of the literature suggest that telehealth interventions are acceptable, feasible, and as effective as in-person mental health services.^3,4 There is also evidence that an equally strong therapeutic alliance can be developed over telehealth as with in-person therapy.⁵

However, the literature on telehealth for mental disorders remains limited in several ways. For example, a major limitation of the research to date is that most telehealth studies have been focused on treatment for single disorders, despite high rates of comorbidity across mental disorders.^6,7 The limited number of studies intentionally treating patients with co-occurring disorders over telehealth are largely preliminary in nature and used small samples.^8–10

Most research evaluating telehealth has been conducted in outpatient, individual treatment settings. Data are lacking in assessing the efficacy of other methods of telehealth treatment delivery, including group therapy and partial hospital and other intensive treatment settings. While the existing research shows promise for group cognitive processing therapy for posttraumatic stress disorder,¹¹ behavioral treatment for smoking cessation,¹² cognitive-behavioral therapy (CBT) for anxiety and depression,⁸ and CBT for insomnia,¹³ much research is preliminary and remains limited in evaluating effects for diagnostically heterogeneous patients. Moreover, in partial hospital and intensive outpatient settings, the level of severity and the risk of self-harm and suicidal behavior are greater than in outpatient practice, thereby raising concerns as to whether telehealth treatment could be provided while maintaining patient safety.

In the face of the coronavirus disease 2019 (COVID-19) pandemic, much of behavioral health treatment has been required to transition to a virtual format due to public health recommendations and legal guidelines for social distancing.^14,15 Partial hospital programs (PHPs) and intensive outpatient treatment programs have faced unique concerns and challenges.^16,17 In working virtually with acute psychiatric patients who require a higher level of care, many of whom may be transitioning from inpatient programs, appropriate risk management is essential. Furthermore, for primarily group therapy–based programs, additional considerations regarding privacy and technology constraints are needed.

In response to the pandemic, partial hospital and intensive outpatient programs across the country were faced with the task of rapidly adapting the nature of their services. In an effort to disseminate lessons learned from such adaptations, Hom and colleagues¹⁷ thoroughly described the structural changes of their PHP, discussed its feasibility, and presented findings indicating high rates of acceptability among the first 10 patients to be discharged from the virtual program. Similarly, Childs et al^18,19 shared their observations following their shift to virtual intensive outpatient and group-based services from their routine in-person ambulatory services and described promising feasibility data. However, the question remains as to whether virtual intensive outpatient or partial hospital treatment programs yield similar outcomes in maintaining safety, improving symptoms, and reducing clinical impairment as compared to in-person services.

As part of ongoing efforts of the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we have examined the effectiveness of our partial hospital treatment program as a response to the COVID-19 pandemic. In this article, we compare the safety and effectiveness of our PHP services delivered via telehealth to in-person treatment provided to patients prior to the COVID-19 outbreak.

METHODS

Setting

The study was based at the Rhode Island Hospital (RIH) Adult Partial Hospitalization Program, an acute care setting serving the full range of presenting concerns referred from various clinical services. Inclusion criteria were broad to maximize access to care and encompassed a variety of diagnoses and presenting concerns. Patients were excluded from the treatment program if they had a primary substance use disorder or imminent suicidal or homicidal ideation with plan and intent. Inclusion and exclusion criteria were the same for the virtual and in-person programs.

Adaptation to Telehealth

To participate in the virtual program, patients required access to a computer, tablet, or smartphone with access to wireless internet. For those few patients who lacked necessary equipment or connectivity, we provided Kindle devices and the state of Rhode Island provided time-limited free wireless service. Additional inclusion criteria included availability of a private location where patients could participate confidentially in individual and group therapy sessions.

We selected the Zoom platform to conduct telehealth treatment because of ease of use, reliability with minimal technical glitches, and availability, including free account version options for patient and provider use. Additionally, a number of Zoom platform features allowed us to meet program operational needs, including the ability to conduct video telehealth visits in multiple, large groups running simultaneously; the ability to designate various roles, including administrator, host, and cohost; multiple patient-friendly options for sending session invitations; a group preregistration feature that allowed us an efficient way of collecting updated physical address information each day; and a waiting room feature allowing meeting hosts to control entry into the sessions.

All components of treatment, including the intake assessment, individual therapy, psychiatry, and group therapy sessions, were conducted virtually using real-time audio and visual computer-based communication using the Zoom telehealth platform, business account version. In the weeks prior to the program launch, administrative staff surveyed staff to determine technology needs. The majority of clinicians delivered telehealth visits from their homes using personal computers or tablets. The director and assistant director of the program, along with a small subset of clinicians, elected to deliver telehealth visits based at the clinic using either desktop computers with web cameras or tablets. As time progressed following the launch of the program, additional equipment was ordered, including headsets, web cameras, and larger monitors, which were helpful in allowing group leaders to see all patients simultaneously on the screen during large groups.

After verifying eligibility, staff emailed a scanned packet of written information about the program to patients to orient them to the program. This email also contained detailed instructions with screenshots illustrating how to connect to the Zoom telehealth platform. After participants returned the consent and release of information documents via email, administrative staff arranged a trial Zoom session during which they assisted participants to troubleshoot any technical difficulties. The designated clinical support person and cohost of the groups also served in a technical support role at all times. This was helpful in addressing technical challenges that arose for patients during the day while allowing clinicians to focus on attending to clinical issues with fewer interruptions due to technical glitches.

To address the challenge of tracking patient attendance and location in a virtual program, a check-in procedure was implemented in which patients participate each morning in a brief Zoom check-in group with support and administrative staff who record their attendance and location. Patients were also required to identify an emergency contact support person and to submit release of information paperwork upon beginning the program.

In the virtual program, a clinical and technical support person was available. That is, a therapist was available at all times throughout program hours to respond by phone or Zoom visit to urgent patient needs, including needs for urgent clinical support outside of individual sessions, assistance with troubleshooting technical problems, and assistance connecting patients to their individual providers.

The team developed and administered guidelines for patients upon intake for effective participation in the virtual format. These included participating from a private location with the camera on at all times to ensure confidentiality; guidelines on procedures for leaving groups for breaks and individual sessions; recommendations for engaged, effective participation in virtual groups (eg, participating in group seated and minimizing distractions); and contact information for the clinical and technical support person. Finally, recommendations to staff were made to ensure patient confidentiality and safety when conducting group and individual sessions, including recommended settings to select when setting up sessions in the Zoom telehealth platform (eg, enabling the waiting room feature to allow group leaders to verify that each member they admit to a group session is a patient in the program who is presenting to the correct group).

Similar to the in-person RIH PHP, admissions to the virtual program are on a rolling basis, with patients completing varying lengths of treatment depending on the symptoms and problems being addressed.

Intervention

The therapeutic orientation of the RIH PHP is based on acceptance and commitment therapy (ACT) and related evidence-based psychotherapy techniques (eg, CBT, dialectical behavior therapy) delivered in a manner consistent with ACT principles.²⁰ The components of the program are summarized in Table 1.

Patients attended daily individual therapy sessions of approximately 30 to 45 minutes, as well as daily meetings with their psychiatrist. In the virtual program, the patients also attended 3 daily therapy groups plus an optional mindful meditation group. The therapy groups included 2 didactic groups focused on delivering ACT concepts and skills and 1 process-oriented group in which a group leader assists patients to apply ACT concepts and skills to day-to-day concerns they raised in the group. The focus, content, and structure of these groups have remained consistent with the in-person RIH PHP (see Morgan et al²⁰ for details).

The in-person program included 4 group therapy sessions per day plus the optional mindful meditation group.²⁰ This was reduced to 3 groups per day in the virtual program, with the removal of the 30-minute morning group focused on values. Values content was integrated into the remaining 2 didactic groups in the virtual program in order to provide additional time required in the virtual format to accommodate morning check-in, conducted by support and administrative staff via Zoom, and to connect with patients for individual sessions. An additional change to the group structure involved the incorporation of a cohost. A licensed clinical therapist and a psychologist served in the two cohost roles. A cohost accompanied the group leader in all therapy groups, observing with the camera off while assisting with various group management functions, including admitting patients to the group from the waiting room, responding to chats from patients, and engaging in individual interactions outside of groups via private video meetings or phone calls with patients exhibiting distress or group-interfering behaviors.

Measures

As part of a department-wide outcomes evaluation initiative, patients presenting to the PHP were asked to complete the Clinically Useful Patient Satisfaction Scale (CUPSS) at the end of their intake session with their psychiatrist.²¹ The partial hospital version of the CUPSS includes an item assessing overall global satisfaction with the initial evaluation on a 5-point scale (0 = not at all satisfied; 4 = extremely satisfied) and an item assessing expectation of improvement in the program (“After the evaluation, I was more hopeful I would get better”: 0 = definitely not; 4 = definitely yes). On the day of discharge from the program, the patients completed a satisfaction scale on which they rated their overall satisfaction with treatment (0 = not at all satisfied; 4 = extremely satisfied) and their overall level of improvement (0 = no better; 4 = very much better).

The primary outcome measure was a modified version of the Remission from Depression Questionnaire (RDQ-M).²² The domains covered on the RDQ were based on a literature review, our previous study of patients’ ratings of the relative importance of 16 factors in determining remission,²³ and 2 focus groups. We modified the RDQ to enhance its applicability to a diagnostically heterogeneous sample as seen in the PHP. Symptom items were added assessing anxiety, anger, and physical pain, as well as adding items to the coping, functioning, and well-being subscales. The 60-item RDQ-M includes 25 symptom items, 5 coping ability/stress tolerance items, 12 positive mental health items, 10 functioning items, and 8 general well-being/life-satisfaction items. The items refer to the prior week and are rated on a 3-point rating scale (not at all or rarely true, sometimes true, often or almost always true). The items are scored 0, 1, and 2, with higher item values reflecting greater symptomatology, greater coping ability, more positive mental health, better functioning, and greater well-being. The internal consistency (Cronbach α) of the RDQ-M subscales was high in both the in-person and telehealth samples (symptom scale, 0.94 and 0.94; coping/stress tolerance subscale, 0.75 and 0.74; positive mental health subscale, 0.93 and 0.94; functioning subscale, 0.86 and 0.88; well-being/life satisfaction subscale, 0.92 and 0.93).

During the in-person program, the admission and discharge scales were handed to the patients by their treating clinicians. In the virtual program, patients were sent links by both support staff and their clinicians to complete the scales online. The Rhode Island Hospital institutional review committee approved the research protocol, and all patients provided informed consent. Consent in the in-person program was obtained on a paper consent form, whereas in the virtual program it was obtained on an electronically signed form.

Data Analysis

T tests were used to compare the telehealth and in-person groups on continuously distributed variables, and χ² statistics were used to compare categorical variables. For each RDQ-M subscale, we used paired t tests to compare follow-up scores to baseline values. We computed the effect size (Cohen d) on each subscale. An effect size of 0.2 was considered small, 0.5 medium, and 0.8 large.²⁴ An analysis of covariance, controlling for baseline scores, was used to compare the amount of change from admission to discharge on the RDQ-M subscales between the in-person and telehealth groups.

RESULTS

Patient Characteristics

During the 4 months from May 2020 to September 2020, 348 patients were admitted to and discharged from the virtual PHP program. Forty-nine patients (14.1%) had previously been treated in the program and thus were not considered further. Because of delays in getting approval from the institutional review board and clinician oversight in requesting consent for participating in the study, 63 patients (18.1%) did not undergo consent procedures. An additional 29 patients (8.3%) refused consent. This left a sample of 207 first admission patients who were treated virtually and whose treatment episode in the PHP had ended. This sample included patients who dropped out during the course of treatment, as this was one of the outcomes that we examined.

We compared the 207 consenting patients to the 92 patients who did not undergo consent procedures or refused consent on demographic and diagnostic variables. There were no significant differences between the groups on any of these variables.

The in-person treatment comparison group was selected from 414 patients who provided consent and were admitted and discharged from the program during the 4 months from May 2019 to September 2019. To match the number of patients who attended the program virtually during the same timeframe in 2020, every other alternate patient was removed from the database, leaving an equivalent sample size of 207 patients.

The in-person and virtually treated groups were similar in age, gender, and race (Table 2). The patients treated virtually were significantly more likely to have graduated high school or achieve equivalency, and they were significantly less likely to have been widowed. There were no differences between the treatment groups in their principal psychiatric diagnosis (Table 3).

The mean ± SD number of current diagnoses (principal and comorbid) in the telehealth and in-person groups was significantly higher in the telehealth group (3.3 ± 1.5 vs 2.9 ± 1.5, t = 2.60, P < .01). Looking at individual diagnoses, the only significant differences between the groups were higher rates of persistent depressive disorder, social anxiety disorder, and specific phobia in the telehealth group (Table 4).

Patient Satisfaction

The rate of completion of the CUPSS after the initial evaluation by the psychiatrist was significantly lower in the virtual cohort (45.8% vs 97.6%, χ² = 136.40, P < .001). In the virtual cohort, we compared the demographic and diagnostic characteristics of the patients who did and did not complete the CUPSS and found no significant differences between the groups.

Most patients in the virtual and in-person groups indicated that they were very or extremely satisfied with the initial evaluation (83.2% vs 88.2%, χ² = 1.24, nonsignificant [NS]). Likewise, the majority of patients in both the virtual and in-person groups were hopeful that they would get better (75.6% vs 79.8%, χ² = 0.61, NS). Upon treatment completion, more than 90% of the patients in the in-person and virtual groups indicated that they were very or extremely satisfied with their treatment (97.4% vs 94.0%, χ² = 1.24, NS).

Program Completion

Significantly more patients completed treatment in the virtual program than the in-person program (72.9% vs 62.3%, χ² = 5.34, P < .05). A nonsignificantly higher percentage of patients were discharged from the in-person program due to nonattendance (14.0% vs 8.7%, χ² = 2.90, P < .10). Transfer to inpatient care was low in both the virtual and in-person programs (1.4% vs 3.4%, χ² = 1.64, NS). Likewise, withdrawal from treatment due to dissatisfaction with the program was low in both the virtual and in-person programs (2.9% vs 1.4%, χ² = 1.00, NS).

The mean ± SD number of days missed while in treatment was similar in the virtual and in-person programs (1.4 ± 1.7 vs 1.5 ± 1.8, t = 0.86, NS). The mean ± SD number of days attending the program was significantly higher in the virtual program (13.5 ± 8.1 vs 8.5 ± 5.0, t = 7.61, P < .001).

Treatment Outcome

Significantly fewer virtually treated patients completed the RDQ-M at admission (65.7% vs 80.2%, χ² = 61.21, P = .001). A similar percentage of virtual and in-person treated patients completed the RDQ-M at discharge (53.6% vs 54.6%, χ² = 0.04, P = .84). For both groups, there were no significant differences in demographic and diagnostic characteristics between the patients with and without complete outcome data.

At admission to the program, there were no significant differences between the virtual and in-person cohorts on the RDQ-M subscales (Table 5). Under both virtual and in-person treatment conditions, the patients significantly improved from admission to discharge on each of the RDQ-M subscales. A large effect size was found for both virtual and in-person treatment conditions for each RDQ-M subscale. Most patients in the in-person and virtual groups indicated that they were a lot or very much better at discharge (79.5% vs 72.0%, χ² = 1.47, NS).

No patients attempted or completed suicide during their treatment in the program.

We also examined the 2 items on the RDQ-M that assessed suicidal ideation. From admission to discharge, there was a significant reduction in the percentage of patients reporting death wishes (in-person: 56.1% vs 23.5%, χ² = 21.62, P < .001; virtual: 55.3% vs 24.7%, χ² = 16.48, P < .001) and suicidal ideation during the past week (in-person: 48.0% vs 21.4%, χ² = 15.22, P < .001; virtual: 44.7% vs 17.6%, χ² = 14.47, P < .001).

DISCUSSION

In an intensive acute care setting consisting of daily group and individual therapy as well as psychopharmacologic management, delivering treatment using a virtual, telehealth platform was as effective as treating patients in person. For both methods of delivering treatment, patients were satisfied with the initial diagnostic evaluation and were optimistic at admission that treatment would be helpful. Both treatment groups reported a significant reduction in symptoms and suicidality from admission to discharge and reported a significant improvement in functioning, coping ability, positive mental health, and general well-being. A large effect size of treatment was found in both treatment groups. The only significant difference in outcome between the patients treated in the different formats was a greater length of stay and greater likelihood of staying in treatment until completion in the virtually treated patients.

While there are many studies demonstrating the efficacy of telehealth, little research has focused on patients treated in an intensive PHP that is largely group therapy–based and includes patients who are more severely ill than patients typically treated in outpatient settings. To be sure, as described in the Methods section, added precautions were taken to ensure that emergencies could be addressed in the virtually treated patients. Only a small percentage of patients in both treatment formats were referred for inpatient admission, and there was no significant difference between the formats in this regard. No patients attempted or completed suicide during their treatment in the program.

Consistent with research in outpatient mental health clinics that found a lower “no show” rate for telehealth visits during the pandemic compared to in-person visits scheduled before the pandemic,²⁵ we found that the treatment completion rate was significantly higher in the telehealth cohort. In part, we attribute the longer duration of treatment and greater completion rate in the telehealth group to the circumstances associated with the COVID-19 pandemic. Because of social distancing recommendations during the pandemic, some patients were socially isolated and attendance in the PHP was a source of social engagement. This may have increased patients’ desire to stay in the virtual program for a longer amount of time. Some patients were furloughed or laid off from their jobs during the pandemic and thus were less pressured to transition to an outpatient level of care in order to return to work. Health insurance company utilization review was suspended during the early phase of the pandemic, thereby reducing pressure to discharge patients sooner than clinicians would have liked. There may also be nonpandemic contributors to the longer duration of treatment and greater treatment completion rate in the telehealth patients. Parents of children were better able to balance their parental obligations with program participation while in the virtual program. The elimination of travel made it easier to attend the telehealth program. Finally, it is also possible that the response to treatment was slower with the telehealth format and that this resulted in a longer duration of treatment.

The present study was not designed to compare the effectiveness of in-person and virtual treatment formats. We transitioned to the virtual platform because of the COVID-19 pandemic. Accordingly, we did not randomize patients to the treatment formats but instead examined the effectiveness of treatment in sequentially recruited cohorts. The only variable we controlled for was the time of year the patient was admitted to the PHP. Fortunately, there were few differences between the patient groups in demographic characteristics, psychiatric diagnosis, and baseline scores on the outcome measure. While a randomized, controlled trial is the gold standard clinical trial design, it would likely be cost prohibitive to conduct such a study because of the need to double the clinical staff needed to run parallel PHPs at the same time.

It was easier to collect data when the patients were treated in person. When our program was conducted in person, almost all patients completed the satisfaction survey at admission. In the virtual program, the response rate was approximately 50%. Direct person-to-person contact, in which the forms are handed to the patients by their treating clinicians, likely enhanced completion rates when compared to sending patients electronic links to surveys to be completed online at home. Importantly, there were no differences in the demographic and clinical characteristics of the patients who did and did not complete the various measures.

While the focus of the present article has been on the impact of the transition to telehealth on patients, we have also been vigilant with regard to the impact of this transition on the clinical staff. During the early phase of the transition, the focus was on implementation and problem solving. After working exclusively in the virtual format for 9 months, the staff was surveyed as to the personal impact of working virtually. Among the positive aspects of delivering telehealth treatment, the staff noted the convenience and economic benefit of not having to commute to work, the ability to circumvent cold and/or inclement weather, and the convenience of attending to some household chores between patient visits. On the other hand, some of the negative aspects of telehealth and working from home have been isolation from coworkers, fatigue, and greater sedentariness.

A limitation of the study is that outcome assessment relied on self-administered questionnaires and did not include clinician rating scales. However, previous research from our laboratory has shown that the effect size of treatment was similar when based on self-report scales and clinician administered measures.²⁶

Several patients whom we treated virtually commented that they never would have presented for in-person treatment even if there was no pandemic. Some of these patients had medical illnesses that made in-person treatment attendance more difficult to manage. For some patients, limited transportation options made in-person treatment more difficult. While telehealth treatment provided access to treatment for some patients who otherwise would not have attended the program, some patients who lacked the necessary hardware or connectivity might not have called to participate. If patients lacked an electronic device, we provided them with one. If they lacked internet service, we directed them to assistance programs that provided temporary connection at no cost. A small number of patients had intermittent connectivity problems, which sometimes delayed or interfered with sessions and resulted in down time for the clinician. Support staff assisted in the resolution of such problems. On severe weather days, when power lines were disrupted, some patients were unable to connect and participate in the program.

Other advantages and disadvantages of telehealth treatment relative to in-person PHP treatment were noted by the clinicians. Telehealth made it easier to conduct family meetings. It was easier to accurately determine the names and dosages of patients’ medications because the patients retrieved their pill bottles. It was sometimes helpful to see a patient’s living environment. On the other hand, observation of a patient’s physical appearance and body language could be compromised. Some clinicians thought telehealth treatment was less personal, that nuanced affective reactions were less apparent, and that “clinical feel” was lost.

A limitation of comparing outcome in sequentially treated cohorts is that circumstances unrelated to treatment efficacy could impact treatment outcome. We adopted the telehealth format because of the COVID-19 pandemic, and we are unsure of the impact of the pandemic on the effectiveness of treatment. The pandemic has had a negative impact on mental health of the general population as well as psychiatric patients.^27–31 Before concluding that a telehealth PHP is as effective as in-person treatment, it will be important to compare treatment formats when pandemic-related issues have subsided. It will also be important to conduct post-discharge follow-up studies to evaluate the maintenance of the benefits of treatment.

The results of the study suggest comparable effectiveness, patient satisfaction, and safety of in-person and telehealth treatment of acutely ill patients who, on average, had multiple psychiatric disorders. Yet, we are uncertain how we will deliver treatment in the future when the pandemic resolves. Will we return to an in-person program, or might we possibly create a hybrid program in which some patients are in-person and some are virtual? Or, will we run separate in-person and virtual programs? As described above, there are advantages and disadvantages to telehealth treatment. Increased access to treatment for patients is clearly a major advantage. However, decisions about how care is delivered in the future might be largely influenced by insurance reimbursement. Hopefully, regulations will be passed to ensure that virtually delivered treatment is compensated at the same level as in-person treatment. If not, this will likely undermine efforts to make telehealth treatment more widely available to patients. It will also be important to determine if there are subsets of patients who respond better to one approach over another. Of course, patient and clinician preferences will also need to be considered. For example, the significantly higher percentage of patients with social anxiety disorder in our telehealth cohort suggests that the availability of telehealth treatment might provide distressed, depressed, socially anxious patients with a treatment option they might otherwise have avoided. As our sample size grows, we plan to conduct future analyses to determine if there are specific subpopulations that are better treated in telehealth or in-person formats.

Submitted: November 28, 2020; accepted February 1, 2021.
Published online: March 16, 2021.
Potential conflicts of interest: None.
Funding/support: None.

Clinical points

• There is a lack of research assessing the efficacy of telehealth treatment in acute, intensive treatment settings such as a partial hospital.
• In patients with an average of 3 psychiatric diagnoses, half of whom reported suicidal ideation upon admission to treatment, telehealth partial hospital level of care was as effective as in-person treatment in terms of patient satisfaction, symptom reduction, suicidal ideation reduction, and improved functioning and well-being.