Comparison of Teleintegrated Care and Telereferral Care for Treating Complex Psychiatric Disorders in Primary Care

Key Points

Question

Which is more effective, an integrated or referral approach to using clinic-to-clinic interactive video to deliver evidence-based mental health treatments to patients with complex psychiatric disorders in primary care clinics?

Findings

In this pragmatic randomized comparative effectiveness trial including 1004 adult participants, both approaches significantly and substantially improved clinical outcomes. The referral approach used substantially more mental health specialist time than the integrated approach.

Meaning

Based on findings from this trial, from a health care system perspective, clinical leadership should implement whichever approach is most sustainable; from a societal perspective, policy makers should incentivize the integrated approach because it required less scarce mental health specialist time.

This pragmatic randomized comparative effectiveness trial compares 2 clinic-to-clinic interactive video approaches to delivering evidence-based mental health treatments to patients in primary care clinics.

Abstract

Importance

Only one-third of patients with complex psychiatric disorders engage in specialty mental health care, and only one-tenth receive adequate treatment in primary care. Scalable approaches are critically needed to improve access to effective mental health treatments in underserved primary care settings.

Objective

To compare 2 clinic-to-clinic interactive video approaches to delivering evidence-based mental health treatments to patients in primary care clinics.

Design, Setting, and Participants

This pragmatic comparative effectiveness trial used a sequential, multiple-assignment, randomized trial (SMART) design with patient-level randomization. Adult patients treated at 24 primary care clinics without on-site psychiatrists or psychologists from 12 federally qualified health centers in 3 states who screened positive for posttraumatic stress disorder and/or bipolar disorder and who were not already receiving pharmacotherapy from a mental health specialist were recruited from November 16, 2016, to June 30, 2019, and observed for 12 months.

Interventions

Two approaches were compared: (1) telepsychiatry/telepsychology–enhanced referral (TER), where telepsychiatrists and telepsychologists assumed responsibility for treatment, and (2) telepsychiatry collaborative care (TCC), where telepsychiatrists provided consultation to the primary care team. TER included an adaptive intervention (phone-enhanced referral [PER]) for patients not engaging in treatment, which involved telephone outreach and motivational interviewing.

Main Outcomes and Measures

Survey questions assessed patient-reported outcomes. The Veterans RAND 12-item Health Survey Mental Component Summary (MCS) score was the primary outcome (range, 0-100). Secondary outcomes included posttraumatic stress disorder symptoms, manic symptoms, depressive symptoms, anxiety symptoms, recovery, and adverse effects.

Results

Of 1004 included participants, 701 of 1000 (70.1%) were female, 660 of 994 (66.4%) were White, and the mean (SD) age was 39.4 (12.9) years. Baseline MCS scores were 2 SDs below the US mean; the mean (SD) MCS scores were 39.7 (14.1) and 41.2 (14.2) in the TCC and TER groups, respectively. There was no significant difference in 12-month MCS score between those receiving TCC and TER (β = 1.0; 95% CI, −0.8 to 2.8; P = .28). Patients in both groups experienced large and clinically meaningful improvements from baseline to 12 months (TCC: Cohen d = 0.81; 95% CI, 0.67 to 0.95; TER: Cohen d = 0.90; 95% CI, 0.76 to 1.04). For patients not engaging in TER at 6 months, there was no significant difference in 12-month MCS score between those receiving PER and TER (β = 2.0; 95% CI, −1.7 to 5.7; P = .29).

Conclusions and Relevance

In this comparative effectiveness trial of patients with complex psychiatric disorders randomized to receive TCC or TER, significantly and substantially improved outcomes were observed in both groups. From a health care system perspective, clinical leadership should implement whichever approach is most sustainable.

Trial Registration

ClinicalTrials.gov Identifier: NCT02738944

Introduction

Only one-third of individuals with bipolar disorder (BD) and posttraumatic stress disorder (PTSD) receive specialty mental health care during the course of a year.¹ In primary care settings, only one-tenth of patients with BD and PTSD receive adequate care compared with more than half in specialty mental health.¹ Managing complex psychiatric disorders is especially challenging for primary care clinicians in federally qualified health centers (FQHCs). There are nearly 1400 FQHCs with more than 13 000 clinic locations that provide services to 30 million individuals in the US.² Almost half (44%) of patients treated at FQHCs live in rural areas,³ 91% live in poverty,⁴ and 62% are from racial or ethnic minority groups.⁴ While 97% of FQHCs offer on-site mental health services, only 12% of mental health staff are psychiatrists or licensed clinical psychologists.⁴ The shortage of psychiatrists and psychologists in rural and poverty-stricken areas precipitates this chronic staffing problem in FQHCs.^1,5

The widespread adoption of telepsychiatry and telepsychology owing to the COVID-19 pandemic could potentially increase access for primary care patients with complex psychiatric disorders living in underserved areas. The Study to Promote Innovation in Rural Integrated Telepsychiatry (SPIRIT) trial was a pragmatic trial (PCS-1406-19295) designed to identify the best approach to delivering telemental health services to primary care clinics. Specifically, the SPIRIT trial compared the effectiveness of telepsychiatry collaborative care (TCC) and telepsychiatry/telepsychology–enhanced referral (TER) to treat BD and PTSD.⁶

TCC is an integrated population-based model of care.^7,8,9 By integrating BD and PTSD treatment into primary care and taking a population-based care management approach, TCC is expected to engage a higher proportion of patients in treatment than TER, which is a traditional referral model of care that focuses exclusively on patients attending scheduled appointments. However, TCC telepsychiatrists only provide consultation to the primary care team, while TER telepsychiatrists and telepsychologists provide direct ongoing treatment to patients. Therefore, the as-treated effectiveness of TCC may be lower than TER. TCC and TER represent clinical equipoise with respect to intent-to-treat effectiveness, with TCC expected to have greater engagement but lower as-treated effectiveness for those engaged in treatment and TER expected to have lower engagement but greater as-treated effectiveness for those engaged in treatment. We hypothesized that the greater engagement in TCC would result in better intent-to-treat effectiveness. Although not a common practice, telemental health referral models could adopt a more population-based approach, and the SPIRIT trial was also designed to test the intent-to-treat effectiveness of such an approach. For patients not engaging in treatment, we also hypothesized that phone-enhanced referral (PER), which uses telephone outreach to encourage patients to initiate or reengage in treatment, would result in better engagement and outcomes than continued TER.

Methods

Human subjects protection oversight was provided by the institutional review boards of the University of Arkansas for Medical Sciences, University of Michigan, and University of Washington. Written informed consent was obtained for all study participants. The trial was designed and conducted in close collaboration with consumer and policy advisory boards. The trial protocol can be found in Supplement 1.

Study Sites

A total of 24 clinics from 12 FQHCs in 3 states (Arkansas, Michigan, and Washington) participated. Clinics were eligible if they had no psychiatrists or licensed clinical psychologists practicing on site. Telepsychiatrists and telepsychologists from state medical schools were credentialed and privileged to practice at the FQHC and documented their clinical assessment and treatment plan in their electronic health record.¹⁰ Study participants received up to 12 months of treatment and presented to the clinic for interactive video encounters.

Study Population

Patients screening positive for BD and/or PTSD were enrolled from November 16, 2016, to June 30, 2019 (Figure 1). Inclusion criteria were based on screening instruments (Composite International Diagnostic Interview version 3.0¹¹ for BD and PTSD Checklist-6 for PTSD¹²) administered during annual wellness visits rather than structured clinical assessments to reflect real-world practice and to maximize external validity. To minimize false-positives and screening burden,¹³ only patients screening positive for depression (Patient Health Questionnaire-9 score of 10 or greater) were screened for BD and PTSD. Exclusion criteria were minimal: (1) age younger than 18 years, (2) unable to communicate in English or Spanish, (3) lacked capacity to consent, (4) no future FQHC visits planned, and (5) already being prescribed psychotropic medications by a psychiatrist or psychiatric nurse practitioner at baseline. Patients prescribed psychotropic medications by a primary care clinician were included.

Figure 1. CONSORT Flow Diagram.

FQHC indicates federally qualified health center; PER, phone-enhanced referral; TCC, telepsychiatry collaborative care; TER, telepsychiatry/telepsychology–enhanced referral.

Randomization

A sequential, multiple-assignment, randomized trial (SMART) design was used because TER is an adaptive intervention.¹⁴ Patients were initially randomized to TCC and TER using blocking and stratification by FQHC and screening status. Patients screening positive for both BD and PTSD were categorized as having BD for stratification purposes. At 6 months, patients assigned to TER with 2 or less interactive video encounters were randomized a second time (using blocking and stratification by FQHC and screening status) to either continued TER or PER.

TCC

On-site behavioral health care managers (eg, social workers or nurses) and off-site telepsychiatrist consultants supported primary care clinicians, who prescribed all psychotropic medications. Care managers provided psychoeducation, conducted outreach and treatment engagement activities, and delivered behavioral activation (BA) psychotherapy.¹⁵ Care managers used a web-based registry, the Care Management Tracking System (CMTS),¹⁶ to monitor engagement and symptom severity (ie, measurement-based care¹⁷). Telepsychiatrist consultants conducted an initial diagnostic assessment for all patients. The telepsychiatrists met weekly with care managers for case reviews (approximately 10 minutes per patient) to identify patients not engaging in or responding to care and suggested treatment recommendations to primary care clinicians. TCC care was documented in CMTS and the electronic health record.

TER

Patients initially had a telepsychiatry encounter to establish diagnosis and develop a treatment plan. Telepsychiatrists ordered laboratory tests and electronically prescribed medications in the FQHC’s electronic health record. If referred by telepsychiatrists, telepsychologists delivered either cognitive processing therapy (CPT)¹⁸ for PTSD or cognitive behavioral therapy (CBT)¹⁹ for BD. FQHC staff scheduled appointments and sent appointment reminders. Telepsychiatrists and telepsychologists monitored symptoms using CMTS, and treatment was documented in CMTS and the electronic health record.

PER

The adaptive intervention involved reaching out telephonically to patients. FQHC staff scheduled telephone appointments with the telephone psychologist, who encouraged patients to attend telepsychiatry/telepsychology interactive video encounters in the clinic. In contrast to TER, which focuses on treating patients attending appointments, PER focuses on engaging patients in treatment.

Survey

Telephone or web-based surveys were administered at baseline and 6 and 12 months later. Treatment group assignment was masked for telephone interviewers. The primary outcome was mental health functioning at 12 months as measured by the Veterans RAND 12-item Health Survey Mental Component Summary (MCS) score (range, 0 to 100).²⁰ The MCS is a non–disease-specific assessment of vitality, role functioning, social functioning, and feeling calm and peaceful, and scores represent an outcome that is highly relevant to patients. Secondary outcomes included the (1) PTSD Checklist-5 (PCL-5)²¹ for PTSD, (2) Hopkins Symptom Checklist (SCL-20)²² for depression, (3) Altman Mania Rating Scale (AMRS)²³ for mania, (4) Internal State Scale (ISS)²⁴ for mood state, (5) General Anxiety Disorder-7²⁵ for general anxiety, (6) Recovery Assessment Scale,²⁶ and (7) number of moderate and severe adverse effects from psychotropic medications. Access was measured using the Assessment of Perceived Access to Care (APAC).²⁷ Engagement mediators included self-reported psychotropic medication prescription and adherence and number of psychotherapy encounters (BA, CPT, or CBT) documented in CMTS.

Statistical Analysis

Intent-to-treat hypotheses were tested using a 2-level model with longitudinal observations (level 1) nested within patients (level 2). To account for stratified randomization, screening status and FQHCs were included as fixed effects.²⁸ Mixed models included a random intercept, random linear slope, and 6-month and 12-month indicators to allow for nonlinear change over time. The adjusted difference between TCC and TER at each time point was tested with a group × time interaction term, using a 2-sided α level of .05. Outcomes at 12 months for the TER group represented the average effect of being randomized to PER or continued TER in months 6 to 12 for nonresponders. The same modeling approach was used to analyze the second-stage randomization, except that only 1 group × time interaction term was used to estimate the difference between TER and PER at 12 months. Disorder-specific outcomes (PCL-5, ISS, and AMRS scores) were analyzed only in the subgroup that screened positive for the disorder. Euthymic mood based on the ISS was specified as a binary variable and modeled with a log link for relative risks. Adverse effects were analyzed as a count variable and modeled with a negative binomial distribution.

Assuming a 30% loss to follow-up, a sample size of 1000 was needed to have 80% power (α = .05) to detect a small effect size (Cohen d = 0.21) for MCS between first-stage randomization arms and 80% power to detect a medium effect size (Cohen d = 0.43) between second-stage randomization arms (assuming 50% did not engage in TER). For secondary outcomes, we calculated 95% CIs but did not test hypotheses.

To test hypotheses about baseline treatment effect modifiers, moderator × group × time interaction terms were added. In this model, baseline MCS scores were incorporated as a level-2 covariate, and time was centered at 6 months. To test mediation hypotheses, a multivariate structural equation model was specified. Treatment effects on the mediators (A paths in eFigure 1 in Supplement 2), psychotherapy engagement, and medication engagement (prescribed medications and always/mostly adherent) were modeled using negative binomial and logit links, respectively. The mediators were included as time-varying predictors (B paths in eFigure 1 in Supplement 2) of changes in MCS, with a treatment-mediator interaction²⁹ to account for the possibility that psychotherapy and medication management had differential effectiveness across condition. Indirect effects were calculated using the delta method.³⁰

To account for missing data at follow-up, 100 complete data sets were imputed³¹ using random forest imputation.^32,33,34 Imputation was stratified by initial randomization group to allow potentially different effects and separate covariance structures by group.³⁵ Potential missingness mechanisms were examined by correlating loss to follow-up status with key baseline characteristics. To gauge the sensitivity of results to violations of the missing-at-random assumption, we calculated how the primary outcome estimate changed under different proportions and effect sizes of nonignorable missingness. Statistical analyses were conducted using Mplus version 8 (Muthen & Muthen).

Results

Most patients screening positive for BD and/or PTSD and assessed for eligibility (1931 of 2464 [78.4%]) did not meet any exclusion criteria, and two-thirds of eligible patients (1214 of 1931 [62.9%]) consented to participate (Figure 1). The most common reason for ineligibility was already being prescribed psychotropic medications by a psychiatrist or psychiatric nurse practitioner at baseline (453 of 533 [85.0%]; Figure 1).

Of 1004 included participants, 701 of 1000 (70.1%) were female, 660 of 994 (66.4%) were White, and the mean (SD) age was 39.4 (12.9) years (Table 1). A total of 503 of 1003 patients (50.1%) lived in a rural area, 222 of 1002 (22.2%) did not graduate from high school, 660 of 1002 (65.9%) were unmarried, 789 of 976 (80.8%) were not employed full time, 620 of 946 (65.5%) lived below the 2016 federal poverty level, and 827 of 997 (83.0%) were publicly insured or uninsured. Most participants (880 of 975 [90.3%]) reported a perceived need for mental health treatment, and 691 of 975 (70.9%) were taking psychotropic medications prescribed by their primary care clinician at baseline. The mean (SD) MCS score (range, 0 to 100) at baseline was 30.8 (11.2). Nearly all participants screened positive for PTSD (978 [97.4%]) with moderate severity (mean [SD] PCL-5 score, 48.0 [17.7]), and 760 (78.1%) reported a traumatic event meeting PTSD diagnostic criterion. One-third of participants (367 [36.6%]) screened positive for lifetime BD, and of these, only 34 (9.3%) were euthymic at baseline. Physical and mental health comorbidities were common. Of the 767 participants with a TCC or TER telepsychiatry encounter during the 12-month treatment period, 357 (46.5%) were diagnosed with PTSD only, 135 (17.6%) were diagnosed with BD and PTSD, and 57 (7.4%) were diagnosed with BD only.³⁶ Other telepsychiatrist-assigned diagnoses included unipolar depression (457 [59.6%]), anxiety (287 [37.4%]), alcohol use disorder (103 [13.4%]), other substance use disorder (109 [14.2%]), personality disorder (75 [9.8%]), and schizophrenia (29 [3.8%]).

Table 1. Baseline Characteristics of Patients Enrolled in the Study to Promote Innovation in Rural Integrated Telepsychiatry (SPIRIT) Trial.

Characteristic	No./total No. (%)
	TCC (n = 508)	TER (n = 496)
Demographic characteristics
Age, mean (SD), ya	39.8 (13.0)	39.0 (12.8)
Sexb
Female	354/507 (69.8)	347/493 (70.4)
Male	146/507 (28.8)	137/493 (27.8)
Transgender or nonbinary	7/507 (1.4)	9/493 (1.8)
Sexual orientationc
Heterosexual	242/298 (81.2)	232/287 (80.8)
Lesbian or gay	9/298 (3.0)	12/287 (4.2)
Bisexual	29/298 (9.7)	30/287 (10.5)
Other	18/298 (6.0)	13/287 (4.5)
Self-reported race and ethnicity
African American	64/506 (12.7)	54/488 (11.1)
Hispanic	38/506 (7.5)	39/488 (8.0)
Non-Hispanic White	336/506 (66.4)	324/488 (66.4)
American Indian/Alaska Native	22/506 (4.4)	14/488 (2.9)
Multirace	25/506 (4.9)	38/488 (7.8)
Other	21/506 (4.2)	19/488 (3.9)
Marital status
Married or living with a partner	171/508 (33.7)	171/494 (34.6)
Widowed	18/508 (3.5)	18/494 (3.6)
Divorced or separated	159/508 (31.3)	141/494 (28.5)
Single, never married	160/508 (31.5)	164/494 (33.2)
Education
≤8th Grade	10/507 (2.0)	16/495 (3.2)
Some high school	101/507 (19.9)	95/495 (19.2)
High school graduate	158/507 (31.2)	157/495 (31.7)
Some college	174/507 (34.3)	165/495 (33.3)
College graduate or postgraduate	64/507 (12.6)	62/495 (12.5)
Employment
Working full-time	91/492 (18.5)	96/484 (19.8)
Working part-time	64/492 (13.0)	61/484 (12.6)
Laid off, on strike, unemployed, or disabled	270/492 (54.9)	265/484 (54.8)
Retired	50/492 (10.2)	46/484 (9.5)
Student	17/492 (3.5)	16/484 (3.3)
Veteran	25/508 (4.9)	28/496 (5.7)
Household income below 100% federal poverty level	314/478 (65.7)	306/468 (65.4)
Health insuranced
Uninsured	32/508 (6.3)	39/496 (7.9)
Medicaid	352/496 (71.0)	332/486 (68.3)
Medicare	132/500 (26.4)	108/484 (22.3)
Government insurance	20/503 (4.0)	20/488 (4.1)
Private insurance	82/502 (16.3)	87/494 (17.6)
Living in a rural areae	253/508 (49.8)	250/495 (50.5)
State
Arkansas	127/508 (25.0)	119/496 (24.0)
Michigan	176/508 (34.6)	178/496 (35.9)
Washington	205/508 (40.4)	199/496 (40.1)
Clinical characteristics
Positive screen for bipolar disorder	186/508 (36.6)	181/496 (36.5)
Positive screen for posttraumatic stress disorder	494/508 (97.2)	484/496 (97.6)
Alcohol usef
Low risk	409/502 (81.5)	402/491 (81.9)
Risky	52/502 (10.4)	48/491 (9.8)
Harmful	9/502 (1.8)	14/491 (2.9)
Severe	32/502 (6.4)	27/491 (5.5)
Drug use
No drug use	306/500 (61.2)	297/488 (60.9)
Low	108/500 (21.6)	112/488 (23.0)
Moderate	53/500 (10.6)	42/488 (8.6)
Substantial	24/500 (4.8)	24/488 (4.9)
Severe	9/500 (1.8)	13/488 (2.7)
Physical health comorbidities, mean (SD)	4.0 (2.6)	4.0 (2.8)
Self-reported perceived need for treatment	452/491 (92.1)	428/484 (88.4)
Past use of psychotropic medication	420/493 (85.2)	417/483 (86.3)
Current use of psychotropic medication at enrollment	350/492 (71.1)	341/483 (70.6)
Past use of psychotherapy	390/494 (79.0)	381/484 (78.7)

Abbreviations: TCC, telepsychiatry collaborative care; TER, telepsychiatry/telepsychology–enhanced referral.

^aData available for 505 patients in the TCC group and 495 in the TER group.

^bBaseline and 12-month follow-up data.

^cTwelve-month follow-up data.

^dNot mutually exclusive groups.

^eDetermined using Rural-Urban Commuting Area (RUCA) codes.

^fMeasured using the Alcohol Use Disorders Identification Test.

A total of 464 of 508 patients randomized to TCC (91.3%) had 1 or more care manager encounters, and among these patients, the mean (SD) number of care manager encounters was 10.3 (7.9). A total of 403 (79.3%) had 1 or more BA encounters, and of these, the mean (SD) number of encounters was 9.6 (7.7). A total of 389 (76.6%) had a telepsychiatry consultation, and of these, the mean (SD) number of encounters was 1.4 (0.9). Overall, the TCC sample used 1.06 telepsychiatry encounters per randomized patient. In addition, 447 patients (88.0%) had a case review by the care manager and telepsychiatrist, and of these, the mean (SD) number of encounters was 6.4 (4.1). A total of 388 of 496 patients randomized to TER (78.2%) had 1 or more telepsychiatry encounters, and of these patients, the mean (SD) number of encounters was 4.3 (3.1). A total of 223 (45.0%) had 1 or more telepsychology encounter, and of these, the mean (SD) number of encounters was 6.4 (4.4). Overall, the TER sample used 6.4 telepsychiatry/telepsychology encounters per randomized patient.

Baseline MCS scores were 2 SDs below the US mean; the mean (SD) MCS scores were 39.7 (14.1) and 41.2 (14.2) in the TCC and TER groups, respectively. MCS scores were not clinically or significantly different between those randomized to TER and TCC at 12 months (β = 1.0; 95% CI, −0.8 to 2.8; P = .28; Figure 2). Treatment effects measured by MCS were not significantly different across age, sex, race or ethnicity, screening results, or baseline MCS scores. Analyses examining the sensitivity of results to missing-at-random assumption violations found that even under extreme violations, bias was likely to be very small (eFigure 2 in Supplement 2). Patients in both groups experienced large and clinically meaningful improvements in MCS scores from baseline to 12 months (TCC: Cohen d = 0.81; 95% CI, 0.67 to 0.95; TER: Cohen d = 0.90; 95% CI, 0.76 to 1.04; Figure 3). While we could rule out a null effect, group differences in PCL-5 scores were not clinically meaningful (β = –5.1; 95% CI, −7.7 to −2.4; Figure 2). Both groups experienced large clinically meaningful decreases in PCL-5 scores from baseline to 12 months (TCC: Cohen d = −0.72; 95% CI, −0.84 to −0.59; TER: Cohen d = −0.96; 95% CI, −1.12 to −0.88; Figure 3). Group differences in the proportions reporting euthymic mood were near zero (Figure 2), and both groups had large increases in the proportion euthymic by 12 months (TCC: adjusted difference in proportion, 0.17; 95% CI, 0.09 to 0.25; TER: adjusted difference in proportion, 0.19; 95% CI, 0.10 to 0.28; Figure 3). Group differences for the remaining secondary outcomes were not clinically meaningful, and patients in both groups exhibited clinical improvement (Figure 3). For the second-stage randomization, 228 trial participants (46.0%) did not engage in TER and were rerandomized. There was no meaningful between-group difference in medication engagement between those randomized to TER and PER (adjusted difference in proportion, −0.03; 95% CI, −0.15 to 0.07), but a substantially greater proportion randomized to PER had a psychotherapy encounter (adjusted difference in proportion, 0.09; 95% CI, 0.02 to 0.16). However, there was no significant between-group difference in 12-month MCS scores between those randomized to TER and PER (β = 2.0; 95% CI, −1.7 to 5.7; P = .29; eFigure 3 in Supplement 2) or other secondary outcomes.

Figure 2. Observed and Adjusted Outcomes for Stage 1 of the SMART Trial.

GAD-7 indicates General Anxiety Disorder-7; ISS, Internal State Scale; MCS, Veterans RAND 12-item Health Survey Mental Component Summary; NA, not applicable; PCL-5, PTSD Checklist-5; PTSD, posttraumatic stress disorder; SCL-20, Hopkins Symptom Checklist; TCC, telepsychiatry collaborative care; TER, telepsychiatry/telepsychology–enhanced referral.

^aFor clarity, the response set for 3 items was condensed, and scoring was adjusted accordingly.

^bEuthymic mood state was modeled with a binary logistic regression; point estimates represent marginal probabilities at the sample mean of all covariates, and the effect size shown on the dot plot is the adjusted risk difference.

Figure 3. Symptom Trajectories and Within-Group Effect Sizes by Stage 1 Intervention Condition.

GAD-7 indicates General Anxiety Disorder-7; ISS, Internal State Scale; MCS, Veterans RAND 12-item Health Survey Mental Component Summary; PCL-5, PTSD Checklist-5; PTSD, posttraumatic stress disorder; SCL-20, Hopkins Symptom Checklist; TCC, telepsychiatry collaborative care; TER, telepsychiatry/telepsychology–enhanced referral.

Perceived access to mental health, as measured by APAC scores, were not significantly different between those randomized to TER and TCC at 12 months (β = 0.11; 95% CI, −0.03 to 0.25; P = .10). Patients in both groups experienced statistically significant increases in perceived access from baseline to 12 months (TCC: Cohen d = 0.29; 95% CI, 0.17 to 0.41; TER: Cohen d = 0.43; 95% CI, 0.31 to 0.55). Results from the mediation analysis (Table 2) indicated no meaningful differences in self-reported medication engagement between those randomized to TER and TCC at 12 months (adjusted difference in proportion, −0.03; 95% CI, −0.09 to 0.03). However, patients randomized to TER averaged fewer psychotherapy encounters than those randomized to TCC (β = −4.2; 95% CI, −5.0 to −3.5). The effect of psychotherapy engagement on 12-month MCS scores was different between groups (β = 0.55; 95% CI, 0.26 to 0.84). For patients randomized to TER, each additional psychotherapy encounter was associated with an improvement in the 12-month MCS score of 0.33 (β = 0.41; 95% CI, 0.16 to 0.66). For patients randomized to TCC, the association was small and not significantly different from zero (β = −0.14; 95% CI, −0.28 to 0).

Table 2. Observed and Model-Based Marginal Mediation Effects.

Variable	TCC			TER			Adjusted difference (95% CI)
	Patients, No. (%)	Count or %	Model-based estimate (95% CI)	Patients, No. (%)	Count or %	Model-based estimate (95% CI)
Intervention effect on mediators (A paths)
Psychotherapy engagement
Baseline
Change from baseline to 6 mo	508 (100)	4.4	3.9 (3.5 to 4.3)	496 (100)	1.8	1.5 (1.3 to 1.8)	−2.4 (−2.8 to −1.9)
Change from baseline to 12 moa	508 (100)	7.6	6.8 (6.0 to 7.5)	496 (100)	2.9	2.5 (2.1 to 2.9)	−4.2 (−5.0 to −3.5)
Medication engagement
Baseline	486 (95.7)	0.60%	NA	479 (96.6)	0.61%	NA	NA
Change from baseline to 6 mo	353 (69.5)	0.72%	0.15 (0.10 to 0.20)	328 (66.1)	0.69%	0.12 (0.07 to 0.17)	−0.03 (−0.09 to 0.04)
Change from baseline to 12 mo	305 (60.0)	0.69%	0.06 (0.02 to 0.10)	296 (59.7)	0.64%	0.04 (0 to 0.07)	−0.03 (−0.09 to 0.03)
Mediator effects on MCS (B paths)
Psychotherapy engagement and MCS	NA	NA	–0.14 (–0.28 to 0)	NA	NA	0.41 (0.16 to 0.66)	0.55 (0.26 to 0.84)
Medication engagement and MCSa	NA	NA	–0.63 (–2.59 to 1.34)	NA	NA	0.57 (−1.49 to 2.62)	1.19 (−1.68 to 4.06)
Controlled direct effect (C′ path)
Change in MCS from baseline to 12 mo	NA	NA	10.5 (8.3 to 12.8)	NA	NA	8.6 (6.5 to 10.6)	−2.0 (−4.9 to 1.0)
Indirect and total effects
12 mo Indirect via psychotherapy	NA	NA	NA	NA	NA	NA	2.0 (0.8 to 3.1)
12 mo Indirect via medication	NA	NA	NA	NA	NA	NA	0.1 (−0.1 to 0.2)
12 mo Total effect	NA	NA	NA	NA	NA	NA	0.1 (−2.5 to 2.6)

Abbreviations: MCS, Veterans RAND 12-item Health Survey Mental Component Summary; NA, not applicable; TCC, telepsychiatry collaborative care; TER, telepsychiatry/telepsychology–enhanced referral.

^aThe number of therapy sessions could not decrease, whereas medication usage could either increase or decrease. Therefore, effects of therapy were assumed to be cumulative, whereas the effects of medication were assumed to be dependent on continued use. Psychotherapy engagement was modeled as a count variable with a negative binomial link. Medication engagement was modeled as a binary variable with a logit link. Both were back-transformed to provide marginal effects in terms of counts (psychotherapy) and probabilities (medication).

Discussion

Primary care patients enrolled in the trial had baseline MCS scores 2 SDs below the national mean, a level of functioning that is lower than typically seen in specialty mental health settings.^{37,38,39,40,41} TCC and TER both improved perceived access and treatment engagement, with more than three-fourths of patients attending telepsychiatry appointments. For those not engaging in TER, telephone-based outreach (PER) significantly increased engagement in psychotherapy (but not pharmacotherapy), although clinical outcomes were not improved.

Patients in both the TCC and TER arms experienced fewer adverse effects from psychotropic medications and statistically significant and clinically meaningful improvements in outcomes. These results are in sharp contrast to the Primary Care Research in Substance Abuse and Mental Health for Elderly (PRISM-E) trial, which compared integrated and referral care and found low levels of engagement⁴² and little clinical improvement.⁴³ TCC represents a more intensive intervention than the integrated care provided in the PRISM-E trial, and TER was virtually colocated in primary care whereas the PRIMS-E trial used off-site referrals. TCC and TER also both used measurement-based care, which the PRISM-E trial interventions did not.

The improvement in clinical outcomes is unlikely to reflect regression to the mean for 3 reasons. First, inclusion criteria were based on highly sensitive screeners administered during routine annual wellness visits rather than on structured diagnostic interviews administered to patients seeking treatment for a new episode of care. Second, the 9-point to 10-point increase in the MCS represents an improvement of a standard deviation and is considerably higher than the 3-point to 5-point minimum clinically important difference.⁴⁴ We are aware of only 2 mental health trials in which the intervention group experienced a greater than 9-point improvement in MCS.^7,8 Third, in the usual care groups of BD and PTSD trials, MCS scores do not improve,^37,45,46,47 and with one exception,⁴⁷ disorder-specific symptom severity does not improve.^{9,37,45,48,49,50,51}

Mediation analysis suggested that engagement in telepsychologist-delivered CPT and/or CBT in the TER arm, but not care manager-delivered BA in the TCC arm, was positively associated with MCS improvement. These results are consistent with a meta-analysis of PTSD psychotherapy trials, which found that CPT has a larger effect size than other psychotherapies.⁵²

Importantly, there were no clinically meaningful differences in outcomes between patients randomized to TCC and TER and no evidence of treatment heterogeneity. From a health care system perspective, results suggest that clinical leadership should implement whichever evidence-based practice is most sustainable. TCC is billable under new billing codes.⁵³ From a societal perspective, TCC should be incentivized by policy makers because it leverages scarce telepsychiatrist capacity through consultation and case-review.⁵⁴ Telepsychiatry encounters were 3-fold (mean [SD] of 4.3 [3.1] vs 1.4 [0.9]) greater in TER than TCC.

Limitations

The SPIRIT trial was the largest mental health trial conducted in rural primary care clinics and one of the largest trials ever conducted in FQHCs. However, there were some limitations. The trial design did not include a usual care group. Survey follow-up rates at 6 months and 12 months were relatively low. However, there were only slight differences between survey completers and noncompleters (eTable in the Supplement), and results were not sensitive to alternative assumptions about missing data. In addition, the SPIRIT trial made some compromises regarding pragmatism by using research funds to provide some of the clinical services and using telepsychiatrists and telepsychologists from state medical schools.

Conclusions

In summary, implementing TCC and/or TER in primary clinics in rural and underserved areas increased access to and engagement in effective treatments and substantially improved outcomes. By leveraging scarce telepsychiatrist capacity, TCC is able to serve more patients than TER.

Supplement 1.

Trial protocol.

Supplement 2.

eFigure 1. Path diagram for the mediation analysis.

eFigure 2. Sensitivity of treatment effects across size and proportion of MAR violations.

eFigure 3. Observed and adjusted outcomes for stage 2 of the SMART trial.

eTable. Dropout analysis.

Supplement 3.

Data sharing statement.