Determinants of remote measurement‐based care uptake in a safety net outpatient psychiatry department as part of learning health system transition

Abstract

Introduction

Behavioral measurement‐based care (MBC) can improve patient outcomes and has also been advanced as a critical learning health system (LHS) tool for identifying and mitigating potential disparities in mental health treatment. However, little is known about the uptake of remote behavioral MBC in safety net settings, or possible disparities occurring in remote MBC implementation.

Methods

This study uses electronic health record data to study variation in completion rates at the clinic and patient level of a remote MBC symptom measure tool during the first 6 months of implementation at three adult outpatient psychiatry clinics in a safety net health system. Provider‐reported barriers to MBC adoption were also measured using repeated surveys at one of the three sites.

Results

Out of 1219 patients who were sent an MBC measure request, uptake of completing at least one measure varied by clinic: General Adult Clinic, 38% (n = 262 of 696); Substance Use Clinic, 28% (n = 73 of 265); and Transitions Clinic, 17% (n = 44 of 258). Compared with White patients, Black and Portuguese or Brazilian patients had lower uptake. Older patients also had lower uptake. Spanish language of care was associated with much lower uptake at the patient level. Significant patient‐level disparities in uptake persisted after adjusting for the clinic, mental health diagnoses, and number of measure requests sent. Providers cited time within visits and bandwidth in their workflow as the greatest consistent barriers to discussing MBC results with patients.

Conclusions

There are significant disparities in MBC uptake at the patient and clinic level. From an LHS data infrastructure perspective, safety net health systems may need to address the need for possible ways to adapt MBC to better fit their populations and clinical needs, or identify targeted implementation strategies to close data gaps for the identified disparity populations.

1. INTRODUCTION

Measurement‐based care (MBC) uses validated instruments to systematically assess patients' symptoms, functioning, or quality of life to guide treatment planning ¹ , ² and can be effective across a wide range of clinical settings, age groups, diagnoses, and treatments. ³ MBC can outperform standard care for a wide range of behavioral health conditions and treatment settings and may work particularly well for people who have not responded to treatment. ³ However, many real‐world barriers remain to MBC implementation, particularly in safety net settings. Fewer than 20% of behavioral health clinicians use MBC at all, ³ and only 5% use MBC every 1‐2 sessions as intended. ⁴

MBC has been proposed as a key strategy to meet the rising demand for mental health services, given its potential to inform treatment planning and optimize system‐level resource allocation. ⁵ , ⁶ MBC's systematic nature may reduce some forms of bias in clinical judgment and help assess intervention effectiveness. ¹ For these reasons, MBC has been forwarded as a foundational tool to improve population health and equity in learning health system (LHS) efforts in under‐resourced settings. ⁷ By integrating MBC, LHSs may be able to process real‐time data to monitor behavioral health outcomes, access, and quality, leading to continuous learning cycles and gradual improvements in these outcomes. ⁷ However, gaps in MBC uptake among diverse populations may exacerbate healthcare disparities and represent a challenge for LHS efforts as a whole.

In‐person MBC (e.g., surveys in the waiting room or clinical encounter) can exclude people with high barriers accessing care, including transportation, language, physical challenges, or time constraints. ⁸ Remote MBC has high acceptability and feasibility, and may improve outcomes similarly to in‐person MBC; but has been associated with waning engagement over time. ⁵ In the context of the COVID‐19 pandemic and subsequent increases in telehealth visits, health systems are increasingly adopting digital platforms to remotely measure symptoms or other MBC outcomes asynchronously, thereby increasing patient access by allowing for flexibility to complete measures and saving time for providers during encounters. ⁹ , ¹⁰ , ¹¹ However, many barriers to implementing MBC may not be addressed by moving to remote MBC data collection alone. Patients with public insurance are 60% less likely to complete any measures compared to privately insured patients. ¹² When MBC is performed outside of an in‐person visit, patients may lack the time or maybe too ill to complete instruments. ³ Providers may view MBC as no more useful than routine clinical assessment, or an impediment to establishing rapport with patients. ³ Health systems may lack resources to support provider training and implementation. ³

Given numerous barriers to behavioral MBC implementation generally, and that little is known about drivers of uptake of remote MBC, questions remain about large‐scale data collection in behavioral health MBC as part of population‐level LHS goals for safety net systems. In diverse safety net systems, it is especially critical to identify which patient populations are more likely to benefit from MBC and which experience barriers that limit the benefit of these system‐level interventions. There has been insufficient research demonstrating MBC's effectiveness in under‐resourced safety net systems overall, and ongoing uncertainty regarding how to implement MBC so that it reduces, rather than exacerbates, health inequities.

2. QUESTIONS OF INTEREST OR RESEARCH INTERESTS

To identify potential MBC data collection challenges and health equity gaps in safety net settings, this study used electronic health record (EHR) data to identify populations with lower MBC completion rates by patient and clinic factors, and provider surveys to identify key provider‐reported barriers at a safety net setting during initial MBC implementation.

3. METHODS

3.1. MBC assessment

The Computerized Adaptive Test‐Mental Health (CAT‐MH®) is a web‐based tool consisting of a suite of validated measures that uses multi‐dimensional item response theory to screen for a range of mental health disorders and needs, including depression, mania, anxiety, psychosis, substance use, post‐traumatic stress disorder (PTSD), attention‐deficit hyperactivity disorder (ADHD), suicide risk, and quality of life. ¹³ CAT‐MH® assessments are adaptive; an individual's response to a question is used to determine the next question, allowing for more efficient estimation of the resultant severity scores. In controlled research studies, each module has been shown to take an average of 2 min to complete, for a total of 18 min for all modules. ¹⁴

3.2. MBC roll‐out and workflow

MBC using CAT‐MH® was piloted at a diverse academic public safety‐net system that serves approximately 140 000 patients annually, of whom 65% are publicly insured or uninsured, 45% are insured under risk‐based payment contracts, and more than half speak a language other than English at home. The Department of Psychiatry provides acute inpatient and emergency care as well as ambulatory care for adults and children (with an estimated 104 000 outpatient behavioral health visits provided in 2023). CAT‐MH was implemented at three outpatient psychiatry clinics: “General Adult Clinic” in April 2021 (general adult mental health clinic providing therapy and psychopharmacology), “Substance Use Clinic” in July 2021 (specialty substance use clinic providing medication management, recovery groups, and one‐on‐one psychotherapy), and “Transitions Clinic” in July 2021 (for people transitioning from psychiatric hospitalization or emergency care into outpatient care, providing real‐time assessments, time‐limited psychotherapy and psychopharmacology, and connections to longer‐term care).

Each clinic implemented a workflow in which eight of the CAT‐MH® modules (excluding the suicide risk module) were ordered for patients with upcoming scheduled appointments. When an order was placed (approximately 1 week before the appointment), patients received a patient portal message containing a weblink to complete CAT‐MH® on their own electronic device (computer, tablet, or smartphone) outside of a clinical encounter. The MBC process was designed such that after the initial complete CAT‐MH® assessment, routine follow‐up assessments were requested only for modules on which initial assessment scores were moderate or high severity. The complete CAT‐MH® was to be re‐assessed only annually thereafter. This was done to reduce patient burden from repeat testing and allow patients and providers to focus on key clinical issues.

Patient eligibility was restricted to patients who were enrolled in the patient portal and who spoke English or Spanish (the languages for which CAT‐MH® was available at the time of roll out).

3.3. Data collection

We extracted EHR data from the first 6 months of roll‐out at each clinic, including CAT‐MH® order date, whether or not the patient completed the CAT‐MH® assessment, and the clinic where the order was placed. We merged these data with patient‐level identifiers from the EHR, including patient age as of 01 January 2021, sex on record, race and ethnicity, language of care, and mental health diagnoses (from which we created four categorical variables: (1) depressive disorders, (2) anxiety and/or PTSD, (3) schizophrenia and/or bipolar disorder, and (4) substance use disorders; these were created based on Medicare and Medicaid Services Chronic Conditions Warehouse indicators, or CCW; See Appendix S1 for diagnostic codes). From these data, we created a patient‐level dataset that included patient demographics, the total number of CAT‐MH® orders the patient received, and the total number of completed orders. EHR data were extracted from the analytics database using Structured Query Language (SQL). EHR data cleaning, data restructuring, and data analyses were performed using StataMP 17.0.

Additionally, at the General Adult Clinic only (first pilot clinic), providers were invited to participate in repeated quality improvement surveys to understand primary barriers and facilitators of implementation: Series 1 (months 1‐3) and Series 2 (months 4‐6); each issued six times, with 2 weeks between survey rounds. Series 1 asked providers about their ease of finding and interpreting CAT‐MH® data in the EHR, the utility of using CAT‐MH data to collaborate on treatment decisions with patients, and the feasibility of incorporating CAT‐MH data collection and review into workflows. Series 2 focused on provider and patient adoption of CAT‐MH, with questions focused on how often (in the prior 2 weeks) providers encouraged CAT‐MH completion, discussed CAT‐MH with patients, or knew of a patient completing CAT‐MH; as well as asking about eight potential barriers to discussing CAT‐MH related to feasibility, acceptability, and perceived appropriateness and utility.

This study was designated as Quality Improvement by the health system's Institutional Review Board.

3.4. Statistical analysis

Patient demographic, mental health, and CAT‐MH® order data were summarized in the aggregate, and differential distribution by the clinic was tested (chi‐squared for categorical variables; multivariable means tests for continuous variables; see Table 1).

TABLE 1.

Patient characteristics.

	General Adult Clinic	Substance Use Clinic	Transitions Clinic	All Clinics
	M (SD)	M (SD)	M (SD)	M (SD)
Age a	38.6 (13.4)	41.0 (11.9)	34.4 (12.5)	38.2 (13.1)
Orders per patient	2.9 (1.6)	2.0 (1.1)	1.9 (1.1)	2.5 (1.5)

	n (%)	n (%)	n (%)	n (%)
Race/ethnicity
White	363 (52.2)	197 (74.3)	141 (54.7)	701 (57.5)
Hispanic/Latino	136 (19.5)	14 (5.3)	23 (8.9)	173 (14.2)
Portuguese or Brazilian	63 (9.1)	19 (7.2)	26 (10.1)	108 (8.9)
Black	70 (10.1)	25 (9.4)	50 (19.4)	145 (11.9)
Other b	64 (9.2)	10 (3.8)	18 (7.0)	92 (7.5)
Language of care
English	631 (90.7)	264 (99.6)	256 (99.2)	1151 (94.4)
Spanish	65 (9.4)	1 (0.4)	2 (0.8)	68 (5.6)
Sex in EHR
Female	496 (71.3)	126 (47.6)	164 (63.6)	786 (64.5)
Male	200 (28.7)	139 (52.5)	94 (36.4)	433 (35.5)
Mental health diagnoses
Depressive disorders	282 (40.5)	42 (15.9)	95 (36.8)	419 (34.4)
Anxiety/PTSD	321 (46.1)	32 (12.1)	86 (33.3)	439 (36.0)
Schizophrenia/bipolar	136 (19.5)	38 (14.3)	67 (26.0)	241 (19.8)
Substance use disorders	41 (5.9)	189 (71.3)	23 (8.9)	253 (20.8)

Note: Chi² tests were used to test differences in distributions of categorical data, and multivariable means comparison tests for continuous data (mvtest in Stata). All P < 0.0001.

^a Age is calculated as of 1 January 2021.

^b “Other” includes—Asian/South Asian, Other Race, Unknown, or Missing.

To understand how patient‐ and clinic‐level factors were associated with CAT‐MH® order completion, we conducted two logistic regression models. The first model adjusted for clinic, age, race and ethnicity, sex on record, language of care, and the presence of any of four mental health diagnoses. The second model also included an adjustment for orders per patient over the 6‐month period in which each clinic was eligible. All tests were two‐sided and used a cut‐off value of P = 0.05 to denote statistical significance.

4. RESULTS

Over the first 6 months of implementation at each site, 1219 patients had at least one CAT‐MH® module ordered across the three clinics. The average number of orders per patient was observed to be 2.5 (SD = 1.5) but differed across the three clinics (General Adult Clinic = 2.9, Substance Use Clinic =2.0, and Transitions Clinic = 1.9, P < 0.001). The number of orders that were completed per patient was found to be 0.39 (SD = 0.69) across all sites. Of patients who completed at least one order, an average of 3.0 (SD = 1.6) orders were placed and 1.26 (SD = 0.67) orders were completed. Patients completing at least one CAT‐MH® differed by clinic: General Adult Clinic, 38% (n = 262 of 696 patients who had orders placed); Substance Use Clinic, 28% (n = 73 of 265); and Transitions Clinic, 17% (n = 44 of 258).

4.1. Patient uptake of MBC based on EHR data

Table 1 provides demographic characteristics for our sample. The mean age of our sample was 38.2 years ± 13.1. The sample included 701 patients (57.5%) identifying as White, 173 (14.2%) identifying as Hispanic/Latino, 108 (8.9%) identifying as Portuguese or Brazilian, and 145 (11.9%) identifying as Black. Another 92 patients (7.5%) were included in an Other category, which included 30 Asian/South Asian patients, 26 patients identifying as Other Race, 14 of Unknown race, and 22 patients with missing race data. General Adult Clinic patients were more likely to be Hispanic/Latino, Spanish speaking, female, and in treatment for anxiety and/or PTSD or depression, compared to the other two clinics. The Substance Use Clinic had the highest proportion of White patients, male patients, patients in treatment for substance use disorder, and the highest average patient age. The Transitions Clinic served the highest percentage of Black patients, had the youngest average patient age, and served the highest volume of patients with schizophrenia or bipolar disorder.

Model 1 in Table 2 displays the results of the logistic regression model for the likelihood of completing at least one CAT‐MH® among patients with complete data (n = 1197). This model controlled for clinic, age, race and ethnicity, language, sex, and mental illness diagnoses. Increased age was associated with lower odds of completing CAT‐MH® (OR 0.97 per 1 year increase, 95% CI 0.96‐0.98). Portuguese or Brazilian, Black, and Spanish‐speaking patients were significantly less likely to complete CAT‐MH, with the largest effect size seen for Spanish‐speaking patients compared to English‐speaking patients (OR 0.05; CI 0.01‐0.23). Transitions Clinic patients were less likely to complete CAT‐MH than patients from the other two clinics. All mental health diagnostic categories, except the substance use disorder category, were associated with greater likelihood of completing at least one measure, with relatively higher odds ratios for depressive disorders (OR 1.95) and anxiety and/or PTSD (OR 1.98) compared to Schizophrenia and Bipolar disorders (OR 1.57).

TABLE 2.

Multivariable logistic regression model of order completion.

	Model 1			Model 2
	OR	95% CI	p	OR	95% CI	p
Clinic (General Adult Clinic = Reference)
Substance Use Clinic	0.67	[0.43‐1.05]	0.083	0.77	[0.49‐1.23]	0.274
Transitions Clinic	0.28	[0.19‐0.41]	<0.001	0.34	[0.23‐0.51]	<0.001
Age	0.97	[0.96–0.98]	<0.001	0.97	[0.96–0.98]	<0.001
Race/ethnicity (White = Ref)
Hispanic/Latino	0.86	[0.55‐1.33]	0.497	0.86	[0.55‐1.35]	0.511
Portuguese or Brazilian	0.57	[0.35‐0.92]	0.021	0.61	[0.38‐1.00]	0.049
Black	0.46	[0.29‐0.73]	0.001	0.46	[0.29–0.73]	0.001
Other	0.77	[0.44‐1.37]	0.375	0.82	[0.46‐1.47]	0.505
Sex (Male = Ref)
Female	1.30	[0.98‐1.74]	0.072	1.30	[0.97‐1.75]	0.076
Language (English = Ref)
Spanish	0.05	[0.01‐0.23]	<0.001	0.05	[0.01–0.22]	<0.001
Mental health diagnoses
Depressive disorders	1.95	[1.45‐2.63]	<0.001	1.58	[1.15‐2.17]	0.005
Anxiety/PTSD	1.98	[1.47‐2.65]	<0.001	1.58	[1.15‐2.15]	0.004
Schizophrenia/bipolar	1.57	[1.10‐2.24]	0.013	1.24	[0.85‐1.79]	0.268
Substance use disorders	1.51	[0.98‐2.33]	0.063	1.32	[0.85‐2.05]	0.223
Orders per patient				1.24	[1.12‐1.38]	<0.001

Note: Data and analyses are observed at the patient level.

Abbreviations: CI, confidence interval; OR, odds ratio.

Model 2 in Table 2 displays the results of the logistic model which also includes the continuous variable of orders per patient. Receipt of more CAT‐MH® orders was associated with a higher likelihood of completing at least one CAT‐MH® (OR 1.24 for 1 unit increase, 95% CI 1.12‐1.38). Controlling for the number of orders weakened the significance of the lower likelihood of Portuguese or Brazilian patients completing CAT‐MH compared to White patients (although this result remained significant at P < 0.05 cut‐off), and attenuated the strength of the relationship between the mental health diagnosis variables and CAT‐MH® completion outcomes.

4.2. Provider‐reported MBC barriers and uptake at the General Adult Clinic

Providers at the General Adult Clinic were issued surveys across the first 6 months of the CAT‐MH® roll‐out that provided additional qualitative context for implementation. Version 1 of the survey was repeated every 2 weeks during the first 3 months of the roll‐out (“T1” through “T6”) and Version 2 was implemented during months four through six of the roll‐out (“T7” through “T12”) for 12 total surveyed time points. Between 5 and 12 providers (out of an average of 12 sampled at each time point) responded to each survey.

Over the first 3 months, providers responded to four Likert‐Scale questions which ranged from 1 (Completely Disagree) to 5 (Completely Agree), and were also invited to add comments in a free‐text field to clarify any responses. Averages for each survey cycle are shown in Figure 1. Although averages went up and down slightly over time, providers were generally in agreement that finding CAT‐MH® results was easy to do (grand mean = 3.9), but had lower agreement about the ease of interpreting data (grand mean = 3.4), whether reviewing CAT‐MH® data helped them collaborate with patients on treatment decisions (grand mean = 3.2), and whether the amount of time to incorporate CAT‐MH® into their workflow was reasonable (grand mean = 3.3). Interestingly, over this 3‐month time period, the mean responses were lower for each prompt at the last survey compared to the first survey.

FIGURE 1.

Average provider‐reported MBC barriers at the General Adult Clinic (months 1‐3). The number of providers responding in each cycle was n = 10 (T₁), n = 5 (T₂), n = 9 (T₃), n = 11 (T₄), n = 10 (T₅), and n = 8 (T₆). Likert scale questions ranged from 1 “Completely Disagree” to 5 “Completely Agree.” Surveys were administered every 2 weeks for 3 months.

During months 4 through 6 of the roll‐out, providers responded to three yes/no questions asking about MBC uptake (their own and their patients') over the past 2 weeks. They also responded to eight total yes/no questions related to MBC knowledge barriers (3 questions), attitudinal barriers (3 questions), and systemic barriers (2 questions). Table 3 displays the results of these surveys for time points “T7” through “T12,” corresponding with the 7th through 12th instance of this biweekly survey. Although biweekly proportions varied over surveyed time points, a range of 44–100% of responding providers reported at least one instance over the prior 2 weeks when they had encouraged a patient to complete CAT‐MH®, observed that a patient had completed the survey, or discussed results with a patient. Providers were slightly more likely to endorse knowledge barriers related to interpreting or using CAT‐MH® results (17‐36% and 17‐33% of providers, respectively) vs not knowing what to say to patients about CAT‐MH®. Except at T7 (the first time point of this series), 0% of providers reported attitudinal barriers of their own; however, between 18% and 36% of providers believed that their patients did not want to discuss CAT‐MH® at any given survey timepoint. Finally, at almost every time point providers were most likely to endorse systemic barriers related to not having enough time in the patient visit (56–73%) or not having enough bandwidth to add a new initiative to their workflow (33–55%). Not having enough time in the patient visit was the highest endorsed barrier at every time point in months 4–6.

TABLE 3.

Provider‐reported MBC barriers and uptake in the General Adult Clinic (months 4‐6).

		Percent of Providers Endorsing Item
Survey Item		T7, n = 11	T8, n = 11	T9, n = 11	T10, n = 9	T11, n = 9	T12, n = 12
“In the past 2 weeks …”
Uptake of MBC	I have encouraged at least one patient to complete CAT‐MH.	73	82	55	44	67	58
	At least one of my patients has completed CAT‐MH.	73	73	45	67	78	100
	I have discussed CAT‐MH with at least one patient.	64	73	45	67	56	92
“Barriers to discussing CAT‐MH with my patients include…”
Knowledge barriers	I don't know what to say to patients about CAT‐MH.	0	18	27	11	11	8
	I don't know how to interpret CAT‐MH results.	27	27	36	22	22	17
	I don't know how to use the results as part of treatment.	27	27	27	22	33	17
Attitudinal barriers	I don't believe CAT‐MH gives useful information.	18	0	0	0	0	0
	I don't want to use CAT‐MH in my treatment.	9	0	0	0	0	0
	I don't think my patients want to discuss CAT‐MH.	27	18	36	33	33	25
Systemic barriers	I don't have time in the visit after addressing the patient's primary concerns.	73	73	73	56	56	67
	I don't have the bandwidth to add something to my workflow.	45	55	55	44	33	42

Note: Surveys were administered to providers every 2 weeks during months 4–6 of CAT‐MH MBC implementation at the General Adult Clinic.

5. DISCUSSION

Our study is one of a small number to quantitatively examine the likelihood of completing a remote MBC measure across demographic groups and outpatient treatment settings. We identified significant disparities in remote MBC uptake by ethnicity and race, language, age, and clinic type. Our findings suggest that diverse safety net LHSs may face challenges in addressing inequities when there are systemic differences in who completes measurements like the one studied in this analysis.

Drivers of observed disparities may include differences in cultural acceptability of symptom scales, internet and electronic device access, and/or psychosocial stressors that reduce patients' time or willingness to complete assessments asynchronously. After adjusting for age, race/ethnicity, language, sex, mental illness diagnoses, clinic, and orders sent, our study found lower rates of CAT‐MH® completion by Black or Portuguese/Brazilian patients. Prior research found Black patients were less likely than White patients to complete MBC assessments during the COVID‐19 pandemic, a disparity that had not existed previously with in‐person MBC. ¹⁵ Although the CAT‐MH® had been translated in a culturally sensitive way, ¹⁶ we found Spanish‐speaking patients were less likely to complete CAT‐MH® than their English‐speaking peers (including those of Hispanic/Latino descent). Possible reasons for this include difficulty creating translations that account for medical and cultural literacy, hesitancy to endorse neutral responses, and differing interpretations of closed‐ended questions among US Latinos. ¹⁷ , ¹⁸ , ¹⁹

Spanish speakers may face higher barriers using patient portals, which have generally been designed for English‐speaking audiences with high levels of health literacy in English. ²⁰ , ²¹ Sixty‐three percentage of US adults do not use patient portals, and non‐White patients are less likely to be offered patient portal registration. ²² Compared to White patients, Hispanic patients are less likely to find patient portals easy to understand or useful for monitoring their health. ²³ Improving patient‐centeredness of portals may change perceptions: 72% of Spanish‐speaking safety net patients (and especially Latinos) report that a bilingual patient portal would improve their care and patient–provider relationships. ²⁴ Using the patient portal to distribute CAT‐MH® may have contributed to lower completion among Spanish speakers. Patient engagement with portals can be increased by provider endorsement, accounting for health literacy, improving usability, and increasing personalization. ²⁵ Our health system is also exploring the use of coaches to engage patients on an individual level and address barriers to portal usage. Finally, about half as many Spanish speakers as English speakers are active in the health system's patient portal in this study, which has further upstream implications for access to digital or remote data collection protocols.

On average, each 1‐year increase in patient age was associated with 3% decreased odds of CAT‐MH® completion, consistent with previous literature showing older individuals face steeper challenges to digital health tools ²⁶ including due to attitudes, acceptability, disability, knowledge/familiarity, access, and cost. ²⁶ , ²⁷ Increased telehealth and patient portal use during the COVID‐19 pandemic highlighted the importance of addressing age‐related inequity in technology‐based health interventions. ²⁸ , ²⁹ , ³⁰ Demonstrating usefulness, providing tutoring, and designing interfaces to accommodate visual, cognitive, and motor impairments may be needed to increase remote MBC uptake among older patients. ²⁶ In our study, persons with female sex listed in their EHR were more likely to complete CAT‐MH® than those with male listed sex, which was not significant after covariate adjustment. Previous work finds that males have lower MBC questionnaire uptake than females, possibly due to lower provider engagement in MBC with male patients or higher trait conscientiousness in female patients. ¹⁵

Transition Clinic patients completed a CAT‐MH® less often. Illness severity can be a barrier to patient‐reported outcome measures. ³¹ Patients who received more CAT‐MH® orders were also more likely to complete at least one: these patients may have had more frequent visits or longer episodes of care. For example, the Transitions Clinic is more likely to see patients in between inpatient and routine outpatient care whose visit frequency and duration are more varied. In fully adjusted analyses, a presence of diagnosis for depressive disorders or anxiety and/or PTSD were associated with greater odds of CAT‐MH® completion. While these diagnostic categories do not necessarily reflect symptom severity, and are not necessarily mutually exclusive or exhaustive, our results do suggest remaining patient‐ and clinic‐level differences were not due to differences in diagnoses alone. LHSs implementing MBC in populations with higher symptom acuity, a large variation in mental health needs, or variable‐term treatment settings may find additional efforts are needed to ensure the timing and appropriateness of MBC are tailored to these patients.

Provider reports of challenges and opportunities in the first 6 months of MBC roll‐out provided valuable information for LHSs seeking to incorporate routine data collection to improve coordination and delivery of mental health care. Providers' mean responses to survey questions over the first 3 months generally declined, indicating that they potentially lost confidence in navigating and interpreting newly collected MBC data as they gained experience and faced actual challenges. The survey data patterns in months 4‐6 found that the greatest level of reported challenges was structural, having to do with available time in the visit or in the workflow. Believing that patients did not want to discuss MBC results was a commonly reported barrier. Prior studies have shown that visits for patients with more complex medical conditions take more time. ³¹ Provider‐reported barriers have significant implications for LHS data efforts related to health disparities within and across health systems, as patients with more complicated treatment needs, more access barriers, or more skepticism of quantitative surveys are also likely to require more provider time to encourage data collection and MBC uptake.

Just as there are gaps in the research literature that can contribute to perpetuating health inequities, LHS efforts to understand and address behavioral health access, quality, and outcomes are compromised when there are systematic differences in who completes MBC measures to begin with. At the same time, an LHS perspective and infrastructure can serve as one key mechanism for identifying and improving upon observed challenges to equitable MBC uptake. Maximizing the health equity impact of LHSs will require a quality improvement focus not just on equitable health outcomes themselves, but on equitable processes—including those processes that produce the very data that may help identify disparities and direct LHS activities.

5.1. Limitations and strengths

This analysis has several limitations and notable strengths. First, patient eligibility for MBC was limited to patient portal‐enrolled persons who had English or Spanish language of care and were engaged in mental health treatment at one of the three clinics within the 6‐month window roll‐out at each site. Many drivers of disparities in MBC access (initiating mental health treatment, patient portal enrollment, and accessing care in other languages) are upstream of this analysis. Second, while number of orders was a significant predictor of CAT‐MH® uptake, this finding may be conflated with more contact in the mental health system. Third, EHR data have important limitations, including an inability to assess differences in patients' mental health symptoms or severity within mental health diagnoses at the time that they were asked to complete the CAT‐MH®. Fourth, survey feedback was only gathered from General Adult Clinic providers because their feedback was used to try to address some barriers before roll‐out began at the subsequent two sites, but their perceived barriers may have differed from those providers at the other two clinics (focused on patients with substance use and transitional care needs). While a brief survey of patient barriers was assessed to attempt to inform operational decisions, response rates were too low to be conclusive, and the survey had some of the same constraints as the MBC tool itself; the team has since applied for a grant to support systematic data collection on patient barriers. Despite these limitations, this real‐world analysis over a longitudinal implementation study period quantifies important challenges in ensuring data for measuring and mitigating disparities in safety net systems is collected equally for all patient groups.

6. CONCLUSIONS

Our study found key differences in MBC uptake by patient demographics and adds to the existing literature on behavioral health MBC in diverse patient populations in safety net health systems. Providers reported time constraints and bandwidth constraints, which may have limited their ability to encourage patients to complete MBC. Our study points to important areas of consideration for LHSs seeking to build robust data infrastructures to study mental health care disparities in diverse settings. Safety net health systems may need to adapt MBC to better fit their populations and use targeted implementation strategies for the populations shown to experience disparities during MBC implementation. Richer mixed methods data are needed to explore possible adaptations and implementation strategies and evaluate their ability to ensure equity in behavioral health MBC in LHSs.

CONFLICT OF INTEREST STATEMENT

The authors have no conflicts of interest with the work described.

Supporting information

Appendix S1. Supporting Information.

Aldis R, Rosenfeld LC, Mulvaney‐Day N, et al. Determinants of remote measurement‐based care uptake in a safety net outpatient psychiatry department as part of learning health system transition. Learn Health Sys. 2024;8(Suppl. 1):e10416. doi: 10.1002/lrh2.10416