Learning from Misses: Evaluating a Clinical Decision Support for Chronic Pain Management in Primary Care

Abstract

Objective

This study aimed to assess the effect of a passive, opt-in electronic health record (EHR)-based clinical decision support (CDS), the Chronic Pain OneSheet, on guideline-recommended chronic pain management in primary care.

Methods

A pragmatic randomized controlled trial with a parallel group design was conducted between October 2020 and May 2022. Participants were 137 primary care clinicians (PCCs) treating qualifying patients with chronic pain at 25 primary care clinics within two academic health systems in the United States. PCCs were randomized in the EHR to have access to OneSheet or usual care. OneSheet aggregates guideline-relevant information in a single view and provides shortcuts to guideline-recommended actions (e.g., ordering urine drug screening [UDS] for patients prescribed opioids). We constructed five visit-level binary outcomes: (1) documenting pain-related goals; (2) documenting pain and function via Pain, Enjoyment of Life and General Activity (PEG) scale; (3) reviewing prescription drug monitoring programs (PDMPs); (4) ordering UDS; and (5) ordering naloxone. Analysis used generalized linear mixed models for each outcome.

Results

OneSheet access minimally increased rates of pain-related goal documentation (0.2 percentage point increase, p = 0.013), PEG scale documentation (0.7 percentage point increase, p < 0.001), and UDS orders (2.2 percentage point increase, p = 0.006). OneSheet access decreased the rate of ordering naloxone (0.5 percentage point decrease, p < 0.001). OneSheet access did not affect PDMP review rates (0.5 percentage point decrease, p = 0.382).

Conclusion

OneSheet access did not result in clinically significant improvements in guideline-recommended management of chronic pain in primary care despite a robust user-centered design incorporating clinician input and EHR integration. Several factors likely limited OneSheet effectiveness, including limited ability to target certain patient visits, workflow limits on data collection and ordering, and evolving COVID-19 and opioid epidemic-related policies and procedures. These findings highlight specific limitations of OneSheet and the broader challenges of implementing effective EHR-based CDS in complex health care environments.

Background and Significance

Improving the usability of electronic health records (EHRs) is essential for reducing primary care clinicians’ (PCCs) administrative burden and improving the quality of primary care.^1–3 Yet, PCCs are often unable to efficiently retrieve information in current EHRs, especially for patients with chronic pain.^4–6 Chronic pain treatment in primary care is complex due to its biopsychosocial nature, co-occurring conditions, nuanced risk–benefit tradeoffs between treatment options, and evolving clinical guidelines.^7–12 The 2016 and 2022 Centers for Disease Control and Prevention (CDC) guidelines recommend that PCCs make treatment decisions by assessing and considering various clinical information, including medical history, risks and benefits of specific pain treatments, and patient goals.^11,13,14 Notably, PCCs must complete these tasks and manage other patient needs within 20 minutes or less of the visit.

Clinical decision support (CDS) can synthesize relevant patient information or present clinical recommendations, making it a prime candidate for improving EHR usability and helping PCCs efficiently and effectively manage patients with chronic pain.^15–18 PCCs report that CDS can help with several tasks, including aggregating disparate information relevant to treatment risks, potential benefits, and patient outcomes and goals.^8,19 Several CDSs aimed at improving guideline-recommended pain care have been developed, demonstrating mixed success in affecting opioid prescribing, prescription drug monitoring program (PDMP) review, opioid agreement completion, and urine drug screen (UDS) testing rates.^20–28 Notably, most of these CDSs addressed only one aspect of chronic pain care (e.g., prescribing or screening) and were evaluated using observational study designs, thus limiting causal analysis of their effects. As an exception, a 2022 randomized controlled trial (RCT) employed a nudge-based CDS to streamline PDMP access, leading to increased rates of PDMP queries.²¹ However, no known studies have utilized an RCT design to assess the effect of a more comprehensive EHR-based CDS addressing multiple aspects of guideline-recommended chronic pain management in primary care.

Notably, while various CDSs show promise in affecting clinical care, their effectiveness is ultimately determined by complex implementation factors beyond the technology itself. Extensive literature related and unrelated to chronic pain, describes individual, organizational, social, and technical factors that affect the use and effects of CDS and other health information technology.^29–33 For instance, one predominant sociotechnical model describes eight interrelated dimensions involved in the design, implementation, use, and effects of health information technology. These eight dimensions encompass hardware and software, clinical content, human–computer interface, workflow and communication, internal organizational structures and external rules and regulations, as well as measurement and monitoring.³⁴ The sociotechnical model emphasizes the importance of considering each of these dimensions in CDS design and implementation in a specific clinical context, such as chronic pain management in primary care.

Objective

The purpose of this pragmatic RCT was to assess the effect of an EHR-based CDS, the Chronic Pain OneSheet (hereinafter referred to as “OneSheet”), on guideline-recommended chronic pain management in primary care. OneSheet was developed through user-centered design research, which included PCC interviews, patient observations, and prototype testing. OneSheet development considered the human–computer interface, primary care workflow, PCC communication needs, and internal and external regulations (e.g., internal opioid prescribing policies and national clinical guidelines).^8,35–39 Created during heightened concerns about opioid prescribing for chronic non-cancer pain and a 2016 CDC guideline publication, OneSheet addresses PCCs’ decision-making challenges stemming from information gaps and limited chronic pain management training. OneSheet consolidates guideline-relevant information in a single EHR view, including pain diagnoses, medications, and appointment history. OneSheet also streamlines recommended clinical actions with shortcuts for documenting patient goals, pain levels via the Pain, Enjoyment of Life and General Activity (PEG) scale, ordering naloxone and UDSs, and reviewing PDMP data. Built as a Navigator within the Epic EHR system, PCCs access OneSheet through available tabs (see Fig. 1).

Fig. 1.

Screenshot of OneSheet for a fictitious patient, annotated (data presented are imaginary). ©2025 Epic Systems Corporation.

Methods

This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines.⁴⁰ The Indiana University Institutional Review Board approved all study procedures before the study’s commencement, including informed consent for participating PCCs and a waiver of consent for patients. Patient data were already routinely collected in the Epic EHR system, presenting no additional privacy risks for patients. Furthermore, the high volume of patients with chronic pain seen daily made obtaining individual consent and the Health Insurance Portability and Accountability Act (HIPAA) authorization impractical. The study is registered with ClinicalTrials.gov (identifier: NCT04295135).

Setting and Participants

Our RCT was conducted at primary care clinics from two academic health systems based in Indianapolis, IN (health system 1) and Winston-Salem, NC (health system 2). All clinics used the Epic EHR system (Epic Systems Corp). PCCs were eligible as potential study participants if they were (1) actively seeing patients at a participating clinic; (2) licensed to prescribe medication (physicians, nurse practitioners, and physician assistants); and (3) trained in either family medicine, internal medicine, pediatric medicine, or their respective subspecialties. We enrolled PCCs in two phases (Phase 1: health system 1: August–October 2020; health system 2: September–November 2020; Phase 2: September–December 2021 for both health systems). Recruitment phase 2 was conducted to increase the sample size and enable new PCCs to participate in the study. Additional details of our recruitment strategy are reported elsewhere.^39,41

Randomization and Intervention

Our RCT used a parallel group trial design with a 1:1 allocation ratio. We randomized PCCs to OneSheet access (intervention group) or no OneSheet access-usual care (control group). A biostatistician team member randomized clinicians to intervention versus control using SAS statistical software, version 9.4. PCCs in the intervention group had access to OneSheet starting in October 2020 (health system 1) or November 2020 (health system 2).

PCCs in the intervention group received initial OneSheet-focused training with periodic follow-up usage tips sent by the research team (e.g., how to access OneSheet in Epic, how to enter PEG scores, etc.). They also received a non-interruptive EHR alert during all visits with patients with chronic pain. To minimize alert fatigue and administrative burden, OneSheet use was not required or prompted in any interruptive form. Instead, PCCs in the intervention group were trained on how to add OneSheet to their “favorites” within the EHR and instructed to use it when most relevant, according to their clinical judgment. Blinding was impossible in our pragmatic RCT because PCCs knew if they were in the intervention group via their OneSheet access. To facilitate additional training and follow-up from the study team, PCCs in the intervention group were also not blinded from the study team members. Of note, our RCT occurred exclusively during COVID-19 pandemic, which significantly altered primary care at both health systems and delayed recruitment and go-live by 6 months.

Data Collection and Patient Subgroups

To assess OneSheet effects on guideline-recommended care, we collected visit-level EHR data related to participating PCCs’ visits with patients with chronic pain. We identified these visits retrospectively using a predefined algorithm (see Fig. 2). We extracted EHR data from October 2019 (12 months prior to the OneSheet go-live) through May 2022 (18 months after OneSheet go-live). All visits with patients with chronic pain that occurred during this 30-month period were eligible for inclusion and analysis. PCCs reported their demographic characteristics (sex, race, ethnicity, clinical training credentials, and years in practice) at the time of enrollment.

Fig. 2.

An algorithm for identifying patients with chronic pain. Note: Graphic representation of inclusion criteria for identifying patients with chronic pain.

Since not all patients with chronic pain are prescribed opioids, we classified patients into two subgroups based on the presence of active long-term opioid therapy (LTOT) prescriptions. LTOT was defined for each patient visit as at least a 28-day supply of an opioid analgesic medication.¹³ The first group encompassed all patients with chronic pain, regardless of LTOT. For this group, we evaluated outcomes relevant for chronic pain management (e.g., documenting patients’ pain and function goals). The second group consisted of the subset of patients with chronic pain who were also on active LTOT. For patients on LTOT, we evaluated outcomes specifically relevant to opioid prescribing for chronic pain, such as PDMP review, UDS, and naloxone orders.

Outcomes

Using EHR data, we constructed five visit-level binary outcomes (1, guideline-concordant; 0, non-concordance) that reflected concordance with the 2016 CDC opioid-prescribing guideline recommendations. First, documenting pain-related goals in the last 90 days was drawn from the Epic Goals module. The study team reviewed all free-text goals and identified goals that were related to chronic pain and function. This outcome was evaluated for all patients with chronic pain. Second, documenting pain and function via PEG scale in the last 90 days was defined as a completed structured PEG scale within Epic flowsheets and was evaluated for all patients with chronic pain. Third, reviewing the PDMP in the last 90 days was defined via Epic’s structured record of visit-level PDMP access and evaluated for LTOT patients only. Fourth, ordering a UDS in the last calendar year was defined via a completed UDS order in Epic and evaluated for LTOT patients only. Finally, ordering naloxone within 7 days of the visit in which an opioid was prescribed was defined using the structured medication list and evaluated for LTOT patients only.

Each outcome was considered guideline-concordant if the PCC completed the clinical action (e.g., documenting pain-related goals) within the associated lookback period (i.e., within 90 days before the visit) or 1 week after the visit (a 7-day grace period). The lookback period ensured that visits were not recorded as “discordant” simply because tasks had been completed beforehand, and the grace period ensured guideline-recommended behavior that occurred shortly after a visit occurred. For PDMP review, there was no post-visit grace period because PDMP review is recommended by the 2016 CDC guidelines (and mandated by many states) to happen at the time of opioid prescribing.

As an example, if a LTOT patient had not had a UDS result in the last calendar year, but the PCC ordered a UDS on the day of the visit, the outcome would be considered guideline-concordant. Similarly, if the patient had a UDS result in the last calendar year, the outcome would be considered to have occurred because, at the time of the visit, the PCC was concordant with the 2016 CDC guideline recommendations for UDS review. If both (1) a UDS had not been completed in the last year and (2) the PCP did not order a UDS within 1 week of the visit, the outcome would be considered non-concordant.

Statistical Analysis

We used descriptive statistics to compare the demographic characteristics of PCCs assigned to the intervention group versus the control. We computed outcomes for each qualifying visit with study PCCs before OneSheet access was granted (baseline period: 12 months) to after OneSheet access was granted (18 months). We excluded patients who had visits with both intervention and control PCCs during the study period to ensure that outcomes were not compared across trial arms for the same patient. For our primary analysis, we included all PCCs recruited in both phases. For our sensitivity analysis, we limited PCCs recruited in phase 1 to analyze only participants with shared onset dates. This approach detects potential bias in our effect estimates that may arise from heterogeneous effects among “early” adopters (i.e., phase 1) relative to “later” adopters (i.e., phase 2).

To estimate the effect of OneSheet on our outcomes, we used generalized linear mixed models for each outcome that included patient demographics (gender, race, ethnicity), PCC random effects, and indicator variables for (a) site, (b) intervention and control PCCs, and (c) pre-/post-OneSheet access. Patient demographics were included due to their known influence on pain care experiences and prescribing practices.^42,43 We included an interaction term for intervention and post-OneSheet access, the coefficient for which estimates the causal effect of OneSheet on our outcomes. Statistical tests were two-tailed, and p-values <0.05 were considered statistically significant. Data analyses were performed from November 2022 to May 2024 using the tidy-verse, fixest, and lme4 R packages in RStudio.^44–46 To better understand the clinical significance of our findings, we converted our marginal effect estimates to the number of visits required for one additional guideline-recommended visit (100/Beta). This can be interpreted analogously to a visit-level “number necessary to benefit” calculation, which is frequently employed in RCTs.

Results

Participant Characteristics

We recruited 137 eligible PCCs practicing at the 25 participating clinics (244 PCCs were approached; see Fig. 3 for more details). Sixty-nine PCCs were randomly assigned to an intervention group (50.4%) with OneSheet access and 68 to the control group (49.6%). Most PCCs in the intervention group were medical or osteopathic physicians (68%), female (70%), White (75.4%), or not of Hispanic or Latino ethnicity (78%) with an average of 10 years of experience practicing medicine (standard deviation [SD]: 9.6 years; see Table 1). The credentials and sex of PCCs in the control group were like those assigned to the intervention group. PCCs in the intervention group accessed OneSheet 2,796 times during the trial period.

Fig. 3.

CONSORT participant flow diagram. Note: Graphic representation of the counts of participants included and excluded at each phase of the trial. CONSORT, Consolidated Standards of Reporting Trials; MOUD, opioid use disorder medication; PDMP, prescription drug monitoring program; PEG, Pain, Enjoyment of Life and General Activity; UDS, urine drug screen.

Table 1.

Baseline characteristics of participating primary care clinicians by study group (n = 137)

Characteristic	Total (n = 137)	Control (n = 68, 49.6%)	Intervention (n = 69, 50.4%)	p-Value
Clinical training credentials, n (%)				0.074
Physician	108 (73.4)	51 (75)	47 (68)
Physician assistant	19 (12.9)	5 (7)	14 (20)
Nurse practitioner	20 (13.7)	12 (18)	8 (12)
Sex, n (%)				0.257
Female	88 (64.3)	40 (59)	48 (70)
Male	49 (35.7)	28 (41)	21 (30)
Ethnicity, n (%)				0.372
Hispanic or Latino	12 (8.8)	8 (12)	4 (6)
Non-Hispanic or Latino	101 (73.7)	47 (69)	54 (78)
Prefer not to answer	24 (17.5)	13 (19)	11 (16)
Race, n (%)				0.302
Asian	9 (6.6)	2 (3)	7 (10.1)
Black or African American	10 (7.3)	6 (9)	4 (5.8)
White	104 (75.9)	52 (76)	52 (75.4)
More than one race	1 (0.7)	0 (0)	1 (1.5)
Prefer not to answer	13 (9.5)	8 (12)	5 (7.2)
Experience (SD)				0.750
Mean years actively practicing medicine		9.7 (9.4)	10.2 (9.6)

Abbreviation: SD, standard deviation.

Patient Demographics

Among patients who had a qualifying visit during the 18-month intervention (n = 36,577), the mean age was 55.7 years (SD: 15.7 years). Most patients with chronic pain were female (63.5%) and White (57.4%). Approximately 11% of patients with qualifying visits were on LTOT (n = 4,206). Most patients on LTOT were female (64.2%), White (69.9%), or of non-Hispanic Latino ethnicity (95.9%). Patients in the sample had an average of 12.9 (SD: 15.4 visits) qualifying primary care visits during the trial period (see Table 2).

Table 2.

Baseline demographic information of patients with chronic pain included in the trial

Characteristic	Patients overall (n = 36,577)	Patients on LTOT (n = 4,206)
Mean patient age (SD)	55.7 (15.7)	60.3 (14.3)
Qualifying primary care visits (SD), during the study period	12.9 (15.4)	12.0 (17.2)
Gender, n (%)
Female	23,231 (63.5)	2,702 (64.2)
Male	13,344 (36.5)	1,504 (35.8)
Unknown	2 (0.0)	0 (0.0)
Ethnicity, n (%)
Hispanic or Latino	3,320 (9.1)	115 (2.7)
Non-Hispanic or Latino	33,868 (89.9)	4,033 (95.9)
Unknown	389 (1.1)	58 (1.4)
Race, n (%)
Black or African American	11,626 (31.8)	1,079 (25.7)
White	20,985 (57.4)	2,941 (69.9)
Other race	3,966 (10.8)	186 (4.4)

Abbreviations: LTOT, long-term opioid therapy; SD, standard deviation.

Note: Please see Supplementary Table S2 (available in the online version only) for a presentation of patient information between intervention and control groups.

Primary Analysis Results

Figure 4 displays the estimates from our visit-level linear mixed models with the pre- and post-go-live treatment and control group averages for each outcome. The coefficients presented can be interpreted as percentage point changes in the probability of a guideline-recommended visit for a given outcome, accounting for secular trends and physician random effects in a manner that a simple pre–post comparison cannot (see Table 3). OneSheet access resulted in statistically significant increases in pain-related goal documentation (0.2 percentage point increase, p = 0.013), PEG documentation (0.7 percentage point increase, p < 0.001), and UDS orders (2.2 percentage point increase, p = 0.006). Additionally, OneSheet access decreased the rate of ordering naloxone (0.5 percentage point decrease, p < 0.001). OneSheet access did not affect PDMP review rates.

Fig. 4.

Estimated trial outcomes by study group and time point, using generalized linear mixed models. Blue line: intervention group (OneSheet access). Gray line: control group. Pre: baseline period (12 months before OneSheet access). Post: intervention period (18 months with OneSheet access). Pain-related goals: documenting pain-related goals in the last 90 days; PEG scale: documenting pain and function via Pain, Enjoyment of Life and General Activity (PEG) scale in the last 90 days; PDMP review: reviewing prescription drug monitoring program in the last 90 days; UDS orders: ordering urine drug screen results in the last calendar year; naloxone orders: ordering naloxone within 7 days of the visit. Alt Text: Five graphs illustrating trial outcomes pre- and postintervention, by study group. PDMP, prescription drug monitoring program; UDS, urine drug screening.

Table 3.

Estimated effects of OneSheet access on guideline-concordant prescribing for patients with chronic pain (n = 137 participants)

	Coals (n = 472,804)	PEC (n = 472,804)	PDMP (n = 50,434)	UDS (n = 50,434)	Naloxone (n = 50,434)
Effect estimates
Intervention group (vs. control)	0.001 (0.003)	−0.003 (0.003)	−0.005 (0.038)	−0.002 (0.038)	0.004 (0.002)
Post-OneSheet access	0.024a (0.001)	0.013a (0.001)	0.049a (0.004)	−0.062a (0.006)	0.008a (0.001)
Intervention: Post-OneSheet access	0.002b (0.001)	0.007a (0.001)	−0.005 (0.006)	0.022c (0.008)	−0.005a (0.001)
Patient gender (ref. = female)
Male	0.001c (0.0004)	−0.0005 (0.0004)	0.023a (0.003)	0.031a (0.004)	0.002c (0.001)
Unknown	−0.013 (0.023)	−0.019 (0.023)	–	–	–
Patient age at go-live	0.00001 (0.00002)	0.0002a (0.00001)	−0.001a (0.0001)	0.001a (0.0002)	−0.0001a (0.00003)
Patient race (ref = American Indian or Alaska Native)
White or Caucasian	−0.002 (0.003)	0.001 (0.003)	−0.110a (0.021)	−0.131a (0.029)	0.002 (0.004)
Asian	−0.006 (0.004)	−0.017a (0.004)	−0.185a (0.038)	−0.425a (0.052)	−0.002 (0.008)
Black or African American	−0.0003 (0.003)	0.001 (0.003)	−0.097a (0.022)	−0.111a (0.029)	−0.001 (0.004)
More than one race	−0.003 (0.004)	−0.002 (0.004)	−0.073c (0.025)	−0.144a (0.034)	0.013b (0.005)
Native Hawaiian or Other Pacific Islander	0.006 (0.005)	−0.003 (0.005)	−0.246a (0.040)	−0.433a (0.055)	−0.006 (0.008)
Unknown	0.002 (0.004)	−0.005 (0.003)	−0.149a (0.026)	−0.173a (0.035)	−0.001 (0.005)
Patient ethnicity (ref = Hispanic/Latino)
Non-Hispanic or Latino	0.001 (0.001)	0.002 (0.001)	−0.002 (0.013)	−0.001 (0.018)	−0.002 (0.003)
Unknown	−0.006c (0.002)	0.005b (0.002)	0.002 (0.016)	−0.039 (0.022)	0.001 (0.003)
Site (B vs. A)	0.027a (0.003)	0.031a (0.003)	−0.234a (0.038)	−0.280a (0.039)	−0.010a (0.002)
Constant	−0.008 (0.005)	0.016a (0.005)	0.943a (0.044)	0.604a (0.050)	0.013b (0.006)

Abbreviation: SD, standard deviation.

Note: Standard errors are in parentheses. Goals: documenting pain-related goals in the last 90 days; PEG: documenting pain and function via Pain, Enjoyment of Life and General Activity (PEG) scale in the last 90 days; PDMP: reviewing prescription drug monitoring program (PDMP) in the last 90 days; UDS: ordering urine drug screening (UDS) results in the last calendar year; MOUD: ordering opioid use disorder medications (MOUDs) within 7 days of the visit; naloxone: ordering Naloxone within 7 days of the visit.

^ap < 0.001.

^bp < 0.05.

^cp < 0.01.

Translating the marginal effects into the number of clinical visits, OneSheet access resulted in one additional pain-related goal documentation per 500 qualifying chronic pain visits, one extra PEG documentation per 143 visits, one additional UDS order per 46 visits, and one fewer naloxone order per 200 visits. Our sensitivity analyses using data from PCCs recruited in phase 1 only (n = 127) yielded similar magnitudes and directions of results (see Supplementary Table S1 [available in the online version only]).

Discussion

Our pragmatic RCT found that access to OneSheet did not result in clinically significant improvements in guideline-recommended chronic pain management in primary care despite a robust user-centered design incorporating clinician input and EHR integration. Several micro- and macrolevel social or technical factors likely contributed to the limited effects of OneSheet on assessed outcomes.

Microlevel factors included communication and workflow limits. Communication limits are based on OneSheet’s non-interruptive EHR alerts, which flag visits for all patients with chronic pain. This alert structure may have been too broad, hindering PCC’s ability to use OneSheet during visits, where it could be most valuable. Health care organizations may achieve greater effects by precisely alerting, incentivizing, or requiring the use of OneSheet during high-value visits, such as those with patients whose chronic pain care is not guideline-concordant. This approach would require adequate EHR data quality to assess guideline concordance and EHR rules and alert builds to assess guideline concordance in advance of a patient visit.

Workflow limits affected the documentation of pain-related goals and PEG scores emphasized in clinical guidelines,^14,47,48 but rarely systematically documented in primary care and not mandated by any local or national policies.^49,50 Despite providing training materials and working with individual clinics, competing priorities for staff time and workflow variations limited the documentation of pain-related goals and PEG scores. Our findings suggest that the availability of a CDS to centralize the collection of these patient-centered outcomes is insufficient. Systematic workflow redesigns, assigned responsibilities, or data collection mandates may be required complements to user-centered CDS, such as OneSheet. Empowering patients to report information ahead of visits could also ensure that OneSheet contains current pain-related goals and PEG score data. However, such patient-focused efforts would also need to consider patient usability and burden.

Similarly, workflow limitations may have limited the potential for larger positive effects of OneSheet on UDS orders for patients prescribed LTOT than observed in our analysis. Routine UDS ordering is one required clinical action for safe opioid prescribing practices.^11,13,14 Our findings align with prior evidence that CDS can improve PCC adherence to opioid risk mitigation strategies.^{22,23,25,51,52} However, our study’s lack of clinically meaningful effects raises questions about the power of optional CDS that require manual actions. Future studies should evaluate automated approaches, such as pending UDS orders, for clinician sign-off during visits with patients prescribed LTOT, as these may be more effective alternatives.

Evolving health systems rules and regulations related to COVID-19 and the opioid epidemic may have also influenced our findings. OneSheet implementation was delayed due to the COVID-19 pandemic. OneSheet eventually went live in late 2020, with participating health system policies restricting in-person training and communication. With PCCs, administrators, and staff focused on the COVID-19 response, attention to and implementation of OneSheet was likely diminished.

Finally, concurrent local and national shifts in opioid-related policies between the OneSheet design phase (2015–2019) and implementation (2020–2022) created a new regulatory environment, which may have also influenced our findings. For instance, contrary to prior literature,^22,27,28,52 we found reduced (but clinically insignificant) rates of naloxone ordering among PCCs in the intervention group, reflecting the shifting regulatory environment. Locally, one of the health systems implemented a vending machine-type naloxone dispensing service for patients. Nationally, standing orders issued by the state surgeon general, which eliminated naloxone prescription requirements, likely contributed to this finding.⁵³ However, these policies equally affected PCCs in the intervention and control groups; thus, the specific mechanisms driving this finding remain unclear.

Similarly, OneSheet’s lack of effect on PDMP review rates for patients prescribed LTOTreflects the evolving local and national policy environments. Both health systems integrated PDMP access into their EHRs shortly before the launch of our RCT, and regulatory mandates for PDMP reviews were becoming increasingly common. Despite OneSheet including an integrated PDMP link that PCCs reportedly found useful,⁵⁴ PCCs in both control and intervention groups likely already had established workflows to review PDMP for patients prescribed LTOT. These findings, alongside studies showing the effectiveness of PDMP querying mandates and EHR integration in increasing PDMP checks and reducing opioid-related deaths,^21,55–57 suggest that EHR integration and organization- and state-level policies may more effectively drive PDMP use than optional CDS designed to support several chronic pain-related decisions and guideline recommendations.

Our study has several strengths and limitations. Strengths include its multisite RCT design, enrollment of a moderately large number of PCCs across clinics and geographic regions, and assessment of guideline-recommended actions using both structured and unstructured EHR data. Several limitations, some of which were discussed above, include minimal in-person training and PCC support due to COVID-19 precautions, as well as the lack of precise targeting of the OneSheet’s EHR alerts on patient visits focused on chronic pain care. Additionally, while we used an established definition of LTOT,¹³ there is substantial variability in LTOT definitions,⁵⁸ which can potentially influence our patient cohort size and visit types.

Conclusion

In a multisite RCT, we found no clinically significant evidence of improved chronic pain management in primary care among PCCs randomized to OneSheet access. Our equivocal findings underscore the complexity and variability of individual clinician workflows, as well as the challenges of developing effective “one-size-fits-all” CDS to enhance guideline-recommended care within already complex EHRs. Consistent with known sociotechnical complexities in CDS design, implementation, use, and evaluation, several micro-and macrolevel factors may have contributed to our findings. Despite the limited effects observed in this study, OneSheet is low-cost to adopt, implement, and sustain in an Epic EHR, and it could be evaluated in other organizations. Organizations adopting OneSheet or similar CDS may observe larger effects by employing additional implementation strategies, such as more intensive user support, incentivizing or mandating use for patients whose care is not guideline-concordant, or combining OneSheet with more extensive workflow redesigns. Moreover, in the future, researchers evaluating CDS should carefully weigh the relative strengths and weaknesses of rigorous, long-term RCTs versus more rapid evaluation or adaptable study designs that may better accommodate evolving macrolevel organizational and environmental factors.

Clinical Relevance Statement.

In our pragmatic RCT, we found no clinically significant evidence of improved chronic pain management in primary care among PCCs randomized to access OneSheet. Our equivocal findings underscore the complexity and variability of individual clinician workflows, as well as the challenges of developing effective “one-size-fits-all” CDS to enhance guideline-recommended care within already complex EHRs. Despite limited results, OneSheet remains a cost-effective EHR CDS that could be more effective with enhanced implementation strategies that account for the sociotechnical complexities of health care.

Multiple-Choice Questions.

1. What was the primary objective of the study involving the chronic pain OneSheet?

   1. To evaluate the cost-effectiveness of EHR-based tools.
   2. To assess the effect of OneSheet on guideline-recommended chronic pain care.
   3. To compare different pain management strategies.
   4. To analyze patient satisfaction with chronic pain treatments.

   Correct Answer: The correct answer is option b.

2. Which of the following outcomes was significantly increased by access to OneSheet?

   1. Ordering naloxone
   2. Reviewing PDMPs
   3. Documenting pain-related goals
   4. Ordering opioid use disorder medications

   Correct Answer: The correct answer is option c.

3. What unintended consequence was associated with the use of OneSheet?

   1. Increased rates of PDMP review
   2. Decreased rates of naloxone orders
   3. Increased patient satisfaction
   4. Decreased documentation of pain-related goals

   Correct Answer: The correct answer is option b.