Design and rationale of an intervention to improve cancer prevention using clinical decision support and shared decision making: A clinic-randomized trial

Abstract

Background

Despite decades of research the gap in primary and secondary cancer prevention services in the U. S. remains unacceptably wide. Innovative interventions are needed to address this persistent challenge. Electronic health records linked with Web-based clinical decision support may close this gap, especially if delivered to both patients and their providers.

Objectives

The Cancer Prevention Wizard (CPW) study is an implementation, clinic-randomized trial designed to achieve these aims: 1) assess impact of the Cancer Prevention Wizard-Clinical Decision Support (CPW-CDS) alone and CPW-CDS plus Shared Decision Making Tools (CPW+SDMTs) compared to usual care (UC) on tobacco cessation counseling and drugs, HPV vaccinations, and screening tests for breast, cervical, colorectal, or lung cancer; 2) assess cost of the CPW-CDS intervention; and 3) describe critical facilitators and barriers for CPW-CDS implementation, use, and clinical impact using a mixed-methods approach supported by the CFIR and RE-AIM frameworks.

Methods

34 predominantly rural, primary care clinics were randomized to CPW-CDS, CPW+SMDTs, or UC. Between August 2018 and October 2020, primary care providers and their patients who met inclusion criteria in intervention clinics were exposed to the CPW-CDS with or without SDMTs. Study outcomes at 12 months post index visit include patients up to date on screening tests and HPV vaccinations, overall healthcare costs, and diagnostic codes and billing levels for cancer prevention services.

Conclusions

We will test in rural primary care settings whether CPW-CDS with or without SDMTs can improve delivery of primary and secondary cancer prevention services. The trial and analyses are ongoing with results expected in 2021.

1. Introduction

Cancer remains a major cause of death and disability in the United States, despite significant advances in treatment, early detection, and prevention over the last 50 years.[1, 2] Although tobacco use has decreased over the last 40 years, obesity has risen steeply, and healthy diet and physical activity have not substantially improved.[3] Reducing tobacco use and improving receipt of human papilloma virus (HPV) vaccinations could significantly address primary cancer prevention.[4] Appropriate cancer screening may prevent some cancer deaths, but is often underused.[5–7] Delivery of cancer prevention services is a core responsibility of primary care providers.[8–11] Eligible patients should be offered periodic breast, colorectal, cervical and lung cancer screening during primary care visits.[8–10] However, substantial proportions of eligible adults are not up to date with cancer screening.[5–7] Indeed, over the last 15 years some decline in cancer screening has been observed in the U.S.[5–7] Rural populations have been reported to have significantly lower rates of cancer screening than urban populations.[12–17] Thus, innovative approaches are urgently needed to address primary as well as secondary prevention of cancer.

Most U.S. healthcare is delivered in clinics by primary care providers (PCPs), but cancer prevention often receives insufficient attention due to time constraints, competing priorities, lack of clinical decision support, patient and PCP knowledge deficits, and complicated and sometimes conflicting clinical guidelines.[7–11] In this scenario, electronic health record (EHR)-linked clinical decision support (CDS) may have great potential to improve primary and secondary cancer prevention care, especially if it is delivered to both patients and their PCPs and provides up-to-date, evidence-based recommendations.[18–20]

EHR systems have been leveraged to deliver clinical decision support for cancer prevention on a large scale using simple prompts and reminders.[20] Often times, prompts and reminders are ignored or dismissed due to alert fatigue created by the overuse and workflow timing of CDS.[21] More effective EHR-based CDS systems are needed. The objective of this study is to improve provision of evidence-based, personalized cancer prevention services in rural populations by designing and implementing a sophisticated Web-based, EHR-linked CDS that uses algorithms based on USPSTF guidelines and other evidence-based recommendations for primary and secondary cancer prevention. To date, CDS systems have not been studied in rural settings. The Cancer Prevention Wizard (CPW) is integrated within an existing CDS system that identifies and addresses actionable clinical priorities for patients at office visits such as poorly controlled cardiovascular risk factors or glycemic control. We have shown in previous randomized controlled trials that this Wizard CDS system significantly improves chronic disease care in high-risk patients.[18–24] It was deployed using a primary care workflow that achieved sustained high use rates and high provider satisfaction.[24]

In this implementation and dissemination study we use the Consolidated Framework for Implementation Research (CFIR) to guide planning, modification, and conduct of the study.[25] The RE-AIM framework is used to evaluate impact, processes, and outcomes of the study.[26] The focus of this report is the design and methods of a study to evaluate the effectiveness of the CPW-CDS.

2. Trial design and methods

2.1. Overview

The Cancer Prevention Wizard (CPW) study is a three-arm, pragmatic, controlled, clinic cluster-randomized trial with post-intervention assessment of endpoints to compare (a) usual care (UC) (b) cancer prevention CDS alone (CPW) and (c) cancer prevention CDS enhanced with Shared Decision Making Tools (CPW + SDMT). The CPW interventions are delivered to patients and providers in printed form during the rooming process at primary care office encounters, but are also available for clinicians to view within the EHR. This study builds on a decade of previous work, is specifically designed for widespread use in primary care settings that use EHR systems, and is designed to deliver care recommendations to both the primary care provider and the patient in real time at the point of care.[18–24]

The specific aims of the study are: Specific Aim 1: assess the impact of the CPW-CDS ALONE, and CPW-CDS+SDMTs compared to UC on a) primary cancer prevention (tobacco cessation, referral for tobacco cessation counseling, prescription of smoking cessation medications, HPV vaccinations), and b) secondary cancer prevention (screening for breast, colorectal, cervical, or lung cancer); Specific Aim 2: assess the cost of the CPW-CDS intervention from the healthcare system perspective through measurement of healthcare utilization and use of diagnostic codes and billing levels related to cancer prevention services; and, Specific Aim 3: describe critical facilitators and barriers for the CPW-CDS implementation, use, and clinical impact using a mixed-methods approach supported by the CFIR and RE-AIM conceptual frameworks. This study seeks to improve our understanding of how to implement and disseminate effective CDS systems in diverse primary care settings.

2.2. Study setting

This study is being conducted at Essentia Health (EH), a multi-specialty healthcare system that spans Western Wisconsin, Central and Northern Minnesota, and Eastern North Dakota. EH has more than 2,100 physicians and credentialed practitioners working in 13 hospitals and 69 clinics. This study includes the 36 largest EH primary care clinics with two clinics serving as non-randomized pilot sites and 34 clinics randomized to one of the three study arms. In these 36 study clinics approximately 450,000 patients are cared for by 300 primary care providers (PCPs), and 17% of patients have Medicaid insurance. These clinics have used EpicCare® EHR since 2004, and have, on average, 13,235 patients per clinic (range, 805–20,334 patients). Twenty of the 34 randomized study clinics are classified as rural by the RUCA2-UR coding system. Most patients with cancer screening needs are seen by general internists or family practice providers.

2.3. Eligibility and exclusion criteria

The study population includes: (1) 34 predominantly rural randomized primary care clinics, (2) PCPs working in only one of these clinics, and (3) patients of these PCPs receiving care in these clinics. Study eligibility criteria were designed to capture a broad selection of providers and patients. Eligible PCPs include general internists, family medicine physicians, pediatricians, physician assistants, and nurse practitioners in family medicine or pediatrics. To be eligible for the study patients must not be up-to-date at the time of an index visit for a primary or secondary cancer preventive care intervention, and meet all these additional criteria: (a) age from 11 to 75 years at the time of an index encounter with their PCP during the 15-month study accrual period, (b) have no evidence of pregnancy on or within 12 months before the index visit, (c) have no hospice codes in the previous 24 months, (d) have no Alzheimer’s diagnosis codes in the last 12 months, and (e) have no non-skin cancer codes in the 12 months prior to the index visit.

2.4. Recruitment strategy

Patients are enrolled using automated methods that assess eligibility at the point of care using patient data stored in the EHR. The index visit is the patient’s first clinic visit at which a cancer prevention intervention is due within the 15-month accrual period. Table 1 presents the conditions that make a patient eligible for exposure to the CPW-CDS and inclusion in the analytical cohort. By systematically prompting providers to implement cancer prevention guidelines, the clinical decision support may reduce potential biases that contribute to cancer disparities. However, further research is needed to determine the impact on cancer screening outcomes for diverse racial and ethnic communities.

Table 1.

Patient Eligibility for Cancer Prevention Wizard.

Cancer screening or prevention need	Target population	Reference
Breast Cancer	Women ages 50–75 with average risk and no documentation of mammogram in the last 2 or more years	USPSTF, 2009 & 2016
	Women ages 35–49 with a 5-year BCRAT risk score of >2% or a lifetime risk of >16.8%,	USPSTF, 2016
Cervical Cancer	Women of average risk ages 21–29 and no documentation of PAP in last 3 years	USPSTF, 2012
	Women of average risk ages 30–65 with no PAP done in last 3 years or no PAP with HPV co-test in last 5 years	USPSTF, 2012
Colorectal Cancer	Men and women ages 50–75	USPSTF, 2008
Lung Cancer	Men and women ages 55–75, with 30 pack-year smoking history, current smoker or quit within last 15 years	USPSTF, 2014
HPV vaccine	Men and women ages 18–26	ACIP, 2007 & 2019

2.5. Group randomization and clinic selection

From EH’s 69 primary care clinics, the 34 clinics with the highest patient volume were selected for randomization into the three arms. In addition, two other primary care clinics served as pilot sites to test the intervention prior to full implementation to allow appropriate modification and adaptation. The 34 clinics were stratified on rural vs. urban location using RUCA2-UR codes and the proportion of patients at each clinic receiving Medicaid assistance and then randomized in a 1:1:1 ratio to UC (12 clinics), CPW-CDS intervention (11 clinics) or CPW-CDS +SDMT intervention (11 clinics). Eligible PCPs at each clinic are allocated to the same study arm as their clinic, and eligible patients of each PCP are allocated to the same study arm as their PCP. Figure 1 depicts the study design with clinic allocation by randomized condition.

Figure 1.

Study Flowchart

2.6. Clinical Decision Support Intervention Strategy

The CPW-CDS is a component of a sophisticated point of care clinical decision support system that prioritizes health care needs for a given patient and gives patient-specific treatment suggestions for cancer prevention, diabetes, prediabetes, and cardiovascular risk factors.

The CPW-CDS uses Web-based algorithms to identify evidence-based prevention options that address each unmet cancer prevention need. Specific CDS recommendations given to a patient are based on both (a) statements from USPSTF,[27–34] NCl,[35] CDC,[36, 37] and ACIP;[36, 37] and (b) the specific patient’s current clinical state, including age, sex, smoking status, BMI, HPV vaccination status, comorbid conditions, current medication regimen, allergies, and past screening tests for the four target cancers. These cancer prevention algorithms are based on clinical principles that can be found in a document located in the Supplemental materials. Cancer screening and treatment recommendations for smoking cessation, HPV vaccination, and obesity management are given as needed for individual patients. The CPW-CDS uses EHR data applied to algorithms to determine each individual patient’s cancer prevention needs and then includes cancer prevention needs on the list of clinical priorities along with diabetes and cardiovascular clinical priorities. The clinical priorities are presented with the cancer prevention interventions first, followed by the CV risk reduction interventions based on the amount of absolute reversible risk reduction. The printed CDS interfaces are kept to one-page by limiting the patient-specific list to six potential priorities at the time of each primary care clinic encounter. To help mitigate alert fatigue, the alert system to promote use of the CDS is suppressed if the patient had already had an encounter with CDS printed within the last two weeks.

The CPW-CDS also contains three interactive cancer risk calculators that are made available to providers when the CDS is displayed within the EHR. These risk calculators include NCI Breast Cancer Risk Assessment Tool (BCRAT),[38] NCI Colorectal Cancer Risk Assessment Tool (CRCRAT),[39] and Lung Cancer Risk Assessment Tool (LCRAT)[40, 41] to help both patient and PCP understand the patient’s risk for these cancers. An example of a cancer risk assessment tool can be found in the Supplemental materials. Active guidelines within the EHR display of the CDS are used to efficiently create orders for any clinical suggestions generated by the cancer prevention algorithms such as procedures (e.g. mammogram, lung CT, Pap), medications (e.g. for smoking cessation), referrals (e.g. for colonoscopy, weight management), and vaccinations (e.g. HPV). Figure 2 shows an example of an ‘Active Guidelines’ page.

Figure 2.

Active guidelines embeds a direct link to ordering based on prioritized recommendations

The remainder of this report will focus entirely on the Cancer Prevention CDS functions of the CPW-CDS and the design and methods used to study effects, dissemination and implementation.

2.7. Provider Interface

CPW-CDS generates a PCP-facing interface that is patient specific and indicates whether the patient is due for a cancer screening or HPV vaccination. Clinics assigned to CPW only and to CPW + SDMT receive the same provider interface. Prototype CPW-CDS algorithms are based on recommendations published by USPSTF, NCI, CDC, and ACIP.[27–37] They will be updated over time to ensure ongoing congruence with national evidence-based guidelines. Primary prevention and secondary cancer screening information displayed on the CPW-CDS provider interface include smoking cessation and obesity management options and medical interventions (HPV vaccination status, and screening status for breast, colorectal, lung, cervical cancer). If the CPW-CDS determines that the patient has modifiable cardiovascular risk factors, these are presented on the interface as well. All treatment recommendations are labeled as suggestions, and the interface page emphasizes that CPW-CDS suggestions do not take the place of clinical judgment or override a PCP’s detailed knowledge of a particular patient. The provider interface is a powerful visit-planning tool that most PCPs prefer to view in print just before entering the exam room.[22] This one-page interface is printed during the rooming process and either placed in the exam room entry box or given to the provider before seeing the patient. A prototype provider interface is shown in Figure 3.

Figure 3.

Provider interface shows priorities, labs, and considerations

2.8. Patient Interface

For patients, a simple visual approach is preferred, because patients may have low levels of numeracy and misinterpret probabilistic information. A visual display of recommended lifestyle modifications (smoking cessation, weight reduction/counseling), medical interventions (HPV vaccination, medications for tobacco cessation), and screening tests for the four target cancers are included, depending on the needs of the patient identified by the CPW-CDS algorithms. CPW-CDS determines that the patient has modifiable cardiovascular risk factors, these are presented on the interface as well. Similar visual patient interfaces used in previous studies have been well received by most patients,[18–24] accommodate low numeracy, and have been shown in studies by others to be a strong motivational strategy.[42] A prototype patient interface is shown in Figure 4, but may be substantively modified with extensive input from our patient representatives from EH Patient Councils and extensive pilot testing before intervention implementation. This one-page interface is printed during the rooming process and given by the rooming staff to the patient for review shortly before the PCP enters the exam room.

Figure 4.

Printable patient interface shows priorities displayed in symbol form, goals and recommendations

2.9. Cancer Prevention Wizard plus shared decision making tools

In addition to the CPW-CDS, providers and their patients in the CPW-CDS + SDMT group have access to shared decision-making tools. SDMTs are decision aids that providers and patients use to make joint decisions that aim to elicit the patient’s values and preferences[43, 44] We created our SDMTs based on the existing literature, expert opinion, and pilot testing. These SDMTs were reviewed and edited by Health Education experts. In clinics randomized to the CPW-CDS + SMDT group, patients receive printed versions of the appropriate SDMT in addition to the one-page CPW-CDS patient interface. Patients that are identified by CPW-CDS as being in need of breast, lung, or colorectal cancer screening or HPV vaccinations, abbreviated versions (1/4-page) of four separate SDMTs are automatically printed along with the CPW-CDS interface print out. More complete versions of the SDMTs (each ranging in length, 1 to 4 pages) are available in the interactive Active Guideline page, where each tool has five components: 1) overview of screening or treatment, 2) screening or treatment options, 3) benefits, 4) risks, and 5) how do I make a decision using structured questions. These SDMTs can be found in the Supplemental materials section.

2.10. Usual Care

In the usual care group of the study, clinics and their PCPs will have no access to the CPW-CDS. Some existing EpicCare® prompts and reminders have been used for years in EH clinics, but these pop-up alerts are easily ignored. These alerts are marginally effective, resulting in current suboptimal rates of cancer screening tests, very low rates of referrals for weight management and smoking cessation, and HPV vaccinations.[45] Prior studies indicate that simple alerts have limited effectiveness due to many factors, such as alert fatigue, provider burden, and limited patient engagement.[46, 47] There is no other systematic cancer prevention clinical decision support system in EH clinics other than what will be implemented as part of this project in the two intervention arms.

2.11. Clinical workflow

In this project, CPW-CDS recommendations are presented to the patient directly and the PCPs using a sequence of clinic staff steps successfully implemented in previous studies and pretested interface formats.[18–24] Participating PCPs and clinic staff in intervention clinics are trained to use the provider and patient interfaces of CPW-CDS. When a patient potentially needing cancer prevention interventions has a clinical encounter with a PCP, the following protocol is automatically implemented: (i) After vital signs are entered into the EHR by rooming staff, CPW-CDS assesses cancer prevention needs by using algorithms to evaluate the patient’s EHR data, identifies target patients who need a cancer prevention intervention, and provides a best practice advisory (BPA). In response to a single click, CPW-CDS displays the interface screen to the rooming staff within 1 second (with no additional prompts or triggers needed). The rooming staff prints the patient and PCP versions of the CPW-CDS interface. (ii) If a patient’s mental and physical status appear stable, the rooming staff hands the patient printed interface to the patient, saying “the caution marks show how you can prevent health problems such as cancer, diabetes, stroke, and heart attacks. If you are ready to work on any of these things, please talk with your doctor during your visit today.” (iii) For both the CPW-CDS and CPW-CDS + SDMT groups, a printed version of the provider CPW-CDS is either placed in the basket outside the exam room for rapid review by the PCP before entering the exam room or displayed on the EHR screen with one click on the EHR navigator bar, depending on the PCP preference. (iv) The CPW-CDS + SDMT group patients and providers also receive any relevant abbreviated-format SDMT as printed pages. (iv) In both the CPW-CDS and CPW-CDS + SDMT groups, the PCP uses the CPW-CDS provider interface to guide changes in cancer prevention care and uses the CPW-CDS patient interface to reinforce patient actions. The printed patient interface may be provided as part of the After-Visit Summary that is given to the patient. After discussion with the patient, the PCP can order screening tests, medications, or make referrals to internal programs or community resources to address smoking and/or obesity by using the CPW-CDS Active Guidelines. Figure 5 presents the CPW-CDS workflow and can be found in the Supplemental section.

Figure 5.

CDS Design Flow

2.12. Intervention implementation

Throughout the implementation of the CPW-CDS, we conducted key informant and patient interviews, surveys, usability testing, and continuous quantitative and qualitative feedback between researchers and participants to measure the implementation processes and outcomes as recommended by the CFIR and RE-AIM frameworks. Organizational engagement was accomplished in three phases. In Phase 1, engage EH clinic leadership and managers, informatics personnel, PCPs, rooming staff, and patients through meetings to identify potential influences of implementation of this project. In Phase 2, all CPW-CDS algorithms, SDMTs, and interfaces were extensively pilot tested among stakeholders to adapt the implementation strategy. Representatives from EH Patient Councils evaluated the patient interfaces to maximize patient-centeredness. We recruited two EH primary care clinics not in the randomized study groups to pilot test CPW- CDS and its tools in eligible patients for four weeks. In-depth interviews were conducted with up to 10 PCPs at the two pilot clinics pre- and post-pilot, asking probing questions about use of CPW-CDS, workflow issues and shared decision making tools. After further modification of the CPW-CDS, the project entered Phase 3. EH has 18 volunteer patient advisory councils that engage patients and families as advisors, mentors, and educators. The main goal of this program is to improve the patient and family experience and quality of healthcare. Research team members recruited representative members to regularly review and critique project pilot phase activities, CDS interfaces, SDMTs, survey tools, interview objectives, and project deliverables. This existing infrastructure is ideal for introducing CDS technology to primary care, gaining the reaction and input of patients and families, and refining adaptation of the intervention.

We trained intervention clinics to use CPW-CDS using strategies similar to those EH routinely uses to inform clinic teams of changes to the EHR. These include face-to-face group or individual meetings with all intervention clinic PCPs, rooming staff, and clinic managers, plus email reminders with links to a short instructional video demonstrating rooming staff and PCP roles in CPW-CDS use. Baseline training was conducted at the initiation of the CPW intervention and booster training at intervention clinics occurred throughout the study period.

Strategies to ensure use of the intervention:

Following implementation, all intervention clinic staff receive monthly email reports showing CPW-CDS use rates. Study team personnel had ongoing communication with the intervention clinic nurse managers throughout the duration of the intervention period to assess continued use of CPW-CDS and to gather feedback ensuring real-time observation of implementation fidelity. Support from EH leaders and clinic PCP leaders, plus monitoring and feedback, has helped maximize provider adherence to CPW-CDS study protocols, as demonstrated by CDS use at 75%−80% of targeted visits using similar strategies in previous projects.[18–24] The study experienced drastic drops in use rates due to COVID-19 as many healthcare systems suspended non-essential in person visits. The study team is evaluating the full impact of COVID on the project. Preliminary plans are described in more detail under trial status.

Intervention timeline:

The study period from CDS go-live date to the end of post-index visit patient follow-up was 27 months in duration. Patient accrual for the analytical cohort at index visits occurred for 15 months, followed by a period of 12 months after each patent’s index date to obtain outcome data.

Additional quality assurance (QA) was conducted prior to the CDS go-live date to ensure appropriate functioning of the CPW-CDS algorithms, which determine clinical recommendations made by the CPW-CDS to the provider and patient. The QA required manual review of selected patients’ electronic medical records by key study personnel at Essentia Health. Manual chart review was also required during implementation for QA of CPW-CDS algorithms. Information reviewed during QA period was not retained for study analyses.

2.13. Stakeholder engagement, training and feedback strategy

At the onset of the study, the project team engaged Essentia Health leadership who agreed to have the CPW-CDS active in their care system as it reinforces recommendations in current national and regional clinical guidelines. Through an ongoing and iterative process, the study team engaged with Essential Health leadership, clinic leaders, and patient advisors with regular meetings to discuss the intervention, successes, problems, and determine solutions. Training was continuous throughout the intervention. Feedback occurs by monthly performance reports delivered by email and at clinic business meetings. These reports include CPW-CDS printing volume of provider and patient interfaces for each provider as a proxy for use of the CPW-CDS.

2.14. Aim 3 strategy: Implementation and dissemination activities.

The RE-AIM and the CFIR provide conceptual frameworks to study the implementation and dissemination of the Cancer Prevention Wizard.[25, 26] To discover the barriers and facilitators to implementation and dissemination of the CPW-CDS, mixed-methods research was used guided by these conceptual frameworks. Although CFIR has five domains, we selected four that are relevant to our study. I. Intervention Characteristics that included these constructs: adaptability, trialability, evidence strength and quality, complexity; II. Outer Setting that included these constructs: patient needs/resources, external policy and incentives; III. Inner Setting that included these constructs: networks and communications, implementation climate, readiness for implementation, available resources; IV. Characteristics of Individuals that included these constructs: knowledge and beliefs about intervention, self-efficacy. CFIR metrics include these variables: 1. PCP and staff perceptions of the CDS, its strength and quality of evidence base; 2. adaptability of the CDS; 3. usability testing; 4. PCP engagement; 5. fidelity of implementation and modifications needed; and, 6. PCP knowledge, attitudes, beliefs, and self-efficacy about the CDS. RE-AIM metrics include these variables: 1. types and % of patients and providers reached; 2. for whom the intervention was effective; 3. % of clinics and providers that adopted the intervention; 4. consistency of intervention implementation; and, 5. proportion of intervention components and effects maintained. Using a mixed-methods approach, the following activities were used to assess these metrics: 1. Semi-structured interviews pre-intervention and post-intervention of EH leaders, PCPs and clinic staff; 2. review of meeting minutes from organizational engagement and study implementation activities; 3. patient interviews and surveys before, during, and after the intervention to learn about their experiences with the CDS and their perceived barriers and facilitators to act on their care recommendations.; and, 4. PCP interviews and surveys before, during, and after the intervention to learn about their experiences and their perceived barriers and facilitators to using the CPW-CDS. Interviews of EH leaders, PCPs, and clinic staff were conducted by research staff at EH and were selected to be representative of their work groups.

3. Study variables and outcomes

3.1. Primary outcomes

3.11. Cancer Screening Tests

The dependent variable for Hypotheses 1 (H1) (see listing of hypotheses in section 4.3.1) is a binary variable indicating that the patient is up to date on all screening tests that were due at the index visit among breast, cervical, and colorectal cancer by 12 months after the index visit. Each patient will be classified as up to date or not on all appropriate screening tests (composite endpoint), depending on sex, age, risk factors, and date of last needed screening tests according to active USPSTF recommendation statements. This variable will be based on EHR-captured procedure codes, and data will be obtained from the index visit date through the end of the study.

3.12. HPV Vaccination rates

New CDC recommendations put into effect at Essentia Health in January 2017 are reflected in changes to the HPV vaccination sample and recommendations in this study. Children ages 9–14 are now recommended to receive 2 HPV vaccine doses, while 3 HPV vaccine doses are still recommended for adolescents and young adults ages 15–26. The dependent variable for Hypotheses 2 (H2) is a binary variable indicating that the patient has completed the course of HPV vaccinations by having all 2 or 3 HPV vaccinations recommended by ACIP/CDC, dependent on age group, within 12 months of the index visit. HPV vaccination data will be obtained from the EHR from the index visit date through the end of the study.

3.2. Health economic outcomes

3.21. Implementation costs from a payer perspective

Implementation costs will include costs directly related to the intervention as well as the incremental medical care costs associated with the intervention from the health system perspective. Intervention costs will include CDS implementation and maintenance, training, and incentives but exclude intervention research and development costs. Medical care costs will include costs of all services associated with primary medical care—including laboratory, physician services, and screening tests—incurred in the 12-month post-index date period by participants in each study group, as indicated by EH billing and clinical encounter data. Costs associated with emergency visits and hospitalizations will be excluded from the assessment of medical costs, as these events are expected to be too infrequent in the study sample to accurately predict a population-wide impact of the CDS on these costs. Reliance on EH billing records for measuring medical care use may miss costs incurred in other health systems; however, this opportunity is expected to be equal across randomized study arms, and cross-system medical utilization is expected to be relatively limited in the primarily rural study population.

The charged and paid amounts in this billing system are specific to EH at a particular time and may provide a biased view of costs between pre- and post-intervention periods due to variation in billing practices in contracts with payors. To address this, we will use a previously developed approach to standardize costs using Total Care Relative Resource Values (TCRRVs),[48] a nationally representative and standardized set of pricing measures derived from Centers for Medicare and Medicaid Services relative value units (CMS RVUs).[49] TCRRVs extend CMS RVU measures to include additional utilization categories, such as laboratory services and medications, which do not have CMS RVU weights. In addition, TCRRVs have been endorsed by the National Quality Forum.[50]

3.22. Coding for cancer preventive services and complexity in primary care

Use of ICD-10 diagnostic codes and CPT-4 billing codes in study clinics will be identified using EHR data. ICD-10 diagnostic codes associated with screening for breast (Z12.3, 3014F, 77055–77057, 77065–77067, G0202, G0204, G0206, G9899, G9900), colorectal (Z12.1, 3017F, 45378–98, 82270, 82272, G0104-G0106, G0120-G0122, G0328, G0464), cervical (Z12.4, 3014F, G0101, P3000-P3001, Q0091), and lung cancer (G0296-G0297, S8032), HPV vaccination (Z11.51, 90649–90651), smoking cessation (F17 and Z72, 4000F-4001F, 4004F, 99406–99407, G0436-G0437, G8402-G8403, G8453-G8454, G9016, G9458, G9906, S4995, S9075, S9453), and obesity mitigation (E66, Z68.3-Z68.4, G0447, G0473) will be included. Coding complexity of clinical encounters will be measured by billing levels for evaluation and management services during clinic visits. Specifically, complexity levels will be measured using CPT-4 codes for new patients, from 99201 (level 1, low complexity) to 99205 (level 5, high complexity), and established patients, from 99211 (level 1, low complexity) to 99215 (level 5, high complexity).

3.3. Independent variables

The primary predictor is the treatment arm to which a clinic is randomized: UC, CPW-CDS, CPW-CDS+SDMT. This variable is linked to the patient based on the clinic at which the patient has an index visit. The index visit is the patient’s first encounter at which a particular screening is due within the 15-month accrual period. Patient characteristics are obtained from the EHR and include demographics, pre-intervention comorbidities (derived from dated ICD-10 diagnosis codes), Charlson score, insurance status, vital signs, height, weight, smoking status, HPV vaccine status, family history of breast or colorectal cancer, and previous cancer screening test dates and results, among others. Provider characteristics are obtained from EH administrative data and include age, years since graduation, sex, full-time or part-time status, physician or allied provider (i.e., nurse practitioner), specialty board certification status, years with EH, and proportion of linked patients up to date on cancer prevention at baseline.

Patient and provider characteristics are gathered to describe patients and PCPs, as potential covariates in analytic models, in secondary analysis to explore heterogeneity of treatment effects across subgroups of patients, and to assess the extent to which PCP characteristics modify intervention efficacy. Patient sex will serve as a covariate in the primary analyses because screening and vaccination rates are likely to be different by sex. Clinic randomization may introduce random or selection-induced patient or PCP covariate imbalance, necessitating adjustment by patient or provider factors.

3.4. Secondary outcomes

Secondary endpoints include binary indicators for (a) patient receipt of a referral to a smoking cessation counselor (b) prescription of smoking cessation medications by 12 months post-index date, and (c) examination of intervention effects for heterogeneity across pre-specified subgroups of study subjects. Procedure codes and other information from the EHR serve as the sources of the secondary outcomes.

4. Data sources and analysis

4.1. Data sources, quality and management

We have created and validated programming to (i) extract pharmacy, laboratory, vital signs, demographic, comorbidity and other data from the EHR, and (ii) export these data to a secure Web site inside the HealthPartners Medical Group firewall. Web site algorithms use these EHR-extracted data to identify eligible patients needing primary and/or secondary cancer prevention at the time of a routine, non-urgent primary care index visit, and at any subsequent primary care visits during the patient-specific 12-month post-index follow-up period. These data extracts are also the data source for primary and secondary outcomes, as well as patient demographic and medical care characteristics. In the Supplement materials section, Table 2 presents the dependent and independent variables and their sources.

Table 2. List of Dependent and Independent Variables.

Variables for Analysis including Description, Data Source, and Classification.

Variable	Description	Data Source	Variable Classification
Dependent Variables
Up-To-Date Cancer Screening	A binary variable indicating the patient is up to date on screening tests for breast, cervix, and colorectal cancer by 12 months following the index visit.	EHR Procedure Results	Binary
Up-To-Date Colorectal Cancer Screening	A binary variable indicating the patient is up to date on colorectal cancer screening by 12 months following the index visit.	EHR Procedure Results	Binary
Up-To-Date Breast Cancer Screening	A binary variable indicating the patient is up to date on breast cancer screening by 12 months following the index visit.	EHR Procedure Results	Binary
Up-To-Date Cervical Cancer Screening	A binary variable indicating the patient is up to date on cervical cancer screening by 12 months following the index visit.	EHR Procedure Results	Binary
All HPV Vaccinations Complete	A binary variable indicating the patient has had all 3 or 2 HPV vaccinations by age as recommended by ACIP within 12 months of the index visit.	EHR Procedure Results	Binary
Some HPV Vaccinations Complete	A binary variable indicating the patient has had at least 1 HPV vaccination as recommended by ACIP within 12 months of the index visit.	EHR Procedure Results	Binary
Referral to Obesity Management Programs	A binary variable indicating that a referral was provided to a patient for internal or communitybased obesity management programs within 12 months of the index visit.	EHR Procedure Results	Binary
Referral to Smoking Cessation Programs	A binary variable indicating that a referral was provided to a patient for internal or communitybased smoking cessation programs within 12 months of the index visit.	EHR Procedure Results	Binary
Prescription of Smoking Cessation Medications	A binary variable indicating that a prescription was made for smoking cessation medications within 12 months of the index visit.	EHR Procedure Results	Binary
Provider satisfaction with CPW use***	Provider post-intervention period: Proportion indicating highest satisfaction	Provider surveys	Interval
Patient visit experience: perceptions of CPW, provider communication, exposure to CPW and SDMTs***	Pre and multiple post intervention time points. Proportion indicating highest satisfaction. Description of change in proportion over time.	Patient survey and interviews	Interval
Provider perceptions of and experience with CPW–CDS ***	Pre- & post-intervention time points: Proportion indicating highest satisfaction. Description of change in proportion over time.	Provider surveys	Interval
Provider knowledge, attitudes, & beliefs of cancer prevention care ***	Pre- & post-intervention time points Change over time.	Provider surveys	Interval
	Independent Variables
Study Arm	UC, CPW, CPW+SDMT	Administrative	Nominal
Patient Sex	Male or Female	Administrative	Nominal
Patient Age	Years	Administrative	Interval
Patient Race	Standard categories	EHR Clarity	Nominal
Patient Ethnicity	Standard categories	EHR Clarity	Nominal
Body Mass Index (BMI)	Weight in kilograms divided by the square of height in meters. Used as a descriptive variable and to select patients for analysis.	EHR Vitals	Interval
Smoking Status	Indicator variables for current, former, or never smoker. Used as a descriptive variable and to select patients for analysis.	EHR Clarity	Nominal
Medicaid Status	Insurance status. Medicare, Medicaid, Commercial, Other, None	Administrative	Nominal
Date of Death	Exclusion criteria for study eligibility.	EHR, administrative, state death data	Date
Family History of Cancer	A binary variable indicating a family history for colorectal or breast cancer is present.	EHR Family History	Binary
Coronary Heart Disease Status	Indicator variable for CHD comorbidity based on one or more inpatient or outpatient ICD-10 codes.	EHR Diagnosis Codes, Administrative Claims Data	Nominal
Diabetes Status	Indicator variable for diabetes mellitus based on inpatient and outpatient diagnosis codes.	EHR Diagnosis Codes, Procedure Results, Pharmacy	Nominal
Charlson Comorbidity Score (Modified)	Indicator of serious comorbid conditions that may shorten life expectancy. Modified to exclude CV components. Limited to 2 levels, and excludes from study eligibility if > 3 at any time in intervention period (Deyo modified score).	EHR Encounters, Clarity	Nominal
Cancer Diagnosis	Indicator variable for existing cancer diagnosis other than non-melanoma skin cancer. Exclusion criteria for study eligibility.	EHR Diagnosis Codes, Administrative Claims Data	Nominal
Cancer Chemotherapy	In a defined period of time, one or more ICD9/ICD-10 procedure codes for cancer chemotherapy. Exclusion criteria for study eligibility.	EHR Procedure Codes and Administrative Claims Data	Nominal
Hospice Care	In a defined period of time, one or more HPMG special codes for Hospice Care on EHR Problem List. Exclusion criteria for study eligibility.	EHR Problem List	Nominal
Alzheimer’s Disease Status	Indicator variable for Alzheimer’s disease based on one or more inpatient or outpatient ICD-10	EHR Diagnosis Codes	Nominal
Atherosclerotic Cardiovascular Disease (ASCVD) Event	A set of indicator variables a defined time period, for occurrence of inpatient primary discharge codes indicating: acute myocardial infarction/acute coronary syndrome, ischemic stroke, transient ischemic attack/RIND, peripheral artery disease including abdominal or thoracic aneurysms or occlusions, or bypass or stent placement in the arterial tree. Exclusion criteria for study eligibility.	Administrative Inpatient Claims Data	Nominal
Patient Enrollment	An indicator variable for sufficient enrollment during defined period of time. Exclusion criteria for study eligibility.	Administrative	Nominal
Number of Office Visits	Outpatient primary care and subspecialty visits, separately, in a defined period of time.	EHR Encounters	Interval
CDS Activations	Count of CDS activations within 12 months of the patient index date within the intervention arms	CDS	Interval
Clinic ID	Clinic identification number needed for randomization and for use as a random effect in the analysis.	Administrative	Nominal
Clinic Location: Rural vs. Urban	Based on RUCA2-UR codes	Administrative	Nominal
Provider Type	Primary care physician, NP, other. Used to determine provider study eligibility.	Administrative	Nominal
Number of Patients per Provider	Number of study eligible patients cared for by provider. Used for provider study eligibility	Administrative	Nominal
Provider Age	Years	Administrative	Interval
Provider Years Since Graduation	Number of years elapsed since provider completed training.	Administrative	Interval
Provider Gender	Male or female	Administrative	Binary
Provider Full-Time Status	Full-time of part-time	Administrative	Binary
Provider Tenure at Essentia	Number of years a provider has been affiliated with Essentia Health	Administrative	Interval
Protocol for implementation metrics using the RE-AIM framework
Domain	Measure	Data Source
Reach	Identification rate among patients –Number and proportion of patients identified as needing cancer prevention interventions.	EHR prescription, patient surveys, & Smart Form data [Intervention only])
	Representativeness of patients –age, gender and clinical characteristics of CDS clinic patients exposed and not exposed to the CP-CDS intervention	EHR data
Effectiveness	Proportion who received a cancer prevention intervention (includes SA #1, H1 & H2)	EHR data
	12-month adherence to a cancer prevention intervention (SA #1, H1 & H2)	EHR data
	Proportion of physician practice patterns meeting evidence-based recommendations of the CP-CDS	EHR data (CPW-CDS & UC clinics), CPW-CDS Smart Form use
	Proportion of patients reporting participation in weight or smoking counseling or programs	Patient surveys
Adoption (Intervention Clinics)	Proportion of primary care clinics agreeing to participate and, within clinics, proportion of primary care providers consenting to participate	Meeting minutes, consent forms
	Representativeness of settings – Proportion of potential clinics and providers who participate; differences in their urban/rural location, size, specialty, etc.	Essentia clinic & provider data
Implementation (Intervention Clinics)	Proportion of patients needing cancer prevention identified with the ‘Best Practice Alert’ (BPA’s) among all identified patients CPW-CDS rates of use, defined as number of times used in eligible patient visits, divided by number of eligible patient visits	EHR data Web-service data
	Print use of CPW-CDS interface	Web-service print data, provider & patient surveys
	Use rates of CPW-CDS Smart Form	EHR and Web-service data
Maintenance	Percent Intervention clinics continue using CPW-CDS at 12 months	Meeting minutes, webservice data
	Percent primary care providers using the CPW-CDS at 12 months	Web-service data
	Organizational commitment to sustaining the CPW-CDS	Provider focus groups, key informant interviews of leaders
	Implementation of CPW-CDS in non-Intervention primary care clinics, including spread beyond usual care clinics	Key informant interviews with organizational leaders, training of nonintervention clinics

^***denotes CFIR related metrics

For each eligible patient, the study period begins on the date of the first qualifying visit (index visit) during the 15-month accrual period, which is occurred 8/1/18–10/31/19, and ended 12 months after the index encounter. To extract Wizard CDS-related clinical data at the end of eligible patient’s study period, Wizard was re-run manually outside a visit.

4.2. Sample size and power

Hypothesis 1 is powered at 80% to detect a difference in the proportion of patients up to date on breast, cervical and colorectal cancer screening (women) and colorectal screening (men) at 18 months of 11% in UC compared to 18% in CPW, and 11% in UC compared to 24% in CPW+SDMT. Power calculations originally assumed an 18-month follow-up period and were based on a data pull of patient visits from 2011–2013 which showed 9390 male patients and 33,264 female patients with clinic visits over the study accrual time period and eligible for cancer screening. Power calculations assumed 10 clinics per study arm, ICC=0.02 for the study endpoint reflecting the clustering of patients within clinics, α₂=0.05, R²=0.1 of cancer screening with other covariates, and a pooled weighted average for men and women in the UC arm of 11% up to date on cancer screening 18 months after the patient index visit.

Hypothesis 2 is powered at 80% to detect a difference in the proportion of patients who have completed the HPV vaccination series of 20% in UC compared to 33% in CPW, and 20% in UC compared to 40% in CPW+SDMT. Power calculations were based on a pull of patient visits from 2011–2013 which showed 21,277 patients age 11–26 with clinic visits over 18 months who had not completed the course of HPV vaccinations. Power calculations assumed 10 clinics per study arm, ICC=0.02 for the study endpoint reflecting the clustering of patients within clinics, α₂=0.05, R²=0.1 of HPV vaccination with other covariates, and a pooled weighted average for males and females in the UC arm of 20% with completion of the HPV vaccination series.

4.3. Statistical analyses

4.31. Study hypotheses

Hypothesis 1 (H1).

Eligible study subjects will have significantly different rates of appropriate screening tests for a composite of breast, colorectal, and cervical cancer screening, based on screening frequencies recommendations made by the US Preventive Services Task Force (USPSTF) during the 12 months following the index visit by study arm, with CPW-CDS higher than UC, and CPW-CDS + SDMT higher than UC.

Hypothesis 2 (H2).

Eligible study subjects will have significantly different rates of completion of the human papilloma virus (HPV) vaccination series in appropriate cases, as defined by the Advisory Committee on Immunization Practices and Centers for Disease Control and Prevention during the 12 months after the index visit by study arm, with CPW-CDS higher than UC, and CPW-CDS + SDMT higher than UC.

Hypothesis 3 (H3).

After controlling for demographics and baseline clinical risk factors, eligible study subjects will have significantly different overall healthcare costs during the 12 months after the index visit by study arm, with CPW-CDS higher than UC, and CPW-CDS + SDMT higher than UC.

Hypothesis 4 (H4).

After controlling for demographics and baseline clinical risk factors, eligible study subjects will have significantly different frequencies of diagnostic codes related to cancer prevention services and billing levels in the 12 months after the index visit by study arm, with CPW-CDS higher than UC, and CPW-CDS + SDMT higher than UC.

4.32. Aim 1 analytical approach

Generalized linear mixed models with a logit link and binomial error distribution will be used to test Hypotheses 1 and 2. For Hypothesis 1, the composite binary endpoint of up to date status on cancer screening by 12 months post-index date for breast, cervical, and colorectal cancer will be predicted by two study arm terms representing contrasts in UC vs. CPW and UC vs. CPW+SDMT, patient sex, a random intercept for clinic to acknowledge the clinic-randomized design, and a random intercept for provider. Patient, provider, and clinic characteristics unbalanced by study arm and determined a priori to be potentially prognostic of the study outcome will be included as additional covariates in multilevel models. Prognostic covariates will be pre-specified in a statistical analysis plan prior to the conduct of the main analysis. The analytic model for Hypothesis 2 concerning completing the course of HPV vaccination by 12 months post-index date will be the same as Hypothesis 1 but will also include patient age on the index date as a covariate. All statistical comparisons will be 2-sided. P-values of less than .05 will be considered statistically significant.

In secondary analyses, the H1 analytic model will evaluate endpoints of lung cancer screening and the individual screening components from the H1 composite endpoint in separate models. The H1 analytic model will also be applied to the denominator of smokers at the index date to evaluate endpoints of smoking cessation at 12 months post-index, and any referrals to smoking cessation counseling or prescription of smoking cessation medication by 12 months. The H2 analytic model will evaluate the endpoint of any HPV vaccination in the 12 months following the index date. Treatment group heterogeneity by sex will be assessed by including interaction terms between patient sex and the two study arm indicators in the H1 and H2 analytic models.

Patient denominators for each analysis will be specific to each analysis and include only those patients who are eligible for a particular screening (based on age, sex, risk factors, prior screening activity), or who have not completed HPV vaccination (based on age, prior HPV vaccination activity), or who are smokers at the time of their index visit. Patients are evaluated at each encounter for their eligibility for each analytic denominator. However, the patient index visit for an analytic denominator is the first encounter at which the patient meets eligibility for that specific analytic denominator.

4.33. Aim 2 analytical approach

A generalized estimating equation with a time × study group interaction term will be used to estimate costs and use rates of diagnostic and billing codes for cancer preventive services or the encounter billing level by study arm while allowing for clustering by clinic and controlling for demographics and baseline clinical risk factors. The marginal effect of being assigned to an intervention clinic will provide an estimate of the incremental effect associated with the intervention. All statistical comparisons will be 2-sided. P-values of less than .05 will be considered statistically significant.

4.4. Ethical Considerations

The study is governed by appropriate Research Services and Data Use Agreements and the EH Institutional Review Board (IRB). We have requested and received a waiver of written informed consent for patients for the following reasons: (a) recommendations delivered through the CPW-CDS are based on evidence-based national guidelines, (b) there is no direct contact between the study team and patient participants, (c) it would be impractical to consent the large number of patients who are potentially eligible, and (c) this is a minimal risk study. Furthermore, it is emphasized in training and displayed on the CPW-CDS interface for providers to practice independent clinical judgement when using the CPW-CDS.

5. Discussion

The health of the U.S. could be significantly improved if cancer prevention interventions, and mitigation of cardiovascular risk factors, were systematically offered to all eligible persons according to evidence-based recommendations. Unfortunately, large gaps in delivery of these interventions persist across the nation.[3–7] Innovative, scalable, and cost-effective approaches are needed to address this unacceptable status.

As early as 1991, the Institute of Medicine anticipated that EHR systems linked with CDS systems would rapidly improve chronic disease and preventive care services as well as patient outcomes.[51] However, this potential has not been realized. In this report, we described the design and methods for developing, implementing, and evaluating a Web-based, EHR-linked CDS system in rural, primary care practices that is known as the CPW-CDS. The CPW-CDS integrates primary and secondary cancer prevention care with cardiovascular disease risk reduction interventions. Results of this study may: a) lead to improved delivery of personalized, evidence-based primary and secondary cancer prevention services and cardiovascular risk factors; b) advance understanding of how best to integrate EHR-linked, Web-based, personalized CDS systems focusing on cancer prevention and cardiovascular care into primary care workflows; and c) guide effective dissemination and implementation of similar CDS systems to other rural primary care clinics and healthcare delivery systems.

The intervention is well grounded in previous research published by our research group and others over the last 15 years.[18–24] The study is enhanced by incorporating key principles from two theoretical frameworks, CFIR and RE-AIM, designed to inform future implementation and dissemination of innovative interventions into practice.[25, 26] The research design is a pragmatic, clinic-cluster randomized trial that allows for adaptation and modification of the intervention throughout implementation. Using mixed methods research the facilitators and barriers to successful implementation will be determined, which can improve likelihood of effective dissemination and scalability in future settings and applications. In addition, the research approach supports testing two clinical hypotheses and two economic hypotheses that may result in: 1. Increasing appropriate screening tests for breast, colorectal, cervical, and lung cancer. 2. Increasing rates of HPV vaccination in appropriate subjects. 3. Assessing healthcare costs of the interventions. 4. Assessing the frequencies of diagnostic codes related to cancer prevention services and billing levels attached to primary care visits. The planned formal cost analysis will be important to wider adoption of the Cancer Prevention Wizard, if shown to be favorable to healthcare systems. If data support this hypothesis, it would be evidence that the CPW-CDS benefits not only patients, but also healthcare systems, because of higher revenue collection for the system from increased delivery and documentation of cancer prevention services.

The research approach has many innovative features that may increase its impact, improve its implementation, and advance the field. Firstly, the CDS integrates multiple cancer and cardiovascular disease preventive interventions, which has not been previously studied. TheCDS has 11 targets that are individualized for each patient by sophisticated algorithms: five cancer-specific targets (four screening tests and HPV vaccination), four cardiovascular-specific targets (BP control, lipid management, glucose control, appropriate aspirin use), and two overlap targets (obesity, tobacco control). Secondly, the CPW-CDS interface presents a one-page, dynamic list of the top six priorities for each patient based on patient’s needs and clinical actions having most benefit. Thirdly, the CPW-CDS contains three cancer risk calculators (breast, colorectal, lung cancer), 10-year ASCVD risk calculator using ACC/AHA pooled risk equation, four SDMTs (breast, colorectal, lung screening and HPV vaccination), and an ‘Active Guideline’ interface. The ‘Active Guideline’ facilitates personalized ordering of CDS-suggested clinical services using a “shopping cart” format. Fourthly, CPW-CDS engages both patient and clinician before the clinical encounter with a one-page printout of the personalized CDS interface that can be used before, during, or after the clinical encounter in the exam room. Fifthly, CPW-CDS is a real time system that integrates various clinical decision support approaches onto one platform for clinician and patient review at the point of care.

As planned, CPW-CDS adheres to the principles of effective clinical decision support systems including the “10 Commandments” promulgated by Bates, et al[52] and effective implementation strategy described by Sperl-Hillen, et al[22] in a study of an EHR-linked, Web-based diabetes CDS. Furthermore, CPW-CDS addresses the features of effective CDS systems described by Roshanov,[47] and the facilitators and barriers to uptake of CDS systems studied by Liberati.[53] Our study also addresses the failed strategies of computerized CDS systems in many cancer preventive care studies reported by Souza, et al.[54] The CPW-CDS makes use of patient-specific information to deliver priorities for guideline-recommended preventive services at the point of care.[55–57] Previous studies of CDS systems on preventive interventions in primary care settings have reported mixed results.[46, 53, 54] Although some studies found variable effects on improving processes of care, evidence on clinical, economic, workflow, and efficiency of outcomes were sparse.[46] This study will assess several clinical and economic outcomes. In addition, using CFIR and RE-AIM frameworks in aim 3, this study assesses workflow, usability, and facilitators and barriers to implementation. To date, CDS system application to lung cancer screening and HPV vaccination has not been reported. Furthermore, CDS systems that integrate cardiovascular and cancer prevention interventions have not been conducted. CPW-CDS addresses these gaps in knowledge.

Despite some substantive strengths, our study has several limitations and potential challenges. Firstly, it is being conducted in one large healthcare system in the upper Midwest, which may limit its generalizability. Secondly, using pragmatic trial methods may cause the intervention to evolve during its application, complicating efforts to assess intervention fidelity. Thirdly, PCP and staff turn-over at various primary care clinics may impact the effect of the training program and suggests the need for ongoing support to promote high CDS acceptance and use. Lastly, using EHR-based data to measure outcomes may be limited by the accuracy, completeness, and timeliness of the available digital data.

5.1. Trial status

The CPW-CDS is being conducted between 2018 and 2020 in mostly rural primary care practices and will be evaluated in this pragmatic clinic cluster randomized trial. Analyses are ongoing, and trial results are expected in 2021. In addition, new information will be discovered regarding the facilitators and barriers to implementation and dissemination of the CPW-CDS in integrated healthcare systems. During the study we are monitoring the CDS and providing continuous training in its use and providing feedback to clinicians to help mitigate barriers and enhance facilitators using CFIR and RE-AIM conceptual frameworks. After the evaluation is complete, the performance site, Essentia Health, may choose to further disseminate the integrated CDS across all primary care clinics, including control clinics and clinics not in the randomization groups.

If successful, this innovative Web-based, EHR-linked CDS can be easily adapted and implemented in virtually any primary care practice with a suitable EHR, both in rural and urban settings. Additionally, using the CFIR and RE-AIM conceptual frameworks, this study may provide an opportunity to evaluate best strategies for further dissemination and implementation of CDS systems in primary care settings.

5.2. COVID-19 impact and modified analytical strategy

This study was planned and initiated long before the COVID-19 pandemic started. However, the pandemic evolved toward the end of our study when follow up data were being collected on a portion of our study population. The intervention commenced with patients accrued by an index date August 1, 2018 through October 31, 2019. Each patient was followed for 12 months post index date to determine outcomes, which started August 1, 2019 and ended October 31, 2020. We determined that March 17, 2020, was the date when the pandemic had a major impact on outpatient visits to the 34 primary care clinics in our study with a drop of 80% to 90% for several subsequent months. Therefore, from mid-March through summer 2020, most patients were not seen in study clinics and did not undergo any cancer screening tests. We are still evaluating the extent and impact of COVID-19 on our study. Our current analytical strategy is to restrict our primary analysis for hypotheses 1 and 2 to the “pre-COVID cohort” that was accrued and had at least 12 months of follow up prior to March 17, 2020. This cohort was accrued August 1, 2018 through March 16, 2019, which represents approximately half of the total study cohort. Preliminary power analysis of the pre-COVID cohort is nearly as strong as the original power of the whole projected sample. Patient accrual was completed prior to the onset of COVID-19. If patients with an index visit were to develop COVID-19 during the follow up period, they would still be considered for inclusion in the analysis. If possible, we may conduct secondary analysis of the entire sample that includes both pre-COVID and post-COVID cohorts. We expect to report our primary outcomes in 2021.

Figure 6.

Risk assessment tool

Abbreviations

CPW

Cancer Prevention Wizard

PCP

primary care provider

HPV

human papillomavirus

CDS

clinical decision support

EHR

electronic health record

USPSTF

United States Preventive Services Task Force

CDC

Center for Disease Control and Prevention

ACIP

Advisory Committee on Immunization Practices

NCI

National Cancer Institute

CFIR

Consolidated Framework for Implementation Research

RE-AIM

Reach Effectiveness-Adoption Implementation Maintenance framework

UC

usual care

CDS+SDMT

clinical decision support + shared decision making tools

EH

Essentia Health

BMI

body mass index

BCRAT

breast cancer risk assessment tool

CRCRAT

colorectal cancer risk assessment tool

LCRAT

lung cancer risk assessment tool

BPA

best practice alert