Design and protocol of a pragmatic clinical trial to improve cognitive impairment detection in primary care

Abstract

Background:

The prevalence of cognitive impairment (CI) including Alzheimer’s disease (AD) and related dementias (ADRD) continues to rise worldwide, but often goes undiagnosed leading to increased burden on patients and families. A clinical decision support system (CDSS) could improve care quality by assisting primary care clinicians (PCCs) to recognize, evaluate, diagnose, and manage patients with CI.

Methods:

38 primary care clinics are randomized to receive the study intervention or usual care (UC). The study intervention consists of a cognitive impairment clinical decision support system (CI-CDSS) and a brief CI-focused training. The CI-CDSS utilizes electronic health record (EHR) data to alert PCCs of patients who may be at high risk for CI, determined by either an abnormal cognitive assessment or identification as high risk using a prototype prediction model developed for this study that estimates likelihood of developing CI in the next 3 years. It also provides tools and resources to evaluate and diagnose CI and gives recommendations for managing care of patients with CI.

Endpoints:

The primary outcome is EHR documentation of CI diagnosis up to 18 months after accrual. Secondary outcomes include healthcare costs and PCC confidence in diagnosis and management of patients with CI.

Conclusion:

This pragmatic, cluster-randomized, Phase III clinical trial aims to assess the effectiveness of a CDSS in increasing detection of CI in primary care. If successful, this system could provide up-to-date personalized recommendations for CI diagnosis and management to improve patient care.

1. Introduction

Cognitive impairment (CI), including Alzheimer’s disease (AD) and related dementias (ADRD), affects over 57 million people worldwide, with the prevalence expected to triple by 2050 [1]. Despite this high prevalence, CI is still unrecognized in 27–81 % of affected patients [2]. While cognitive disorders developing in late life are usually chronic and progressive, timely diagnosis sets the stage for patients and families to learn about the condition and make appropriate decisions about their care. Well-designed care plans for individuals with CI can improve symptoms, reduce the risk of crises, and promote quality of life. In addition, emerging therapies are being tested or have been recently approved for AD, including monoclonal antibody infusions that bind to brain beta-amyloid [3]. Many of these therapies are intended for early stages of the disease, making early detection and diagnosis of CI essential to identify patients eligible to receive such treatments.

The Annual Medicare Wellness (AMW) exam in primary care provides an opportunity for early detection of CI. AMW exams require screening for CI [2], positioning primary care clinicians (PCC) as the first to recognize a cognitive issue. However, screening alone does not necessarily result in subsequent CI evaluation, diagnosis, or treatment. One study suggests that relevant PCC follow-up action occurred in only 17 % of patients who screened positive for potential CI, suggesting barriers between initial CI detection and eventual diagnosis [4]. These barriers can be better understood and addressed using implementation science frameworks like the Behavior Change Wheel (BCW) [5]. The BCW distinguishes individual barriers to clinician actions like capability and motivation from systems barriers like opportunity. In CI care management, clinicians experience individual barriers such as lack of confidence in assessment and diagnosis or beliefs that there will be negative consequences from diagnosing, as well as systems barriers such as inadequate visit time, limited access to specialists, and limited usability of the electronic health record (EHR) to aide in diagnosis and care.

Real-time clinician decision support systems (CDSS) could offer a solution to individual barriers by increasing PCC motivation and ability to diagnose and manage CI. CDSS have been successfully implemented in primary care for diagnosis and management of many diseases, including cancer, cardiovascular diseases, musculoskeletal diseases, and other chronic conditions [6,7]. A survey of PCCs suggests that features of a cognitive impairment CDSS (CI-CDSS) that would be most valuable include screening assessment, diagnostic guidance, and tools for patient management and ongoing care [8]. Additionally, implementing EHR-integrated algorithms as part of a CI-CDSS to identify individuals who would most benefit from targeted assessment could address systems barriers such lack of time or usability of the EHR [9]. Several dementia detection algorithms [10], including eRADAR [11–13], have been developed to identify unrecognized dementia or predict future dementia risk utilizing information from the EHR [10]. While these approaches may be effective at identifying those with undiagnosed or high future risk of dementia, few have combined these algorithms with HER-integrated tools to address clinician-level barriers which may prevent PCCs from using this information and changing their practice. Furthermore, a CI-focused clinician training could address clinician-level barriers by increasing confidence and dispelling beliefs about negative consequences of diagnosis.

This paper describes the design and protocol for a pragmatic, cluster-randomized clinical trial to test an EHR-integrated CI-CDSS that uses existing CI screening and assessment results and a prototype machine-learning CI risk prediction model to alert PCCs in a large healthcare system of patients who are at risk for CI and guide them through the screening, diagnosis, and management processes. The goals of this study are to assess whether this use of this CI-CDSS can increase detection of CI in primary care, decrease healthcare utilization costs, and increase PCC confidence in detection and management of CI.

2. Methods

2.1. Study overview

The objectives of this pragmatic, clinic-randomized trial are to evaluate the effectiveness of a CI-CDSS designed to increase CI detection in patients at elevated risk for CI (Fig. 1). Clinics are randomized into either intervention or usual care (UC). In all study clinics (both intervention and UC), the CI-CDSS runs in the background to identify study-eligible patients and store EHR data related to their care. In intervention clinics, the CI-CDSS also identifies care gaps and provides recommendations via EHR alerts and PCC- and patient-facing handouts. Eligible patients are accrued at clinic visits and then followed for up to 18 months to assess study outcomes.

Fig. 1.

Study design to test the cognitive impairment clinical decision support system (CI-CDSS, n = 19) vs. usual care (n = 19) in a cluster-randomized clinical trial.

2.2. Ethical and regulatory approval

All procedures have been reviewed and approved by the HealthPartners Institutional Review Board (IRB).

2.3. Trial registration and status

This clinical trial is registered on clinicaltrials.gov, NCT05723523. Accrual started on 08/23/2023 and closed on 04/23/2024. The patient follow-up period continued until 02/28/2025. Results of this study are expected in 2025.

2.4. Study setting

This study takes place in 38 primary care clinics with over 450 PCCs at a large integrated health system in the Midwest that provides care to approximately 1.4 million patients. These primary care clinics currently provide cognitive care through a team-based approach—rooming staff members conduct cognitive screening during annual wellness visits and communicate the results with the PCC.

2.5. Randomization

Primary care clinics were eligible for randomization if health system leadership had no operational concerns about implementing CI-CDSS at the clinic, and the clinic had no active CI care improvement initiatives. Clinics are randomized 1:1 to implement the CI-CDSS intervention (n = 19) or to UC (n = 19) using a simple randomization allocation scheme. In keeping with an intent-to-treat principle, all study-accrued patients and their outcome data will be attributed to the treatment group of the clinic where their first (index) visit took place.

2.6. Eligibility and accrual

The CI-CDSS uses algorithms to automatically and uniformly assess the following patient eligibility criteria at every primary care visit in both CI-CDSS and UC clinics: (1) Age 65 years or older, (2) Not in a hospice or palliative care program, (3) No stage 4 or equivalent cancer diagnosis or active parenteral chemotherapy within the last year, (4) No ICD-10 code for a cognitive disorder (see Supplemental Table 1) documented prior to the visit, and (5) evidence of elevated CI risk. CI risk is defined as having 1) an abnormal cognitive assessment (MoCA <26, SLUMS <27, MMSE <24, or Kokmen<30) in the last 18 months, 2) an abnormal Mini-Cog (score < 3) in the last 18 months without evidence of further cognitive assessment, or 3) no cognitive assessments in the last 18 months but a risk prediction model-estimated risk ≥15 % of CI diagnosis in the following 3 years.

Each patient is accrued into the study at their index visit: the first visit when they meet all eligibility criteria. The index visit date marks the beginning of a patient-specific observation period of up to 18 months. As there is no study-determined visit schedule, patient visits with their PCC (and by extension, their exposure to the intervention) over the course of the observation period take place at a frequency determined by the patient and their PCC, with no interference from the study team. The accrual period is 8 months and begins and ends at the same calendar time in all randomized clinics. The intent to treat (ITT) patient sample is comprised of all patients who have an index visit during the accrual period except for those who request that their data not be used for research purposes. The safety sample includes all patients who have an index visit during the accrual period.

PCCs who practice at the randomized clinics are invited via email to participate in a survey prior to CI-CDSS implementation (baseline) and again 8 months post-implementation. A $100 gift card is offered for each survey completion. The PCC sample includes anyone who provides a complete or partially complete baseline or follow-up survey.

2.7. Informed consent

This study is being implemented with a waiver of informed consent for patients due to the intervention posing minimal risk beyond the risk of primary care encounters, the CDSS providing evidence-based guideline-concordant care, and the inability to conduct the study without such a waiver. Patients who have opted out of having their data used for research will be excluded from all analyses except for those evaluating safety. A waiver of documentation of informed consent is being implemented for PCC surveys.

2.8. CI risk prediction model

A risk prediction model was developed to identify patients who are at elevated risk of a CI diagnosis within 3 years. A cross-sectional cohort was assembled to include patients (n = 100,998) who at the time of a primary care visit or AMW exam in 2017 were age 65 years or older and did not have a CI diagnosis, defined as two or more encounter diagnoses of dementia or dementia documented on the problem list within the EHR. EHR data from up to 5 years prior to the visit were extracted for 1017 indicators of potential CI risk. These factors fell into categories such as: annual wellness questionnaire items, clinical encounters, cognitive screens, diagnosis and problem lists, health modifiers, laboratory values, current and prior medications, procedures, patient reported outcomes, social history, vaccinations and vitals.

LASSO was used to identify a subset of risk factors that would maximize the area under the curve (AUC) in predicting the likelihood of a new MCI or dementia encounter diagnosis or addition of dementia to the problem list in the subsequent 3 years. LASSO was used as the variable selection method based on results from the R SuperLearner package. The LASSO-selected variable set included 33 covariates representing demographics, prior diagnoses, healthcare utilization, laboratory values, medications, vitals, and social behavior (Supplemental Table 2). Logistic regression models were then fit from the LASSO-selected variables in randomly selected training (70 %) and testing (30 %) datasets so that accuracy and classification metrics could be calculated.

In both datasets, approximately 6 % of patients were observed to have a CI diagnosis within 3 years. The AUC of the final model was 0.80 (95 % CI: 0.79–0.81) in both datasets, comparable to the reported AUC of the Mini-Cog alone, which is 0.84 (95 % CI: 0.76–0.89) [14]. Model-estimated risk ≥15 % was selected as the threshold to represent high CI risk in collaboration with primary care leadership to balance burden on PCCs while also maintaining their familiarity with the CI-CDSS. Both datasets suggested that about 9 % of patients over age 65 with no prior CI diagnosis would be flagged for further assessment using this threshold, and that about 23 % of those patients would go on to have a CI diagnosis in the coming years. Additional information regarding model performance is summarized in Supplemental Table 3.

Addition of MCI to the problem list was inadvertently left out of the outcome calculation during model development. A sensitivity analysis was performed which broadened the outcome to include addition of MCI to the problem list without a documented encounter diagnosis. The addition of this outcome did not appreciably change the regression coefficients or fit statistics of the model.

2.9. Intervention description

The intervention is designed based on a COM-B theory-aligned conceptual model (Fig. 2). The intervention consists of the CI-CDSS and a brief training to educate PCCs both on CI diagnosis and management as well as use of the CI-CDSS.

Fig. 2.

Conceptual model depicting the hypothesized relationship between the study intervention and patient outcomes in CI assessment, diagnosis, and care management in primary care. CI-CDSS: cognitive impairment clinical decision support system.

2.9.1. CI-CDSS

The CI-CDSS is integrated into an existing system that provides decision support for diagnosis and management of cardiometabolic and other chronic health conditions and prioritizes health issues based on what would be most beneficial to the patient to address. The system is comprised of a web service that gathers EHR data and runs the data through evidence-based and expert consensus algorithms at every primary care encounter. When a blood pressure measurement is entered in the EHR during the rooming process, a Best Practice Advisory (BPA) displays for eligible patients which prompts rooming staff to print a page containing a personalized health priorities list, including brain health/CI content (Table 1). A layperson version of this page is provided to the patient to review while waiting for the PCC. The professional version of this page is given to the PCC to review before entering the exam room. PCCs and support staff are encouraged, but not required, to use the CI-CDSS with eligible patients.

Table 1.

CI-CDSS display to patient and PCCs based on patient status.

Status	Patient Display	PCC Display
Diagnosis of Dementia	Please talk to your clinician if you or your caregiver need help finding support or services related to memory problems or dementia	Consider talking to patient or caregiver about support or services for memory problems or dementia (see the Wizard Tools Cognitive Health tab for more information)
Diagnosis of Mild Cognitive Impairment	Please talk to your clinician if you or your caregiver need help finding support or services related to memory problems or dementia.	Consider talking to patient or close contact about memory concerns and conduct assessments regularly (e.g. using MoCA) to monitor for progression and support needs (see the Wizard Tools Cognitive Health tab for more information).
Abnormal Mini-Cog in the last 18 months	Consider scheduling a visit to talk to your doctor about memory health. If there is a problem with memory beyond what is expected with normal aging, they can help you understand what it means, address possible treatable causes, and plan for the future.	Patient screened positive for potential cognitive impairment on the MiniCog (score less than 3). Consider completing a cognitive health evaluation, such as the MoCA. See the Wizard Tools Cognitive Health tab for more information
Abnormal Cognitive Assessment in the last 18 months	Consider scheduling a visit to talk to your doctor about memory health. If there is a problem with memory beyond what is expected with normal aging, they can help you understand what it means, address possible treatable causes, and plan for the future.	A prior cognitive assessment was outside the normal range. Consider additional evaluation and adding a diagnosis if appropriate
and...
Does not have a B12 test within the last year	–	Consider ordering a B12 test.
Does not have a TSH test within the last year	–	Consider ordering a TSH test.
Does not have a CT or MRI within the last 5 years	–	Consider recommending a brain imaging test (CT or MRI).
Age < 65	–	Based on age less than 65, consider referring to neurology for further evaluation.
Cognitive assessment score is < 10	–	Based on high concerns about cognitive health, consider close monitoring with follow up visits to assess progression and support needs for the patient and/or care partners.
No outstanding care opportunities	–	Consider monitoring with frequent follow up visits to assess progression and support needs for the patient and/or care partners.
Identified as highrisk using CI risk model	Consider scheduling a visit to talk to your doctor about memory health. If there is a problem with memory beyond what is expected with normal aging, they can help you understand what it means, address possible treatable causes, and plan for the future.	Cognitive impairment risk in next three years is estimated to be elevated based on comorbidities, medications, utilization patterns, previous cognitive and lifestyle assessments, demographics, and other factors. Consider completing a cognitive health evaluation, such as the MoCA. See the Wizard Tools Cognitive Health tab for more information.

Additionally, the CI-CDSS has interactive modules within the EHR that support PCCs in assessing patients with elevated risk for CI. A clickable tab opens a page where PCCs can view relevant test results and order referrals and cognitive assessments (Supplemental Fig. 1). There are four additional modules to assist PCCs with the diagnosis and management of CI.

1. Assessments Module: This module (Supplemental Fig. 2) offers multiple tools to assess cognitive function for patients. The Mini-Cog is available for initial evaluation along with 4 options for cognitive assessment: Montreal Cognitive Assessment (MoCA), Mini-Mental Status Exam (MMSE), St. Louis University Mental Status Exam (SLUMS), and the Mayo Mini Mental (MMM, Kokmen). A link to the functional activities questionnaire (FAQ) is available to assess instrumental activities of daily living (IADLs).

2. Additional Evaluations Module: This module (Supplemental Fig. 3) gives personalized recommendations for blood and imaging tests and tips for identifying patients with atypical symptoms of CI who may benefit from referral to a neurologist.

3. Diagnostic Criteria Module: This module (Supplemental Fig. 4) provides a drop-down menu that contain information on common and less common CI diagnoses to assist in diagnosis.

4. Care and Support Module: This module (Supplemental Fig. 5) provides support for management of CI care after diagnosis including pharmacologic options for dementia and comorbid conditions (insomnia, depression/anxiety, agitation), referrals for driving and home safety evaluations, and printable educational resources in English and Spanish.

The CI-CDSS content was developed and vetted by PCCs and a Dementia Expert Panel consisting of a geriatric psychiatrist, a neuropsychologist, a behavioral neurologist, and a geriatrician, followed by a pilot phase in 3 primary care clinics.

2.9.2. PCC and staff training

Intervention PCCs and clinic support staff were invited to participate in a 1-h webinar training focused on complexities of assessing and managing patients with CI and workflow, features, and functionality of the CI-CDSS. PCCs were offered 1 CME credit for attending the training and lunch was provided to increase attendance. The webinar was recorded and posted online for asynchronous viewing.

2.9.3. Fidelity monitoring

Treatment fidelity practices for this study are intentionally aligned with the NIH treatment fidelity framework [15]. Use of the CI-CDSS is measured primarily by assessing how often the CI-CDSS content is printed, and secondarily by how often PCCs open the online content or click on the CI-CDSS interactive modules. We expect high print rates based on previous studies using this system for management of cardiovascular risk and diabetes [16], with a goal of a 70 % print rate of the CI-CDSS at eligible encounters. This target rate was chosen based on previous work (chart reviews) showing that use of the CDSS was not appropriate in roughly 20–30 % of encounters due to the presence of more pressing issues, such as acute illness or injury or when urgent psychosocial issues take priority.

2.10. Data sources

2.10.1. Administrative data

Data for the CI-CDSS algorithms and for study and safety outcome calculations will be extracted from EHR and administrative databases. The CI-CDSS automatically collects data elements required for its algorithms from the EHR at each patient encounter, including demographics, diagnosis codes, lab values, medications, and orders/referrals. Algorithm results (e.g., intervention eligibility, treatment recommendations) are stored in the CI-CDSS repository and used to retrospectively identify study-eligible patients and characterize intervention delivery. Data elements required to calculate safety and study outcomes will be extracted from EHR (e.g., CI diagnosis, safety events) and administrative (e.g., cost of care) databases at the end of the study-wide observation period.

2.10.2. PCC surveys

The PCC surveys are fielded at baseline and 8 months follow-up and designed to take less than 10 min to complete. Each contains a series of items to assess self-reported confidence in, beliefs about, and opportunity to diagnosing and managing CI adapted from existing questionnaires and follow survey design best practices [8,17–19]. Survey respondents use 4-point Likert scales to rate their overall confidence in diagnosing and managing CI, confidence about specific elements such as conducting appropriate testing and patient and caregiver education, beliefs about consequences of diagnosis, and the degree to which systems factors like visit time and EHR usability are a barrier. Details of the survey design have been previously reported [20].

2.11. Outcomes

2.11.1. Effectiveness outcomes

The primary effectiveness outcome is detection of CI, as measured by an ICD-10 diagnostic code for CI (dementia or mild cognitive impairment [MCI]; F00-F03, F06, G30-G31.9) documented in the EHR at outpatient or inpatient encounters or added to the EHR problem list between the index visit and the end of the observation period. A secondary outcome, healthcare utilization costs, will be calculated among accrued patients who have health insurance with the health system. The costs of any service other than self-administered pharmacy that occurs on the date of an emergency department (ED) visit or during an inpatient stay (IP), inclusive of IP admit and discharge dates, will be calculated per person over from the index visit through the end of the observation period. Another secondary effectiveness outcome is change in self-reported confidence in diagnosis and management of patients with CI from prior to CI-CDSS implementation to 8 months post-implementation.

2.11.2. Safety events

A data and safety monitoring board (DSMB) has been appointed by the National Institute on Aging (NIA) to ensure participant safety and scientific rigor. Passive surveillance of EHR data is used to collect data on number of ED visits, in-patient stays, suicide attempts, and deaths that occur among all accrued patients in the year prior the index visit through the end of the observation period.

2.12. Analysis plan

The primary analysis will be carried out in the ITT sample using a generalized linear mixed model (LMM) to normalize the binary outcome via a distribution-appropriate link function (e.g., logit, log) and account for clustering within randomized clinics. The LMM parameters will include treatment group, patient covariates specified in a pre-specified statistical analysis plan and a random clinic intercept. The treatment group parameter is expected to be positive and statistically significant (two-tailed α = 0.05), indicating that patients in clinics randomly assigned to the intervention are more likely to receive a CI diagnosis during the observation period than patients in UC clinics. Pre-intervention characteristics related to the risk of CI or its diagnosis (e.g., age, sex, comorbidities) may be used as moderators in heterogeneity of treatment analyses.

The cost analyses will be carried out using data from the subset of ITT patients who have health system insurance using a 2-part model, in anticipation of a zero-mass of patients with no ED or IP costs. The first part of the model will estimate the likelihood of any ED visits or IP stays using the LMM approach that was used to predict CI diagnosis. The second part will employ a generalized linear model (GLM) [21,22] allowing clustering by clinic and controlling for demographics and baseline risk. Such analyses often specify a gamma distribution for health care expenditures with a log link to the explanatory variables. The distribution family will be chosen based on the data using a modified Park test and the link function using a Box-Cox test [23]. It is predicted that patients accrued from intervention clinics will have lower post-index health care utilization and costs.

Another secondary analysis will compare change in PCC confidence ratings from pre- to post-implementation by treatment group. These analyses will also use generalized LMM to account for correlation in repeated ratings from PCCs and to normalize the outcome distribution (e.g., binary, multinomial). LMM parameters will include treatment group and time indicators, the treatment by time interaction and a random PCC intercept. The treatment by time parameter is expected to be positive and statistically significant, indicating more increase in confidence from pre- to post-implementation among PCCs at intervention clinics relative to UC clinics.

2.12.1. Sample size calculation

An a priori power analysis (power = 0.80, α₂ = 0.05) estimated the minimum detectable proportion of intervention relative to UC patients with a new CI diagnosis given assumptions about analytic sample size (n = 100 accrued patients / clinic), the proportion of UC patients with a new CI diagnosis (8 %, 12 %, 16 %) and clinic intraclass correlation (ICC_clin = 0.03, 0.04, 0.05). The correlated sample size estimate, N, was divided by the design effect (deff; 1 + (n-1)*ICC_clin) to estimate an effective independent patient sample size (i.e., N_eff = N/deff). Given these assumptions, the primary analysis is powered to detect a 5.6 % (UC = 8 %, CI-CDSS = 13.6 %, ICC_clin = 0.03) to 8.9 % (UC = 16 %, CI-CDSS = 24.9 %, ICC_clin = 0.05) difference in CI diagnoses.

3. Conclusions

The prevalence of CI is rapidly increasing as the global population ages, and timely diagnosis is essential to ensure quality care. A CI-CDSS that may help PCCs recognize, diagnose, and manage patients with CI is currently being tested in a large healthcare organization, with results expected in 2025. An early AD diagnosis has become critically important with the recent FDA approval of anti-amyloid monoclonal antibodies, which are indicated primarily for early-stage Alzheimer’s disease and likely have optimal efficacy when started sooner in disease process. If effective, this CI-CDSS would be scalable to help guide care for large numbers of patients and may facilitate rapid and consistent translation of evidence-based CI guidelines into personalized patient care.

Supplementary Material

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cct.2025.108080.

Data availability

Deidentified data will be made available upon reasonable request to the corresponding author.