Evaluating Cognitive Impairment in a Large Health Care System: The Cognition in Primary Care Program

Abstract

Background:

The prevalence of Alzheimer’s disease and related disorders (ADRD) is rising. Primary care providers (PCPs) will increasingly be required to play a role in its detection but lack the training to do so.

Objective:

To develop a model for cognitive evaluation which is feasible in primary care and evaluate its implementation in a large health system.

Methods:

The Cognition in Primary Care Program consists of web-based training together with integrated tools built into the electronic record. We implemented the program among PCPs at 14 clinics in a large health system. We (1) surveyed PCPs to assess the impact of training on their confidence to evaluate cognition, (2) measured the number of cognitive assessments they performed, and (3) tracked the number of patients diagnosed with mild cognitive impairment (MCI).

Results:

Thirty-nine PCPs completed the training which covered how to evaluate cognition. Survey response rate from those PCPs was 74%. Six months after the end of the training, they reported confidence in assessing cognition (mean 4.6 on 5-point scale). Cognitive assessments documented in the health record increased from 0.8 per month before the training to 2.5 in the six months after the training. Patients who were newly diagnosed with MCI increased from 4.2 per month before the training to 6.0 per month in the six months after the training.

Conclusions:

This model for cognitive evaluation in a large health system was shown to increase cognitive testing and increase diagnoses of MCI. Such improvements are essential for the timely detection of ADRD.

INTRODUCTION

Alzheimer’s disease and related disorders (ADRD) are the primary causes of cognitive impairment, currently affecting more than six million Americans, and this number is expected to double by 2060 [1]. As a result, health systems will face enormous challenges in caring for patients with ADRD. The availability of serum biomarkers [2] and specialists’ activities related to administering disease-modifying therapies will put primary care even more at the forefront of detecting cognitive impairment and performing initial counseling for patients with early-stage disease [3].

Despite awareness that earlier diagnosis leads to better outcomes [4], ADRD remains significantly underdiagnosed in primary care [1]. Primary care providers (PCPs) indicate that they need more knowledge about early detection and more confidence in managing ADRD to close this gap [5–7].

The Gerontological Society of America (GSA) created the GSA KAER Toolkit (Kickstart, Assess, Evaluate, Refer) to increase early detection of ADRD in the primary care setting [8]. The GSA KAER Toolkit is a nationally recognized process model of expert recommendations, providing teams with resources to choose from when designing early detection initiatives [9]. While care models have been developed creating separate, integrated diagnostic services [10–12], to our knowledge no one previously has studied implementing a care pathway of tools, together with provider training, which are embedded in existing primary care practices.

The aim of the Cognition in Primary Care (CPC) Program was to adapt the GSA KAER Toolkit into a model for promoting the early detection of ADRD. We sought to find an approach which could easily be adopted by PCPs without requiring the creation of new or added services. We report the results of implementing the CPC Program in a network of 14 community-based primary care clinics within a large U.S. health care system.

METHODS

Setting

The CPC Program was developed in 2021 using the GSA KAER Toolkit. It was designed as a model for cognitive assessment which could easily be implemented in existing primary care settings. We then tested this model across a network of 14 community-based primary care clinics at UW Medicine, a vertical-delivery health system serving the western part of Washington State.

UW Medicine Primary Care is an integrated network of primary care clinics affiliated with the University of Washington. More than 150 PCPs work across 14 community-based clinics, providing close to full-time patient care with less than 5% time supervising trainees. Patient demographics at each of the 14 clinics vary across a wide range from 12% to 22% being over age 65 (mean 19%), 38% to 62% identifying as non-white (mean 41%), and 6% to 42% eligible for Medicaid (mean 13%) reflecting a makeup of patients from diverse cultural backgrounds and income levels. Clinics in the primary care network deliver more than 330,000 patients visits per year across an 800-square-mile region.

Development of the CPC program

Beginning in March 2021, two team members (BG, JR) conducted semi-structured interviews with ten PCPs at UW Medicine [13]. During interviews, PCPs were shown options for implementing each step outlined in the GSA KAER Toolkit. We used feedback from those interviews to inform an adapted model for early detection of cognitive impairment, one which PCPs expressed they would be willing to implement and which they believed would be sustainable in their practice.

PCPs identified two evidence-based instruments which would be most acceptable to them to implement based on their feasibility and accuracy: the Montreal Cognitive Assessment (MoCA) and the Ascertain Dementia 8-Item Questionnaire (AD8). The MoCA has been shown to have high sensitivity and specificity for detecting mild cognitive impairment (MCI) in a wide range of primary care settings [14–16]. The AD8 is a simple, validated, and culturally sensitive questionnaire for obtaining observer input about patients’ day-to-day function [17, 18]. The two instruments have demonstrated high accuracy when used together for diagnosing MCI in the primary care setting [19].

We developed a set of checklist tools for PCPs to use in the electronic health record (EHR), utilizing contextual tip-reminders to help PCPs perform efficient, multi-domain cognitive evaluations. We tested and refined these tools, gathering feedback from a group of six PCPs, and then integrated them into the UW Medicine Epic EHR (Epic Systems, Verona, WI).

Education training on the use of these tools was careful to acknowledge scenarios in which the testing may yield equivocal results. PCPs were encouraged to refer patients for further evaluation if the results were uncertain, and to flag patients for repeat testing if patients declined specialist evaluation.

To inform the community support step of the GSA KAER model, several team members (AF, BB, JR, MZS) conducted structured interviews with six experts from dementia care-support fields to compile high-quality community support activities and care resources which would have significant value for PCPs to provide to patients and their care partners.

Early in the process, we learned from PCPs that it would be important to add education on managing ADRD to the training, and not just focus on its detection. PCPs stated that if they had increased confidence in knowing the steps to take after detection, they would be more likely to proceed with evaluations. As a result, we were careful to include training on the management of ADRD in our model, a feature which is especially important given the difficulty many PCPs experience accessing timely specialist care, as well as patients’ sometimes reluctance to follow through with a referral to see a specialist [5].

Description of the intervention

The CPC model was introduced through a PCP education curriculum and integrated into clinical practice via a set of EHR workflow tools meant to streamline cognitive evaluations. The CPC model provides PCPs (physicians, nurse practitioners, and physician assistants) with a stepwise, evidence-based approach to efficiently evaluate cognition in the primary care setting.

Education training

At the center of the CPC model is education. This focus on education is driven by research showing that (1) PCPs have interest in learning how to evaluate cognition and (2) that gaps in their knowledge are obstacles to better ADRD care [5–7]. Informed by our interviews with PCPs, the CPC educational curriculum highlighted actionable steps that PCPs can take to promote brain health, as well as a framework to counsel newly diagnosed patients, and steps to improve management of ADRD.

The educational training was delivered as three 45-minute live web-based seminars by two primary care providers (BG, JR) with extensive expertise in ADRD. Sessions were offered multiple times, both at noon and at the end of the workday. PCPs did not receive protected time to attend the training.

The sessions covered how to make a diagnosis of ADRD, how to set a plan for the newly diagnosed patient, and how to manage ADRD as it progresses. The training gave guidance on when urgent referrals to specialty care are needed, how to compassionately communicate a new diagnosis, interventions to maintain brain health and access quality support resources, how to treat behavioral symptoms, and how to provide better advance care planning, presented in a framework of collaborative care with specialists. The content was presented along with standardized tools for clinicians to use in their practices (Supplement Figure 1).

Standardized workflow tools

Based on the GSA KAER Toolkit and feedback from our interviews, the CPC model presented a step-by-step workflow for cognitive evaluation, using structured EHR tools to put the educational training into practice. The four steps in the workflow are (1) increased awareness of cognitive impairment, especially at annual wellness visits, (2) decision-aid signs which always should prompt full cognitive assessments, (3) how to perform cognitive evaluations in a dedicated, structured evaluation visit, and (4) how to counsel patients and care partners if the evaluation suggests a diagnosis of possible ADRD.

The initial awareness step introduces specific scenarios, such as the Medicare Annual Wellness Visit, when cognitive concerns may arise. PCPs are encouraged to utilize an adapted version of the Alzheimer’s Association’s “Warning Signs of Alzheimer’s” to help decide when to strongly advise a patient to return for a dedicated cognitive evaluation visit [20]. We considered emphasizing population-based screening for all patients in primary care, but heard in our PCP interviews strong guidance that they were much more likely to adopt a model which focused on evaluating cognitive concerns. Evaluating cognitive concerns is a readily identified need in their practices, so we reasoned that a model built to address this need was more likely to be successful. In addition, many PCPs pointed to national guidelines which do not recommend screening all patients with a cognitive test as another reason they would be hesitant to adopt such an approach into their practices [21].

We developed a structured EHR checklist that provides guidance during a dedicated visit to evaluate cognition. We emphasized the importance of a family member or close friend accompanying the patient to a cognitive evaluation visit. The evaluation consisted of administering a MoCA and an AD8 questionnaire and reviewing a concise list of reversible conditions which can impair patients’ cognition. This checklist provides PCPs with easy steps they can take to improve the brain health of their patients (Fig. 1).

Fig. 1.

Checklist embedded in the electronic health record to provide structure for cognitive evaluations. PHQ-9, Patient Health Questionnaire-9; MoCA, Montreal Cognitive Assessment; AD8, Ascertain Dementia-8. Text in brackets are vanishing tips giving additional guidance and disappearing once the note is signed.

In the final workflow step, PCPs were given a structured checklist for counseling patients who are newly diagnosed with ADRD, including guidance on how to set an initial management plan. The CPC training included evidence-based serious-illness communication tips [22] adapted for use in ADRD, as well as community resources to refer patients to, and specific interventions PCPs can initiate to help maintain their patients’ cognition going forward [23].

The Epic checklists were built into the EHR as a set of Smart Phrases which PCPs could access through a single organized Smart Set. Each Smart Phrase contained a tag which allowed us to measure its use as a discrete data element.

Training protocol and evaluation

The CPC training protocol was designed as a quality improvement program to be implemented across 14 clinics in the UW Medicine Primary Care Network. The University of Washington Institutional Review Board reviewed and exempted this initiative as quality improvement.

In October 2021, we offered the CPC training to all 153 PCPs in the UW Medicine Primary Care Network. We added the Epic tools to the EHR system at the same time, making them available to all practicing clinicians at UW Medicine. Each of the 45-minute online training sessions was held four times to maximize opportunity for training enrollment. Attendance was offered free of charge. Participants could pay an optional $25 fee to receive continuing education credits. After the sessions, summary handouts of the training contents and clinical tools were made available on the project website [23].

Six months after the training was complete, we invited all participants to fill out an anonymous survey on REDCap [24], asking participants to rate (on a 5-point scale) the program’s effect on their knowledge and confidence to assess cognitive concerns. A $35 gift card was offered to those who completed this survey.

Participants and outcome measures

Study subjects included all PCPs (physicians, nurse practitioners, or physician assistants) who practice at one of 14 community-based UW Primary Care Network clinics and who completed at least the first session of the training. This session was required because it described how to perform a cognitive evaluation using the outcome measurement tools related to the detection of cognitive impairment.

We defined our outcomes based on Kirkpatrick’s model for education assessment. This model focuses on four levels: (1) the response of the trainee to the training experience (i.e., perceived gain in knowledge); (2) the learner’s increase in attitude about the content (i.e., perceived increase in confidence); (3) changes in behavior (i.e., whether training translated into use of clinical practice tools); and (4) impact on patients (i.e., changes in the number of patients with new diagnoses) [25].

The primary outcomes for this intervention were (1) self-reported efficacy by PCPs of the effect of the program on their knowledge and confidence after taking the training; (2) participant use of checklist tools embedded in the EHR; (3) changes in the number of MoCA results participants entered in the EHR six months before the start of their training and six months after completion of the training; and (4) changes in the number of unique patients for whom PCPs documented a new MCI diagnosis code (ICD-10 G31.84) in the six months before versus the six months after the training. We assessed MCI diagnoses given the CPC program’s goal of improving early detection of ADRD.

Data collection

Working with an Epic data analyst, we created Epic reports specifying the data elements needed to extract the required outcomes. Data from the 6-month post-training survey were compiled, and quantitative data were summarized using means and standard deviations (SDs). To measure the use of the EHR checklist tools, we extracted instances where the full text of the tools appeared in PCPs’ EHR notes. We also extracted the number of MoCA test results entered by PCPs, and the number of unique patients for whom a code of MCI was billed for at least one visit.

Data were collected for the 12-month study period: from May 2021 through October 2021 (the six months before the training) and from November 2021 through April 2022 (the six months after the training.) MoCA tests were identified when the numeric result of the test was entered into the EHR flowsheet at the time of an office visit with a participating PCP. Unique patients seen with a diagnosis code of MCI were counted at the time of their first visit during the 12 months of the study in which a participating PCP attached an MCI code to a patient visit. To increase the likelihood that such patients represented patients who, in fact, were receiving a new diagnosis, we were careful to exclude from this count any patients who had an encounter in the 24 months prior to the start of the study where an MCI diagnosis was coded.

Statistical analysis

Descriptive statistics were used to summarize the survey data. For continuous variables, mean values (SD) are reported. For categorical variables, frequency with associated percentages was used. For survey questions with a ranking of 1 to 5 (not at all likely to very likely), mean values (SD) are presented. For MoCA tests and MCI diagnosis data, a calculation of means, medians, and interquartile ranges was performed. All analyses were calculated using SAS version 9.4 (SAS Institute Inc, Cary, NC).

RESULTS

One hundred fifty-three PCPs were invited to take the training. Thirty-nine completed part one of the training which covered how to perform a cognitive evaluation. This group of PCPs (n = 39) was analyzed for the before and after outcome assessment measurements (entry of MoCA results and new MCI diagnoses) given that it was part one of the training which covered how to perform those activities. Of those 39 PCPs, 22 took all three parts of the training, 14 took two parts of the training, and 3 took only the first part of the training.

Twenty-nine out of thirty-nine PCP study subjects responded to the survey for a response rate of 74.4%. The majority of the respondents were female (75.9%), their average time in practice was 13.8 years (range 1–40, SD 10.6), and the average number of patients over age 50 they estimated seeing per week was 34 patients (range 15–75, SD 13.9) (Table 1).

Table 1.

Survey of PCP study subjects six months after completing training (n = 29)^∗

Respondent characteristics Women, % (n)	75.9% (22)
Years in practice, mean (SD)	13.8 (10.6)
Estimated number of patients over age 50 seen per week (SD)	34.0 (13.9)
Professional role:
Physician, % (n)	72.4% (21)
Nurse practitioner, % (n)	20.7% (6)
Physician assistant, % (n)	6.9% (2)
As a result of this course:
I gained more knowledge about assessing cognitive concerns, mean (SD)†	4.8 (0.4)
I increased my confidence assessing cognitive concerns, mean (SD)†	4.6 (0.7)
I am more likely to assess cognitive concerns when they come up with one of my patients, mean (SD)†	4.6 (0.7)

PCP, Primary care provider; SD, standard deviation.

^∗Out of the 39 PCPs who completed part one of the training (response rate 74%).

^†Five-point rating scale: 5 = strongly agree, 1 = strongly disagree.

Participants strongly agreed that the educational training had a positive impact. Six months after completing training, they reported they had gained knowledge and confidence in assessing cognitive concerns, mean 4.8 (SD 0.4) and 4.6 (SD 0.7) respectively, as well as increased likelihood of assessing these concerns when they arise, mean 4.6 (SD 0.7).

Checklist tools in the EHR were widely utilized among participants to assist in cognitive evaluations. Based on examination of EHR data, 29 PCP participants (74.4%) used one or more of the tools during the six months after their release at the start of the training. Among all 39 eligible PCPs, the average number of MoCA tests entered as numeric results in the EHR (in total for the cohort) increased from 0.8 (SD 1.2) per month in the six months before the training to 2.5 (SD 1.4) per month in the six months after the training (Fig. 2). There were also increases in the number of unique patients receiving new MCI diagnoses during visits with eligible PCPs. A new MCI diagnosis was documented for an average of 4.2 (SD 2.4) patients per month prior to training, and for an average of 6.0 (SD 2.5) patients per month afterwards.

Fig. 2.

Monthly MoCA test results entered by the study group and number of unique patients receiving MCI diagnosis code in the 6 months before and the 6 months after training. MoCA, Montreal Cognitive Assessment; MCI, Mild cognitive impairment. Boxplots present median, interquartile range, minimum, and maximum number per month for each 6-month period.

DISCUSSION

The CPC program, consisting of training for PCPs and tools for early diagnosis of ADRD, was implemented across 14 primary care clinics and found to be acceptable and beneficial in increasing clinician knowledge, confidence, and likelihood of assessing cognitive function. Reflecting their greater knowledge and confidence, their performance of cognitive assessments and the number of new MCI diagnoses they made increased in the six months after implementation.

The strengths of this quality improvement initiative include its design to address known obstacles for primary care, and its implementation within an existing primary care network without needing to build new clinical services. As a result, its applicability to other health settings is high. To our knowledge this is the first time a process model such as GSA KAER for early detection of ADRD has been adapted and shown to be useful in a large network of community-based primary care practices.

An additional strength is its use of clinical tools for PCPs to use in their existing work environments, not just providing clinician education. Incorporation of workable tools like this is recognized as a key feature for effective quality improvement interventions to affect change in clinical practice [26].

Another strength is its emphasis on online training, making it well suited for dissemination to other health systems, especially those in remote and rural areas. Its clinical tools are designed to be simple, making it easy for other organizations to adapt and use in a wide variety of EHRs.

This study has several limitations also. First, it included only MoCA tests which were entered into the EHR as numeric results. We found it impractical to capture MoCA tests charted only as free text in progress notes, or those only entered into the EHR as a scanned document. Selected chart review identified MoCA tests that were performed but which were not entered into EHR flowsheets, leading us to suspect that the MoCA outcome we report in this study is likely an undercount of total MoCA tests performed (data not shown). We report this more limited outcome (only numerically entered MoCA tests) because such results are more easily located in the EHR, more easily tracked over time, and so are more likely to be clinically useful and more reliably measurable by health networks.

Second, as a descriptive study, this analysis was not designed to make causal inferences on the outcomes measured. In particular, we did not obtain self-assessments, before training of PCP confidence. The finding that PCPs identified a high level of value to the training six months after its completion is notable, however. Despite the difficulty in ascribing causal effects for the outcome measures, given the challenges of assessing cognitive concerns in primary care, the findings of a broad implementation for a process improvement such as this do still provide useful information for guiding future efforts to address gaps in the evaluation of ADRD.

Third, this study’s period of analysis coincided with the peak of the COVID-19 pandemic, potentially affecting study outcomes. All 14 clinics remained open throughout the pandemic, but total visit volume fell significantly from March 2020 until the summer of 2020, after which they rebounded to baseline levels. The period before-training (from May to October 2021) coincided with a lull in COVID-19 cases, while the period after-training (from November 2021 to April 2022) coincided with the peak of COVID-19 cases due to the Omicron variant. As a result, if the COVID-19 pandemic did significantly impact study outcomes, we believe that it is more likely that it lowered any efficacy we would have been able to detect, and was therefore unlikely to be a potential confounding source for our positive findings.

Fourth, this analysis was conducted in clinics which are affiliated with an academic medical center. However, almost all the community-based PCPs included in this study were engaged only in direct patient care, spending less than 5% of their time supervising trainees. As a result, findings from this evaluation are more likely to be applicable to other community-based primary care settings.

Fifth, in our analysis of patients who were newly coded with an MCI diagnosis, due to study constraints it was not possible to be certain that this was the first time they had been determined to have MCI. Such determination in conditions such as MCI is especially difficult given its insidious onset and clinicians’ hesitation at times to code this diagnosis even if they believe it is likely to be present [27]. To address this concern, and improve the confidence of our measurement, we excluded patients who had had a visit with an MCI diagnosis code in the 24 months prior to the start of the study. The higher frequency of unique patients in which an MCI code was added in this study does likely reflect increased PCP confidence in identifying MCI.

Sixth, the cognitive test employed in this intervention, the MoCA, has been shown to be less accurate in some racial and ethnic minority groups. Further work is clearly needed to develop cognitive tests which better meet the needs of more diverse patient populations [28].

Planned future efforts for this work include expanding the education training to be offered as publicly available streaming videos and an interactive online module. Such training options could further help reach a wider audience and further improve the care of patients living with ADRD. Future planned work will also focus on tracking patient level outcomes, including how many patients receive subsequent care following their initial evaluation.

This study encouraged PCPs to have discussions with older patients around brain health. These discussions offer an opportunity to detect and diagnose ADRD earlier, to increase treatment and clinical trial enrollment, and to improve health and chronic disease management [4, 29]. Such goals are critical public health strategies aligning with health objectives in widely recognized reports, such as the CDC’s Healthy People 2030 initiative [30].

PCPs report they are eager to learn how to identify and evaluate ADRD [5]. This study suggests that an education intervention, paired with practical EHR tools, can increase PCPs’ confidence and willingness to engage in cognitive evaluations.

The need to facilitate early detection of ADRD is great, and the role that primary care should play in this effort is widely acknowledged [31]. This study describes an effective, innovative model to meet this need. Providing a structured framework to facilitate early detection is essential if we hope to engage PCPs and bring effective treatment to people with ADRD.

As the number of people with ADRD increases, and as disease-specific medications for use in MCI emerge, the benefits of early cognitive evaluations will continue to grow. Initiatives such as the Cognition in Primary Care Program can play an important role in the ability of health systems to provide high-quality, collaborative dementia care.

DATA AVAILABILITY

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy restrictions.