The Rural Older Adult Memory Study (ROAM): A Practice-based Intervention to Improve Dementia Screening and Diagnosis

Abstract

Introduction

The aim of the Rural Older Adult Memory pilot study (ROAM) was to evaluate the feasibility of screening and diagnosing dementia in patients aged 75 years or older in six rural primary care practices in a practice—based research network.

Methods

Clinicians and medical assistants were trained in dementia screening using the ROAM protocol, via distance learning methods. Medical assistants screened patients aged 75 and older. For patients who screened positive, the clinician was alerted to the need for a dementia work-up. Outcomes included change in the proportion of patients screened and diagnosed with dementia or mild cognitive impairment and clinician confidence in diagnosing and managing dementia. Patient, medical assistant, and clinician response to the intervention were also evaluated.

Results

Results included a substantial increase in screening for dementia, a modest increase in the proportion of cases diagnosed with dementia or mild cognitive impairment, and improved clinician confidence in diagnosing dementia. Although clinicians and medical assistants found the ROAM protocol easy to implement, there was substantial variability in adherence to the protocol in the six practices.

Conclusion

This study demonstrated the complex issues that must be addressed in implementing a dementia screening process in rural primary care. Further study is needed to develop effective strategies for overcoming the factors that impeded the full uptake of the protocol, including the logistical challenges in implementing practice change and clinician attitudes towards dementia screening and diagnosis.

Dementia is common in older adults, estimated at 11% - 16% of people older than 70 (1). Often, a primary care clinician (PCP) is the only clinician available to older patients with memory complaints, but numerous studies have found that as many as 50% of patients with dementia do not have a diagnosis of dementia documented in their medical chart (2-5). The subtlety of symptoms and time constraints in primary care practice make it challenging for PCPs to recognize and diagnose dementia (6,7).

Although the US Preventive Services Task Force (USPSTF) has not recommended screening for dementia (8),. Alzheimer's disease experts identify a number of reasons why diagnosis is important, including ruling out treatable conditions that can cause cognitive impairment (e.g., medication effects, cardiovascular conditions, or depression)(9), offering treatments for cognitive and behavioral symptoms of dementia (10), maintaining the patients' safety (11), and support for the family (12). Given clinicians' reliance on patients' memory for symptom reporting and adherence to treatment recommendations, the identification of cognitive impairment in patients is essential (13-15).

Several studies to improve the diagnosis of dementia conducted in academic and/or urban settings have had varying degrees of success (3,16,17). This is the first known study to test a dementia screen and diagnosis intervention in rural primary care. Improving dementia care is particularly challenging in rural areas where access to community resources, including medical specialists, are limited and primary care workloads are greater than in urban areas (18,19).

Practice-based research networks (PBRN's) offer a promising approach to improving primary care as clinicians and their staff are directly engaged in testing practice changes (20,21). Among other resources, PBRNs can provide direct assistance to practices engaged in the quality improvement studies through a “change facilitator” (22).

The ROAM study was carried out in the Oregon Rural Practice-based Research Network (ORPRN) a statewide network of 46 primary care practices in 35 communities that serve approximately 223,000 patients. Our aim was to evaluate the feasibility of ROAM for dementia screening and diagnosis in six rural practices that were members of ORPRN. We adapted the materials and procedures for the dementia-specific component of the Assessing Care of the Vulnerable Elders model (ACOVE), developed by geriatric experts at UCLA and Rand (17). ACOVE uses five clinic-based methods to improve practice: efficient collection of condition-specific clinical data, medical record prompts to encourage performance of essential care processes, patient education materials and activation of the patient's role in follow-up, physician decision support, and physician education. Preliminary tests of this model found it to be feasible in urban and suburban practices but to our knowledge, it has not been tested in rural practices. Furthermore, the ACOVE model has had limited testing of its dementia component as a single-condition intervention.

Methods

Study Setting

The OHSU Layton Aging & Alzheimer's Disease Center worked in collaboration with ORPRN to implement the intervention. Eighteen clinicians and 26 medical assistants in six rural practices in western Oregon participated in the intervention. Using a convenience sample, we recruited practices of varying size, with the proportion of elderly patients in their patient profiles ranging from 6% to 25%, a mix of electronic medical records versus paper medical charts, and both private and publically funded practices. IRB approval was obtained and all study clinicians and MAs signed informed consent.

To facilitate implementation of the intervention, we engaged ORPRNs Practice Enhancement & Research Coordinator (PERC). The PERC served as a liaison between the practices and the research team, assisted practices when difficulties arose in implementing the intervention, and kept an eye on adherence to the protocol and the project timeline.

Intervention

The intervention consisted of a memory screen administered by a medical assistant and, for patients who screened positive for possible dementia, a memory evaluation by the clinician. Patients were screened over a three-month period with an additional two months allowed for memory evaluation by the clinicians for positive screens. Given the short time frame for this study, we did not address management of dementia for diagnosed patients; however, handout materials were provided for clinicians to give patients or family members of persons diagnosed with mild cognitive impairment impaired. We included medical assistant concern as a possible indicator because we theorized that medical assistants might be aware of a patient's memory problems, for example, if the patient appeared confused about their appointment schedule or had problems with medication instructions.

Memory evaluation by clinician

The Memory Evaluation form was provided to the clinicians for use with patients who screened positive. In addition to the single-page Memory Evaluation form, clinicians were provided with tools and instructions for completing the recommended tests. The memory evaluation included tests for five domains of cognitive function (memory, executive function, abstraction, construction ability, and verbal fluency) (25): Mental or dementia.

The “Memory Screen” included four indicators of possible dementia

First, patients were asked if they had noticed a change in their memory that concerned them. Second, if a family member or other informant was present, this person was asked if he/she had noticed a change in the patient's memory that concerned him/her. Although these two questions have not been formally tested for sensitivity and specificity, patient and family concerns about changes in memory are an important component of the diagnosis of mild cognitive impairment and dementia (23). The third item in the memory screen was Three Word Recall (24). Prior to taking vital signs, the patient was asked to remember the words, “pony”, “quarter”, and “orange”; the patient's vital signs were recorded; and then the patient was asked to recall the three words. Failing to recall ≥2 words on the Three Word Recall was considered positive. Finally, the memory screen form included a place for the medical assistant to check if he or she was concerned that the patient was cognitively

Status Examination, abstraction and judgment questions, Clock Drawing test (26), and verbal fluency (27). We also provided an Activities of Daily Living/Independent Activities of Daily Living scale (ADL/IADLs) (28), the “Get up and Go” test (29), the Geriatric Depression Scale (30), and the Caregiver Burden Assessment (31). These tests are consistent with quality indicators (32) and aided clinicians in recognizing alternative diagnoses and family needs. Instructions for the neurologic exam, and lab and imaging tests (10) were provided in the training and included on the Memory Evaluation form. In addition, clinicians were provided print materials on the following: prescribing for cognitive and psychiatric symptoms, driving assessment, referral cards for the Alzheimer's Association, and other information sheets for caregivers.

Training

Study clinicians, medical assistants, nurses, and front office staff participated in a 2-hour training on recognition and diagnosis of dementia, principles of dementia care management, information on community resources, and the intervention protocol. The training was presented by the study geriatrician (EE), the director of the Alzheimer's Association Oregon Chapter, and the PI (LB) via web conference (one-way PowerPoint presentation and two-way audio). Each practice receiving the web conference training had on-site support from a member of the ORPRN staff to ensure smooth operation of the video equipment. The training was audio taped and distributed to clinicians and staff for later review.

Intervention protocol

For the three-month period of the intervention, all patients 75 and older seen by a study clinician were eligible for screening. If the patient was determined by the medical assistant to have a prior diagnosis of dementia, to have been prescribed dementia medications, or to be “too ill” to be screened, these were noted on the memory screen form and it was placed in the box of completed forms. If these were not the case, the memory screen process was explained to the patient at the time he or she was escorted from the clinic waiting room to the examining room. If a family member was present, he or she was invited to accompany the patient for the first part of the visit. If the patient refused the screen, this was recorded on the form and the form was placed in the collection box. If one or more of the 4 screen indicators was positive, the Memory Screen Form and the Memory Evaluation form were placed on the patient's chart to alert the clinician of the need for a memory evaluation. The memory evaluation could be completed either at that visit or at a separately scheduled visit. For patients who screened positive, we collected data on workups carried out during the next two months following the end of the screening period.

Intervention Evaluation

Outcomes of interest included change in clinician confidence and comfort in diagnosing and managing dementia and changes in the incident diagnosis of dementia. Additionally, we evaluated the adherence to the intervention protocol; and patient, medical assistant, and clinician response to the intervention.

Pre-intervention chart review

To compare the intervention with pre-intervention practice, records from patients aged 75 or older who were seen by study clinicians during a two- to four-week period in October 2006 were reviewed for incident dementia diagnosis over the next 4 months. The number of patient charts reviewed per clinic ranged from 24 to 67 (total of 310 charts).

Dementia Care Confidence Scale (DCCS)

The DCCS is an 8-item scale clinicians to rate their confidence level in assessing and diagnosing dementia, treating symptoms, managing care, differentiating delirium from dementia, differentiating depression from dementia, advising patients and family about community resources, and comfort in disclosing a diagnosis of dementia to a patient and family (33). We administered the DCCS as part of a written survey completed by study clinicians before and after the intervention.

Patient satisfaction surveys

Patients were asked by the medical assistant to complete a brief written satisfaction survey immediately after they were screened. The patient survey asked if they had any concerns about the screening and if, in general, they thought it was a good idea for PCPs to assess older patients' memory and thinking. Patients were also invited in the survey to explain any discomfort they experienced and if there was anything that could be done differently in the screening.

Feedback sessions and surveys of clinicians and practice staff

We surveyed clinicians about their response to the ROAM intervention and conducted feedback sessions in each practice with the clinicians and practice staff who participated in the intervention. Post-intervention sessions were audiotaped and transcribed.

Data Analysis

We used descriptive statistics to report the uptake of the intervention. We compared the proportion of dementia-related diagnoses for eligible patients made during the intervention with the proportion of dementia-related diagnoses made for eligible patients in the pre-intervention period covered in the chart review. Wilcoxon Signed Rank Tests were used to compare change in clinician confidence in dementia diagnosis and management before and after the intervention. We used chi-square analysis to assess the effect of gender and age on the proportion of screened patients with positive screens, with Mantel-Haenszel statistic to test for linear effect of age on positive screens. The post-intervention clinician survey and group feedback session transcripts were reviewed by co-authors to gain insight into clinician and medical assistant satisfaction with the intervention and to assess the feasibility of the protocol.

Results

Participation

Eighteen clinicians (11 family practice physicians, 1 internal medicine physician, 4 physician assistants, and 2 nurse practitioners) and 26 medical assistants participated in the study.

Intervention process

Patient screening

Seven hundred three patients 75 or older were seen by study clinicians during the intervention period (Figure 1). Of the 703 patients 75 or older who were seen by participating clinicians during the intervention period, medical assistants initiated a screen form for 495 patients (70% of the eligible patients). Of the 427 patients who were given the 4-component screen, 198 (46%) screened positive. Ninety-seven screened patients (23%) were accompanied by a family member. Of these, 72 family members (74%) reported concerns about the patient's memory. As expected, older patients were more likely to screen positive (chi-square, p=.001, with a highly significant linear trend (Mantel-Haenszel test, p<.0001)). There was no significant difference in gender between the positive and negative screens.

Figure 1. Intervention Protocol.

Of the patients who screened positive, 73 (37%) had “workup scheduled” checked on the screen form. For the four indicators of possible dementia, the proportion of patients who screened positive who were scheduled for memory evaluation were as follows: three-word recall: 39%; patient report: 37%; family report: 43%; and screener (medical assistant) concern: 77%. Medical assistant concern was the only screen item that had a significant association with “workup scheduled” checked (chi sq p<.0001). For the 125 patients with “workup scheduled” not checked, in 45 cases, no reason was given. For the others, the clinician wrote that the patient refused a memory evaluation (n=36), or that the memory evaluation was not necessary (n=39), or gave other reasons for not scheduling a memory evaluation (n=4). Twenty patients (4%) who were offered the screen refused and 36 (18%) patients with a positive screen refused a diagnostic evaluation.

Dementia Evaluations

Sixty-one patients received a clinical evaluation for dementia, which was carried out in a separate 30-40 minute visit.

Outcomes

Dementia diagnosis

The patients included in the pre-intervention chart review and those seen during the intervention were comparable with respect to age and gender. For the 310 pre-intervention charts reviewed, a total of 3 patients (0.97%) had a recorded new diagnosis of dementia (n = 2) or MCI (n = 1). Among the 703 eligible patients seen during the intervention period, a total of 19 (2.7%) were newly diagnosed with dementia (n = 5) or MCI (n = 14)(p=.06 for difference between groups). With respect to change in clinician confidence and comfort in diagnosing and managing dementia, all 18 clinicians completed the DCCS, and there was a significant improvement in confidence in differentiating depression and dementia (p=0.01). We also found trends in improvement in clinicians' confidence in differentiating delirium and dementia, in treating the symptoms of dementia, and in clinicians' comfort in disclosing the diagnosis to the patient (p-values from 0.06 to 0.10).

Response to intervention

Patient response to screening

Three hundred twenty-five (76%) screened patients completed the survey to assess their response to screening. Ninety-eight percent of respondents reported no concerns or that they were “pleased to have [their] memory evaluated.” No patients reported that they were “a lot uncomfortable” with the memory screening. Ninety-one percent responded that, in general, memory evaluation for older patients was a good idea.

Clinician Response to Intervention: Survey Results

All study clinicians completed the post-intervention survey and all rated the ROAM project as moderately or highly successful in increasing the identification of dementia, in increasing their knowledge about dementia diagnosis, and in their overall assessment of the intervention. Clinicians commented positively on knowledge gained and on the materials provided. Some wrote that they appreciated the opportunity for focused time to address a single issue and for communication with patients about dementia. A number of clinicians reported a desire for further training about the diagnostic protocol and interpretation of the diagnostic tests.

Feedback group results

The clinicians spoke highly of the CME training, the ease in which the screening process was implemented in their clinics, and the quality of the materials provided by the project. As in the written survey, some clinicians reported that they would have liked more training on interpretation of cognitive tests used in the diagnostic workup. A number of the clinicians and medical assistants commented on their surprise that the patients responded so positively to the screening. The medical assistants also reported that the screening process was easy to incorporate into their routine.

Individual practice differences

Despite the overall positive results from the intervention, substantial differences among the practices in adherence to the intervention protocol affected the outcomes as shown in Table 1. To better understand these variations, we reviewed the transcripts of the feedback sessions and discussed the implementation of the intervention with ORPRN staff. We identified two primary factors that influenced these process variations:

Table 1. Practice level data for intervention protocol.

Practice ID	(1) No. of participating clinicians	(2) % of patient profile over 75 years*	(3) Patients >75 yrs seen during intervention period (n)	(4) Screen forms initiated n (%)†	(5) Patient screens completed N (%)‡	(6) Positive screens n (%)§	(7) Patients scheduled for workup n (%) ║	(8) Patients evaluated n (%)¶	(9) Patients diagnosed with dementia# n (%)
A	3	13%	156	134(86%)	108 (81%)	68(63%)	29 (43%)	28(97%)	7(10%)
B	2	20%	152	96(63%)	82 (84%)	33(40%)	5 (15%)	3(60%)	2(6%)
C	2	9%	96	87(91%)	77 (88%)	40(52%)	8 (20%)	7 (88%)	2(5%)
D	3	25%	88	83(94%)	77 (93%)	30(39%)	15 (50%)	12 (80%)	6(20%)
E	2	10%	73	50(68%)	43 (86%)	12(28%)	10 (83%)	9 (90%)	3(25%)
F	6	6%	138	45(33%)	40 (89%)	15(38%)	6 (40%)	2 (33%)	0(0%)
Total (N=18)	18	n/a	703	495 (70%)	427 (86%)	198	73 (37%)	61 (83%)	19 (9.6%)

^*Percent of patients over 75 for practice overall. Data is from ORPRN survey conducted in 2005, except for Practice #2 who provided this information for this project.

^†Screen forms initiated (Col.4) refers to screen forms that were partially or fully completed including for patients who refused screening or were excluded from screening for other reasons.

^‡Percent with completed screens of patients who had screen forms initiated.

^§Percent is among patients for whom screen forms were completed

^║Patients who had “workup scheduled” written on screen form. Percent is among patients with positive screen.

^¶Percent is for patients with “workup scheduled” written on screen form who received dementia workup.

^#Percent is among positive screens.

1. Integrating the intervention protocol into clinical routines

Several practices appeared to have set up an effective process for identifying patients eligible for screening (A, C, D, see Column 4) and/or for ensuring that patients identified as needing a memory evaluation actually received one (A, C, D, E, Column 8). Practice F attributed their low proportion of patients screened to their being preoccupied with the implementation of a new electronic medical record. In addition, unlike the others, this practice did not create a centralized system (e.g., using staff at the front desk) to place the screen forms on eligible patients' charts. In Practice B, a participating medical assistant unexpectedly died during the intervention period, understandably interrupting the clinical routine during the early part of the intervention.

A number of start-up issues surfaced at the other practices as well, though these were mainly addressed by the ORPRN research coordinator. For example, in one practice the clinicians and assistants initially presented the screening as an option, i.e., they asked the patient if they would like to be screened (opt in) rather than presenting the screening as part of the routine with the opportunity for patients to refuse (opt out).

Another challenge in integrating the intervention into the practice routine was the allocation of time for training. Clinicians requested shortened training and resisted scheduling more than one training session. Nonetheless, a frequent comment following the intervention was the need for additional training.

2. Clinician attitudes

The low proportion of patients scheduled for a memory evaluation among those with positive screens in five of the six practices suggests that the clinicians' attitudes may have influenced adherence to this step in the process. Comments made by clinicians with a low percentage of evaluations support this:

“I would have to say this: I wonder really is diagnosing Alzheimer's really that big a problem. I mean, how many [patients] do you come up with where you had no idea, you had no suspect (sic) that this person had a problem…. I've never seen [medications] be very efficient.”

“ …it would be a lot easier to get enthused about a project that may have more medical relevance. Something that you might really get jazzed about.”

These comments are in contrast to those made by clinicians from practices with higher proportions of positive screens scheduled for diagnostic workup:

“I thought the study was very useful because it identified patients that I was not aware of that had problems, whether it was Alzheimer's or depression or other memory problems.” “I tend to think that the earlier [diagnosis] the better because it helps you think about your patient. It helps you think about their compliance, and helps you think down the road, what are we going to do, and it helps you get the family involved.”

Additionally, in some cases, the clinicians apparently misapplied the results of the delayed 3-word recall: if the person passed this brief memory test but the patient reported concerns about their memory, the clinician often discounted the patient's concerns and opted to not schedule a diagnostic evaluation.

Discussion

This feasibility study was, to our knowledge, the first application of the ACOVE model for screening and diagnosing dementia in rural primary care. We found that the ROAM intervention was easily integrated into the routine patient process by the medical assistants and was well received by patients. For clinicians, there was a more mixed response. Clinicians reported high satisfaction with the training, increased knowledge about dementia, and showed significant increases in confidence in diagnosing dementia; yet, the overall uptake of the intervention was modest and variable across practices, and the clinicians' responses in post-intervention assessments were mixed. The modest outcomes of the proportion of patients diagnosed with dementia may have been affected by these variations.

A major finding of this study was the reluctance of the clinicians to follow up on a positive dementia screen. Clinicians often determined that the symptoms did not warrant a dementia workup. While better training may have increased the proportion of patients with a positive screen who received a memory evaluation, a similar lack of follow-up on a positive dementia screen has been reported by other researchers (16) suggesting that other factors may be at work. The lack of perceived benefit of diagnosing dementia by some clinicians undoubtedly influenced their decision whether or not to schedule a memory evaluation. Other possible reasons for lack of follow-up might include lack of time, poor reimbursement, large patient volumes, the perceived complexity of the memory evaluation protocol, or lack of clinician comfort and confidence in making a diagnosis; however, these were not discussed in the feedback sessions.

Much of what we learned in this study related to the intensely busy environment in which primary care clinicians practice. Our study offers the following insights:

1. Distance learning using web-based technologies appears to be a feasible method for training busy rural clinicians and medical assistants. We noted the tendency of the clinicians to resist lengthy training but that once the intervention was under way, clinicians actually expressed a desire for further training. The high proportion of patients with a positive screen who did not receive a dementia evaluation suggests that better training on the value of diagnosis may be warranted. Research is needed to further develop and test modes and content of training as a means to improving adherence to quality practice standards.

2. The medical assistants played a key role in this intervention and their positive response suggests that greater attention be given to ways to engage them. Although we did include the medical assistant in our feedback sessions, we did not gather written input from them The increased likelihood that clinicians would schedule a work up for patients when the medical assistant noted a concern about the patient's memory was also intriguing, suggesting high value these clinicians place on their assistants' opinions. We believe, as others have recommended (33), that enhancing the role of medical assistants is a promising approach to improving care for older patients in rural communities. For example, medical assistants could be trained to assist in screening patients, administer cognitive and other tests used in diagnosis, and offer support to family caregivers and access to community resources.

3. While we successfully screened 70% of eligible patients, we recognize that our screening tool was not ideal. Several recent studies using different dementia screening instruments have reported higher sensitivity and specificity (3, 16). We chose our tool to ensure that it could be feasibly administered by medical assistants during a busy clinic, but would recommend a screening instrument with higher specificity and validity for use in primary care, such as the Mini-Cog (34) in future research. Another approach that merits consideration is the two-stage screening process used by Boustani et al. (3). Our screen was similar to the first-stage screen used in this study and had a similar proportion of positive screens (43% compared with 46%). With the addition of a 2^nd-stage, more sensitive 28-item screen, 13% in the Boustani et al. study were referred for diagnostic evaluation. Adding this screen would have increased the time demand on the medical assistant but could reduce the false positive screens.

4. To ensure maximum participation and minimum disruption of practice routines, on-site assistance from research staff is essential. The ORPRN Practice Enhancement Research Coordinator was instrumental at the start of the intervention when the practices were becoming comfortable with the intervention protocol and incorporating it into their clinical process. The research coordinator had good access to most of the clinicians and medical assistants when she visited the practices and this facilitated the intervention process. The intervention uptake would likely have also been improved by delaying formal data collection for a week or two after the intervention began.

Limitations and future research

This study is one of the few attempts to test a way to improve the identification of persons with dementia in rural primary care. Its limitations are its small size, that it was not randomized, and its short time frame. In addition to implementing a larger, randomized treatment-control intervention, in future research it will be important to identify a screening tool with better sensitivity and specificity than the one used in this study and to refine the processes for the diagnostic evaluation. The strengths of this study were that it was relatively easily integrated into the routines of the practices in which it was tested, that it resulted in high rates of screening and that patients responded very positively. In those situations where there were procedural glitches needing to be ironed out before the intervention could move forward smoothly, the ORPRN PERC provided essential assistance.

Further research is needed to address the challenges in implementing practice change in rural practices as well as in addressing the barriers to improving the recognition and diagnosis of dementia. Improving rural practice is especially challenging due to limited resources within the practice and in the community where they are located. As other researchers have emphasized (35), practice-based researchers must find the balance between stringent adherence to research protocol and the accommodation of variations in the practice's time, staffing resources, and clinician commitment.

Changing practice related to dementia diagnosis has its own challenges. The low rate of dementia workup found in this and other studies (16) for patients who screened positive for possible dementia needs to be better understood and strategies to overcome this are needed. The lack of follow-through on positive dementia screens may reflect time pressures on clinicians and staff, concerns about reimbursement for visits, and/or uncertainty about the appropriate procedures for the diagnostic workup. Alternatively, it may be due to clinicians' perceptions that dementia diagnosis is of limited benefit until more effective treatments are available. The mixed response of clinicians to this intervention should be viewed against the positive response of patients who overwhelmingly reported their appreciation about being screened and of the medical assistants who played a largely positive and supportive role in its implementation. This study demonstrated the complex issues that must be addressed in implementing dementia screening in rural primary care. The challenges of practice-change in busy rural practices are substantial. Understanding and accommodating the resources available to primary care practices and the attitudes and practices of clinicians are essential if we hope to develop interventions that will be accepted in real world settings.