Maximizing the “Functional” in the Functional External Memory Aid Tool for Patient-Centered Treatment Planning in Mild Cognitive Impairment

Abstract

Purpose:

The Functional External Memory Aid Tool (FEMAT) is an 11-item performance-based measure that simulates everyday tasks (e.g., medication management) to measure one's use of compensation (e.g., pill organizer). The FEMAT is not a norm-referenced diagnostic tool; rather, it is designed to provide clinicians with information about a patient's function and compensation in a standardized format. To provide further evidence for validity of the FEMAT, the purpose of this study was to conduct a mixed-methods analysis of FEMAT responses to operationalize the types of behaviors elicited during test administration.

Method:

We adopted an embedded single-case study design to analyze the FEMAT administrations of 12 community-dwelling female participants, ages 73–90 years, who met criteria for mild cognitive impairment. These participants were part of the original validation sample during which they completed the FEMAT (Version 1.0) in a single session with an assessor. Sessions were video-recorded and transcribed. We analyzed the data qualitatively in two phases using theme and subtheme codes and then quantitatively analyzed the data for response trends.

Results:

Our thematic analysis revealed four themes to operationalize participants' verbal and behavioral FEMAT responses: (a) used an external memory aid (EMA), (b) described using a strategy, (c) discussed memory and comprehension, and (d) discussed functional information. While completing the FEMAT, most participants described using a strategy (85%) and discussed their self-perceived memory and comprehension (33%). Several subthemes also emerged within each theme (e.g., self-perceived memory strengths and barrier).

Conclusions:

Our analysis suggests that beyond a 4-point item score, the FEMAT elicits behavioral (e.g., used an EMA) and verbal (e.g., discussion of health information) content that may be useful for person-centered treatment planning. This study is part of a comprehensive research agenda establishing the evidence for the validity of the FEMAT to support its use to inform person-centered treatment approaches.

Dementia is characterized by a modest decline in cognitive abilities that affects one's ability to complete daily activities independently (American Psychiatric Association, 2013). Individuals living with dementia from Alzheimer's disease and their loved ones face significant financial and emotional challenges due to the progressive nature and functional symptoms of the disease (Alzheimer's Association, 2020). Behavioral interventions that help community-dwelling older adults maintain their independence and possibly delay the transition to dementia are needed to reduce the burden on patients, families, and the health system (Livingston et al., 2020). Compensatory cognitive interventions are behavioral interventions that teach adults to use strategies or tools (e.g., calendars) to compensate for their cognitive impairments (e.g., declarative memory) to support their completion of daily activities (e.g., recalling the date of a doctor's appointment; Cicerone et al., 2000; Sohlberg & Mateer, 2001).

Research suggests compensatory cognitive interventions help older adults improve their daily task performance when delivered at a possible early clinical stage of the disease referred to as mild cognitive impairment (MCI)—a clinical syndrome characterized by a modest decline in cognitive abilities and preservation of functional independence (Greenaway et al., 2013; Kinsella et al., 2009; Rodakowski et al., 2015; Weakley et al., 2019). Although adults with MCI have preserved independence, the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (American Psychiatric Association, 2013) suggests that these individuals experience a level of cognitive decline that often requires compensatory strategies and accommodations to perform instrumental activities of daily living (e.g., managing finances and taking medication). Their use of compensation may be preventing further disability (i.e., dementia), highlighting the importance of providing compensatory cognitive rehabilitation at this early stage.

It is well within the scope of practice of speech-language pathologists (SLPs) to deliver compensatory cognitive interventions (American Speech-Language-Hearing Association [ASHA], 2016). Findings from a recent ASHA survey suggest that individuals with cognitive-communication disorders from Alzheimer's disease are major consumers of speech-language pathology services (ASHA, 2016). According to a survey conducted by Lanzi et al. (2022), SLPs reported that compensatory approaches are the most frequently delivered intervention for adults with MCI and early-stage dementia (Lanzi et al., 2022). Unfortunately, there is no gold standard for measuring everyday function and compensation, which is necessary for compensatory treatment planning. In fact, applied cognitive skills and attempted compensation are often unmeasured or poorly measured in clinical settings (Gold, 2012; Schmitter-Edgecombe & Farias, 2018). Poor measurement of compensation at baseline is problematic because it leaves the clinician without an effective means to monitor progress and assess treatment efficacy. It is insufficient to simply provide a patient with a calendar and assume it will be used functionally; instead, the external strategy must be appropriately selected, customized, and taught to target individual functional needs to support long-lasting effects (Sohlberg et al., 2022; Sohlberg & Mateer, 1989, 2001).

To address this problem, Lanzi et al. (2020) developed the Functional External Memory Aid Tool (FEMAT)—a performance-based measure of applied cognitive abilities and attempted compensation. The FEMAT has undergone rigorous developmental and psychometric methods and has published evidence for content validity (expert panel evaluation), concurrent validity (correlation with other measures), construct validity (factor analysis), and internal consistency (Lanzi et al., 2020, 2023). Unlike many performance-based measures that are commonly used by SLPs, the FEMAT was designed to help clinicians understand the nature of the cognitive weakness, how the weakness manifests in instrumental activities of daily living, whether and how the patient attempts to compensate for their cognitive challenges, and whether they are successful in compensating (Lanzi et al., 2020, 2023).

The FEMAT items simulate everyday tasks to measure one's use of compensation. The FEMAT encourages individuals to use memory aids to complete the items so the clinician can observe successes and breakdowns and better understand real-world task performance. For example, during Item 4, the assessor provides the patient with a party invitation and asks the patient to show them how they would remember the details of the party (e.g., write the details on the paper calendar on the table or input the details into their calendar application on their smartphone). The quantitative scoring of the FEMAT is unique in that it is based on both task accuracy and successful compensation. Each item is scored using a 4-point scale (score range: 0–3), and the total score is the sum of the values for each of the 11 items (maximum of 33 points; higher scores suggest successful strategy use). There are no normative data to interpret the FEMAT because it is not designed to compare a patient's performance to that of a reference sample. Rather, the FEMAT was designed for the clinician to learn more about the patient's use of compensation to complete tasks in their everyday life and to complement traditional testing to inform the development of a person-centered treatment plan.

Because there is no gold standard for measuring everyday functioning and compensation to support treatment planning (Chaytor & Schmitter-Edgecombe, 2003; Schmitter-Edgecombe & Farias, 2018), the FEMAT was designed to provide a more ecologically valid assessment of everyday skills than traditional structured and decontextualized performance-based measures. We hypothesize that the structure of the FEMAT items would elicit behavioral and verbal responses from patients regarding their everyday functioning and compensation. For example, while completing Item 8 (medication management task), a patient may describe or show how they manage their medications in everyday life, if they use any strategies, and, if so, the types of strategies that they use. The purpose of this study was to capture, code, and categorize FEMAT participant behaviors to (a) provide further evidence for content validity, (b) further refine the scoring and administration prompts of the tool, and (c) develop further interpretation materials to support clinicians. This work will provide further evidence of the content validity of the FEMAT and its refinement to support its use in designing person-centered treatment plans that are customized to the individual's preferences and needs rather than implementing one-size-fits-all treatment plans that do not support long-lasting effects (Lanzi et al., 2018, 2021; Scherer, 2005).

Methods

Study Design

This study is part of a comprehensive research agenda establishing the evidence for the validity and reliability of the FEMAT. We conducted a mixed-methods analysis to explore the verbal and behavioral open-ended responses elicited by participants while completing the FEMAT (Lanzi et al., 2020) to complement the formalized quantitative 4-point item scoring. We adopted an embedded single-case study design to analyze the verbal and behavioral responses provided by 12 participants during the administration of the FEMAT (Lanzi et al., 2020). An embedded design is not purely qualitative; the use of mixed methods and analysis of more than one unit supports stronger conclusions (Creswell et al., 2007; Yin, 2009). Yin (2018) noted that the subunits of analysis in an embedded single-case study design must truly be a part of the original case. In this study, the case was defined as the administration of the FEMAT, while the embedded subunits of analysis were the data sets from each of the individual participants selected. Trends within the whole case, as well as across the embedded subunits, were investigated.

Participants

The present study analyzed data previously collected from Phase 2 of the original FEMAT development and validity study (Lanzi et al., 2020). The original study used the following inclusion/exclusion criteria to recruit community-dwelling older adults who meet the criteria for MCI due to possible Alzheimer's disease: aged 60 years and over, community-dwelling, native English speaker and premorbid English literacy, self-reported difference in cognitive functioning in the previous 6 months to 1 year, self-reported independence for instrumental daily activities, and self-reported negative history of neurological symptoms or conditions that may cause cognitive decline, such as stroke, brain injury, or delirium (Lanzi et al., 2020). If prospective participants met these inclusion/exclusion criteria, they were scheduled to participate in a screening session to determine further eligibility and increase the likelihood that they met the criteria for MCI. To determine the presence of cognitive impairment, prospective participants completed the Montreal Cognitive Assessment (MoCA); a score between 21 and 25 was required to be included in the original study (Nasreddine et al., 2005). The score range of 21–25 was used because of the evidence for sensitivity using a cutoff score of 26 for detecting MCI in a predominantly White non-Hispanic population (Milani et al., 2018; Nasreddine et al., 2005; Tsoi et al., 2015). Prospective participants also needed to obtain a score of less than 6 on the Geriatric Depression Scale (Yesavage & Sheikh, 1986) to rule out major depression and obtain a passing score on the Functional Vision, Hearing, and Communication Screening Measure (Bourgeois et al., 2001) to rule out any major sensory impairments that would alter performance. Prospective participants who met these criteria were consented and enrolled in the original study. The original and present studies were approved by the institutional review board at the University of South Florida.

FEMAT

The FEMAT Version 1.0 English-United States consists of 11 items that ask participants to simulate several everyday tasks (refer to Table 1 for example item prompts). At the start of the administration, the examiner provides the patient with memory aids and states the following prompt to encourage memory aid use, “I'm going to ask you to role-play through several everyday tasks today. Please feel free to use anything on the table or with you to help you complete the tasks (gestures to the items). There are calendars, timers, notepads, and a medication organizer. Throughout each task you can refer to anything you have used. Let's get started.” The FEMAT administration time is approximately 15 min.

Table 1.

Example Functional External Memory Aid Tool item prompts.

Item	Prompt
Item 2	“I am going to play a voicemail for you and ask you some questions about the voicemail immediately after it stops. The voicemail is about an upcoming appointment.”
Item 4	“You just received an invitation to an upcoming party (gesture to the paper invitation). Show me how you would remember the information for the party?”
Item 8	“These pill bottles contain information about when to take the medications (gesture to pill bottles and pill organizer). Show me how you would use this pill organizer to remember to take these medications.”

Data Collection

Adults who met screening criteria and consented to participate completed a single assessment session in person with an assessor. Of the 99 participants who completed the original study, 21 consented to video-recording their session. For the present study, author A.M.L. randomly selected 12 participants (from the 21 recordings) using a random number generator. Of the 12 participants, all were women, 11 were Caucasian, and one was African American. The mean age of the participants was 82 years (Mdn = 82 years, range: 73–90 years). The average years of institutional education for the participants was 13 (range: 11–15). Their mean MoCA score was 22 (Mdn = 22, range: 21–25). Their mean FEMAT score was 12 (Mdn = 11, range: 2–29). The demographics of these 12 participants were comparable to the larger sample of 99 participants in the original study (age, Mdn = 80; education, Mdn = 14; MoCA, Mdn = 23; Caucasian, n = 87; female, n = 93; Lanzi et al., 2020).

Data Analysis

Qualitative Analysis

The video recordings of the 12 participants were manually transcribed verbatim by two research assistants. The speaker (assessor or participant) was explicitly labeled throughout the transcript. Nonverbal physical behaviors/actions that were associated with the task (i.e., occurring after the item prompt) were marked on the transcript in parentheses (e.g., writes information on the calendar). All speech was clear and intelligible, and there were no interjections given the structured nature of the FEMAT administration.

To organize and explore the data for the present qualitative component of our mixed-methods analysis, a similar approach to Malterud's (2012) systematic text condensation method was applied. First, author J.L.M. verified the transcripts using the corresponding video recordings and amended any discrepancies. Second, authors J.L.M. and M.T. broadly read through the 12 transcripts to independently draft preliminary themes. Authors J.L.M. and M.T. met to discuss confluent and divergent issues regarding the preliminary themes developed. Third, authors J.L.M. and M.T. together reviewed the transcript line by line, including the text describing participants' physical actions/behaviors, and organized the transcript into meaning units, to represent a single thought/action related to the research question (Malterud, 2012).

Fourth, the researchers coded the data by identifying, organizing, and sorting the meaningful units to reflect the preliminary themes. The coding process was flexible and involved iterative discussion of commonalities and discrepancies within and across the coding groups and labels. Through investigator triangulation, codes were added, removed, and refined in multiple rounds to support the creation of a coding scheme that was appropriate to best summarize the data into themes and subthemes (Patton, 2015). Four overall themes (see Results) emerged from the coding scheme. Subthemes were also identified during the coding process to further categorize the data in each theme (see Results). Once a focused coding scheme was defined, it was then applied to all 12 transcripts. Interrater reliability (agreements / agreements + disagreements) was established through independent coding of the meaningful units into themes and subthemes. Point-by-point agreement across 20% of the transcribed data was 98.2%; any disagreements were addressed via rater consultation (Patton, 2015).

Quantitative Analysis

We qualitatively analyzed the coded data in two phases. First, we counted the total number of response units for each theme and subtheme to calculate the percentages for each theme (i.e., total number of response units for that specific theme / total number of response units) and subtheme (total number of response units for that specific subtheme / total number of response units for the specific theme). This quantitative analysis was used to understand the most common behaviors of participants.

The second phase zeroed in on trends within the embedded subunits of analysis for this case study and therefore used individual participant transcripts to look for possible patterns and variations. Counts and percentages by theme for each participant were calculated to analyze individual response patterns. Together, these two phases of data analysis supported the transition from code definition to meaning generation using mixed methods (Malterud, 2012).

Results

Our thematic analysis of the 12 transcripts categorized participants' verbal and behavioral FEMAT responses into the following four themes: (a) used an external memory aid (EMA), (b) described using a strategy, (c) discussed memory and comprehension, and (d) discussed functional information. Each meaningful unit (including verbal and physical behaviors) fit into one of these themes. Most participants described using a strategy (40%) and discussed their memory and comprehension (33%) at least once while completing FEMAT items. Participants used an EMA to complete the item task (15%) and discussed functional information (12%) less often. Within each theme, several subthemes emerged from participants' responses elicited during their completion of the FEMAT. Counts and percentages of data falling within each theme and subtheme are summarized in Table 2. The total number of response units in the data set was 214.

Table 2.

Distribution of response units by themes and subthemes.

Theme	n	%
Used an external memory aid (EMA)	32	15
Provided EMA, without assistance	21	65.6
Provided EMA, with assistance	6	18.8
Other EMA, without assistance	4	12.5
Other EMA, with assistance	1	3.1
Described using a strategy	85	40
Provided strategy, without assistance	29	34.1
Provided strategy, with assistance	12	14.1
Other strategy, without assistance	34	40.0
Other strategy, with assistance	10	11.8
Discussed memory and comprehension	71	33
Recalled information, without assistance	22	31.0
Recalled information, with assistance	11	15.5
Misunderstood FEMAT task	9	12.7
Self-perceived memory supports	7	9.9
Self-perceived memory barriers	22	31.0
Discussed functional information	26	12
Biographical information, expected time	6	23.1
Health information, expected time	7	26.9
Technology information, expected time	6	23.1
Biographical information, unexpected time	1	3.8
Health information, unexpected time	4	15.4
Technology information, unexpected time	2	7.7

Note. Theme percentages calculated based on the total number of response units in the data set (N = 214); subtheme percentages calculated based on the total number of response units for that theme only. FEMAT = Functional External Memory Aid Tool.

Theme 1: Used an EMA

The first theme, “used an EMA,” was assigned to any response unit where the participant physically used an EMA to complete a FEMAT item task (EMA; see Figure 1). For example, the participant wrote down the engagement party information (Item 4) in a calendar. Further analysis of this theme revealed two subthemes that described the origin of the EMA used (Subtheme 1: EMA provided by the FEMAT assessor; Subtheme 2: other EMAs used). Within each subtheme, the level of assistance provided by the assessor (i.e., with or without) to use the EMA was also coded.

Figure 1.

Theme 1: used an external memory aid (EMA). FEMAT = Functional External Memory Aid Tool.

Used Provided EMAs

The opening prompt of the FEMAT encourages the participant to use EMAs in front of them that the assessor provided (i.e., pen or pencil, calendar, timer, notepad, sticky notes, and medication organizer) or any EMA that they may have with them (e.g., smartphone, small calendar). Our analysis found that of those who used EMAs to complete the FEMAT tasks, most participants (84.4%) used EMAs that were provided by the assessor. Examples of response units coded in this subtheme include Participant 46 (P46) who asked, “Can I write it?” in response to the doctor's appointment voicemail task (Item 2) and P106 who stated, “Um. Just what I wrote down. I used the note notes. I wrote, ‘Tell Amanda call the insurance before five’” to complete the reminder task (Item 11). The least used EMAs (provided by the assessor) were the timer, calendar, and medication organizer. Based on the content of the responses coded, we suspect this was due to sensory, motoric, and/or cognitive difficulties using those EMAs. For example, P44 said, “Oh. Start, minutes ten. Wow, couldn't make it any smaller could you? Ten minutes. Start,” and P46 said, “Don't know how to set this. I have to go all the way up and then back down? Unless I knew how to work this button” when attempting to use the timer to complete the laundry task (Item 1). Participants also described having some difficulty opening the medication bottles and pill organizer. For example, while attempting to open the pill bottles provided for the medication management task (Item 9), P66 said, “Which I can't open these … I also have—have—the pharmacist knows that you do not put child safety caps because I can't get ‘em off!”

Used Other EMAs

Of those participants who used EMAs, some (15.6%) used EMAs other than those provided by the assessor to complete the tasks. For example, P61 used her smartphone to complete two tasks. During the medication management task (Item 8), two participants described using an alternative EMA method summarized as “flipping over the bottle” to represent how they remember to take medications. For example, P44 said, “This one. Take one tablet by mouth. How else you gonna take it? At bedtime. So this would go upside down in the morning ‘cause I don’t take it ‘til night [Flipped pill bottle over].”

Level of assistance. Of those responses that included the physical use of an EMA to complete the task, the data were further coded by the level of assistance provided by the assessor: “without assistance, independently” or “with assistance, containing an assessor prompt/cue or participant question.” Most responses that were coded as using an EMA were further coded as without assistance (78.1%; i.e., without any prompting or cues from the assessor and without asking any questions). Less frequently, responses were further coded as having occurred with assistance from the assessor. Some participants asked the assessor a question before using the EMA. For example, during the medication management task (Item 8), P44 asked, “The way I would do it or I gotta do it in this thing [pointed to pill organizer]?” and the assessor responded, “Do … whatever.” P46 asked, “Do I have to put these in here?” and the assessor nonverbally prompted her to do so (Item 8).

Theme 2: Described Using a Strategy

The second theme, “described using a strategy,” was assigned to any response that the participant described using a memory strategy or EMA (see Figure 2) to complete the task without physically doing so. Although many of the FEMAT items include “show me” in the task instructions, our analysis revealed that participants were more likely to give a verbal description of memory strategies/EMAs that they would use rather than physically using an EMA to complete the task. We applied the same subtheme levels as described in Theme 1 (Subtheme 1: described EMA provided by the FEMAT assessor; Subtheme 2: described other EMAs). The subthemes were further described as with or without assistance provided by the assessor.

Figure 2.

Theme 2: described strategy. FEMAT = Functional External Memory Aid Tool.

Described Provided Strategies/EMAs

Participants described how they would use the provided EMAs (i.e., timers, notepads/notes, calendars, and pill organizers) to complete specific FEMAT tasks (48.2%). Participants shared how they would use the provided EMAs when speaking about similar FEMAT task experiences in their everyday lives; P105 shared, “Appointments are on my calendar. Yeah on my—it's right up on my wall.” During the medication management task (Item 8), participants shared a great deal about strategies that they used to manage their medications, and several described using a pill organizer. P69 pointed to the pill organizer and stated:

Well I have—I have this for two week. My daughter's a nurse comes and does ‘em. Okay, yeah, I would put like two in for Sunday, two for Monday, and the whole week. In fact, I had double things.

P83 referred to the pill organizer and said, “‘Cause I use one of these. Yeah I do—do it once a week.”

Described Other Strategies/EMAs

Participants also described using strategies and EMAs other than those provided by the assessor to complete FEMAT tasks (51.7%). For example, many discussed using logs, spreadsheets, and/or online banking services to assist during the bill management task (Item 7). P46 stated: “I just go on my computer. I put—I have the online banking. And I go as soon as I get my bill, I go on to the bill thing and I put in, I have a whole list of 'em.” Some participants shared that they would keep the item (i.e., piece of mail, bill, voicemail, e-mail) to serve as a reminder. For example, P86 said, “I keep getting out—paper out to check to see, oh, what time it is.” P106 said, “I would keep the email in my list. Oh, that's right. I'm supposed to pick up the medicine today, what time, you know, and and go back, go back and check it.”

A few described the use of an internal memory strategy to complete the task: making relevant personal connections to the content that they were attempting to remember. For example, while holding the party invitation (Item 4), P66 said:

Uh, how would I remember? Well, I was gonna look at the picture, but that picture doesn't actually tell me that's an engagement party. And uh, uh, the Arnolds. I actually knew people named Arnolds, so I'd have to like put that with them, the Arnolds. And the small gallery, I would put a, like in my brain um, a uh, place that had you know pictures and that. Winter Road, I would have to think about uh, the weather. And Pittsburgh PA, I live there.

P81 said about the party invitation, “Ah, February I'd remember ‘cause it’s my birthday. Twenty-second is because it's my son-in-law's birthday and Friday is the end of the week. And six thirty is past my dinner time.”

Level of assistance. A second level of subthemes defined the level of assistance provided by the examinee to support the participant in describing the strategy or EMA. Most of the response units within this theme were coded as described EMA use without assistance (74.1%)—no prompts, cues, or questions from the assessor. For the remaining 25.9% of response units, the assessor provided some level of assistance (e.g., further prompt, cue, or question). For example, the assessor asked P44 why she did not write down the information during the doctor's appointment form task (Item 3), and the participant said:

No, you know why I wouldn't? Because see this calendar. When I'm at my table if I know it's the doctor's office, I go in and I write it right on the calendar. Oh and the calendar was here, I didn't realize I was supposed to write on it. That's what I do, I write.

In another example, P48 stated, “I have a timer” for the laundry task (Item 1). The assessor then said, “Okay, so what would you do?” The participant continued, “I'd set the timer and, uh, stick around you know, I mean don't go far.”

Theme 3: Discussed Memory and Comprehension

The third theme, “discussed memory and comprehension,” was used for any response unit in which the participant discussed what they did and did not remember during FEMAT tasks; misunderstandings of FEMAT tasks; or information about their memory regarding challenges, strengths, supports, and barriers (see Figure 3). The subthemes that emerged were Subtheme 1, “recalled or misunderstood FEMAT information/attempted task,” and Subtheme 2, “discussed (self-perceived) memory supports and barriers.” Subtheme 1 data were further coded as “recalled” (with or without assistance) or “misunderstood” FEMAT information. Subtheme 2 data were further coded as self-perceived memory “supports/strengths” or “barriers/challenges.”

Figure 3.

Theme 3: discussed memory and comprehension. FEMAT = Functional External Memory Aid Tool.

Recalled or Misunderstood FEMAT Information

A portion of the data within the memory and comprehension theme included content related to a specific task on the FEMAT. Data within this subtheme were further coded as describing content related to recalling information for a task on the FEMAT (Subthemes 1.1 and 1.2; 46.5%) or misunderstanding or miscomprehending the task or being unsure of how to do something (Subtheme 1.3; 12.5%). Much of the data coded within Subthemes 1.1 and 1.2 was provided during the retrieval-based items of the FEMAT.

Recalled FEMAT information without assistance. Most of the response units coded under recalled information/attempted task were further coded as recalled information without assistance (66.6%). Like the first two themes, assistance included an assessor prompt/cue or participant question. For this research study, the use of an EMA did not qualify as assistance, as the FEMAT encouraged the use of these strategies to support memory and retrieval.

Recalled FEMAT information with assistance. At times, assessor assistance was provided to recall information. For example, during the store request form (Item 6), P83 said, “We need to buy a swimsuit.” The assessor replied, “Hold on, read the question.” This prompted the participant to say, “Oh, what size do you need to buy.” P48 required further support to complete the doctor's appointment form (Item 3). After the participant shook her head to communicate that she did not remember the type of appointment, date, or time for the doctor's appointment from the previously played voicemail, the assessor asked, “And what you had to bring?” and the participant responded, “Medicine.”

Misunderstood the FEMAT task. Within this theme, participants sometimes described a misunderstanding of the task itself (12.7%). For tasks that included text material that was given to the examinee (i.e., Item 4's party invitation, Item 7's utility bills, and Item 10's printed e-mail), some simply read the text aloud rather than showing or describing how they would remember that information.

Discussed Memory Supports and Barriers

The second subtheme under MC was used to code response units that included a participant discussing self-perceived supports and barriers that affected their memory. This included supports or barriers that they experienced while completing the FEMAT or in their everyday lives; 24.1% of the data within this subtheme described memory supports/strength, while 75.9% described barriers. For this data analysis, the participant's perceptions of their retained memory or cognition skills were coded as memory supports, while their perceptions of their own memory and comprehension difficulties were coded as memory barriers.

Self-perceived memory supports or strengths. Some recognized support for their memory performance and perceived strengths during the FEMAT administration. Participants verbalized that they were able to remember without any strategies. For example, P69 stated, “And um ‘cause I’m pretty good with that,” when stating that she remembered to pay her bills (Item 7) without a strategy. P106 said, “Um I might write it out. That is the kind of thing I'll usually remember though.” P44 mused, “I always remember to do it, I don't know why. I know I gotta pick up like pick up the pills at one.”

Self-perceived memory barriers or challenges. More commonly, participants described barriers to their memories and perceived weaknesses. Participants often expressed frustration with their memory; for example, P44 exclaimed, “Oh I said the date, now I forgot it. I'll remember this later on. I'd said where you were going first too. Oh I said it right away! Oh my God!” P86 said, “When do you need it by? I don't know, next week? Sometime next week. Don't trust me to remember when I get to the store though!”

Some detailed their perceptions of their difficulties. P48 said, “I always have to ask people to repeat things that's—that's my—that's my real problem.” After talking about writing down important notes, P105 added, “Yeah, ‘cause I can’t—I can't leave it to memory.” P69 wondered why she was experiencing difficulty recalling information. She said, “I don't remember any of that on there. I remember I had a doctor's appointment and—Maybe it was because I knew it wasn't pertaining.”

Theme 4: Discussed Functional Information

The final theme, “discussed functional information,” was assigned to response units that did not directly relate to FEMAT tasks, compensation, memory, or comprehension (see Figure 4). These units were categorized into two levels of subthemes. The first subtheme level described whether the functional information was provided during an expected or unexpected time during FEMAT administration. The second subtheme level categorized the units by topic: biographical, health, or technological information.

Figure 4.

Theme 4: discussed functional information. FEMAT = Functional External Memory Aid Tool.

Timing: Expected and Unexpected Times

Most of the data in the “discussed functional information” theme were shared at an expected time during FEMAT administration (73.1%). An expected time was defined as in between task items or in response to further questioning from the assessor. For example, after the assessor asked, “How long have you had the iPhone for?” P61 replied, “Oh I've had it for a while. Mhmm. It's smarter than I am, it is! I—I can't stand it sometimes.” P66 noted once she saw the assessor pull out a mobile device, “Now you do know that I do not do any of the new stuff. No computer, no email.” These responses were provided at expected times because they did not interrupt a task.

Some functional information (26.9%) was discussed at an unexpected time in FEMAT administration. An unexpected time was defined as the middle of a task item or providing excessive or off-topic explanation following a question from the assessor. It was noted that 42.9% of the functional information provided at an unexpected time originated from one participant, P66. For example, during the party invitation task, she responded to an assessor's question, “Do you use your iPhone often?” with a story about wanting to remove advertisements from her mobile device and speaking about this with her wireless provider. This did not fit the FEMAT task and went beyond the topic of the assessor's question.

Functional Information

In the second subtheme level under functional information, data were coded by topic: biographical, health, or technology information. Of the three topics, health information was the most common.

Biographical information. Biographical information was characterized as anything personal about the participant that was not related to health or technology. For example, P48 shared about her reading habits: “I don't—I don't use much as I used to with books. I read more magazines.” P81 spoke of preparing meals: “I cook for myself, not saying every night. I eat leftovers!” Others shared information about their families and past experiences.

Health information. Any units that shared information about the participant's health history, conditions, medications, doctor's visits, health insurance, and so forth were categorized as health information. This information was often shared during the three medical tasks (doctor's appointment voicemail, doctor's appointment form, and medication management), as well as one of the instrumental daily tasks (prescription e-mail notice). For example, P46 said, “I have to take thyroid medicine. I had my thyroid removed. So, oh yeah, I had uh a tumor.” P86 spoke of discontinuing all her medications: “I probably stopped that uh, may three or four years ago. And just a year ago, like I said, when my husband got sick I just stopped them all and I just threw them all out because they were all getting expired.”

Technological information. Technological information provided details about the individual's personal experiences with technology. For this data set, most of the technological information data were related to mobile phones. For example, P81 said, “Well, that's it, I don't—don't have a cellphone. I don't have a computer. So I have none of these new gadgets.” In response to the assessor asking, “How did you learn to use [the iPhone]?,” P61 said, “Hit and miss. It's—this is my only phone. Well it was a newer one and uh my daughter got me this so that I can carry it wherever I go in case I fall. I can call 911.”

Individual Trends

Data about individual participant response patterns were compared (see Table 3), and similarities and differences were noted. It was found that 11 of 12 participants described strategies more often than used EMAs (the 12th participant had equal amounts of responses for each theme). Two individuals did not utilize any EMAs during the FEMAT; they both described memory strategies in several response units. All participants provided multiple response units related to MC, and three participants had 50% or more of their total responses categorized under this theme. Three individuals had over 20% off-topic responses, but only two of these people expressed any portion of their off-topic responses at an unexpected time. Three others provided no off-topic responses at all.

Table 3.

Distribution of response units by participant.

Participant	Theme								Total
	Theme 1		Theme 2		Theme 3		Theme 4
	n	%	n	%	n	%	n	%
P44	3	13.6	8	36.4	11	50.0	0	0.0	22
P46	6	35.3	8	47.0	1	5.9	2	11.8	17
P48	1	4.5	7	31.8	13	59.1	1	4.5	22
P61	4	22.2	4	22.2	4	22.2	6	33.3	18
P66	2	10.5	9	47.4	4	21.1	4	21.1	19
P69	0	0.0	5	45.4	4	36.4	2	18.2	11
P81	1	5.6	8	44.4	6	33.3	3	16.7	18
P83	5	26.3	6	31.6	3	15.8	5	26.3	19
P86	0	0.0	7	38.9	9	50.0	2	11.1	18
P91	3	21.4	7	50.0	4	28.6	0	0.0	14
P105	1	6.3	8	50.0	6	37.5	1	6.3	16
P106	6	30.0	8	40.0	6	30.0	0	0.0	20

Note. Theme 1: Used an external memory aid; Theme 2: Described using a strategy; Theme 3: Discussed memory and comprehension; Theme 4: Discussed functional information.

Discussion

The present study aimed to operationalize the verbal and behavioral responses provided by participants during the FEMAT assessment. Unlike many formal assessment tools used by SLPs, the FEMAT was not designed to diagnose or quantify cognitive strengths and weaknesses using norm-referenced data. Rather, the FEMAT was designed to provide clinicians with information about the patient's everyday function and use of compensation in a standardized format to inform treatment planning. Previous literature, primarily in the fields of neuropsychology and occupational therapy, has emphasized the importance of measuring everyday functioning, but no gold standard measurement exists due to the difficulties in measuring very early and subtle functional difficulties (Chaytor & Schmitter-Edgecombe, 2003; Schmitter-Edgecombe & Farias, 2018). Thus, many researchers have begun to code behaviors for detailed information to understand better and observe how individuals are completing complex activities of daily living (Bailey et al., 2013; Schmitter-Edgecombe & Parsey, 2014). Operationalizing and analyzing subtle behaviors during open-ended tasks may help better understand functional ability beyond what is easily measured in traditional questionnaires or performance-based measures (Griffith et al., 2003; Schmitter-Edgecombe & Farias, 2018; Schmitter-Edgecombe et al., 2012; Schwartz et al., 2002; Wadley et al., 2009). Our analysis suggests that, beyond a quantitative score, the FEMAT elicits behavioral and verbal content from participants that may be useful for person-centered compensatory treatment planning. The results of the present study contribute to the evidence for the validity and refinement of the FEMAT in several important ways.

First, the findings from this study provide further evidence of the content validity of the FEMAT—a measure of everyday strategy use, defined as “one's ability to use external aids to compensate for cognitive weaknesses when completing everyday tasks” (Lanzi et al., 2020, 2023) Not surprisingly, two themes that emerged from the present study were “discussed using a strategy” and “used an EMA” to categorize participants' verbal and behavioral responses. There are limited evidence-based assessment tools available for clinicians to use to evaluate a patient's everyday skills and compensation, and findings from the present study further support the use of the FEMAT to fill this important gap in clinical practice. To reduce barriers to clinical implementation, the FEMAT record forms and materials are downloadable at no cost on the FEMAT website (Lanzi, n.d.) but do not include any physical EMAs. EMAs are to be provided by the assessor to complete the functional tasks. It is important to note, however, that our findings revealed some participants expressed sensory, motor, and/or cognitive difficulties using the provided EMAs (e.g., unable to open the medication organizer). Therefore, the assessor must consider sensory, motor, and/or cognitive abilities when building their EMA kit.

Second, the present study found that participants often described using a strategy rather than physically using an EMA to complete the tasks. Thus, it is important to interpret FEMAT data beyond simply looking at the quantitative score obtained. For example, a participant may only receive a score of 1 out of 3 for describing their use of an EMA during the task, but their described use may provide valuable information for treatment planning. Our findings suggest that participants often described what they do in their everyday lives to complete their daily activities (e.g., P105 shared, “Appointments are on my calendar. Yeah on my—it's right up on my wall”). SLPs typically conduct informal interviews with patients to collect this type of functional data (Lanzi et al., 2022), which may be challenging due to the unstructured nature of interviews, the length of time needed to complete them, and the lack of consistency across patients and clinicians in collecting information in an informal interview. Our findings suggest that the FEMAT may effectively supplement this approach and/or improve the efficiency of the assessment process by eliciting valuable information about the person's functional skills in an efficient, standardized, and structured manner. It is important to note that although there is value in listening to a participant's described strategy use, their ability to use the EMA successfully remains unknown. Therefore, we have modified the FEMAT to provide the assessor with specific prompts following the assessment for the clinician to observe the patient using the EMAs they described.

Third, across participants, no clear trends emerged in the types of EMAs that the participants physically used or described using to complete the tasks. For example, to complete Item 4, the party invitation task, some participants described using a paper calendar, some described using their smartphone, and some described keeping the invitation in an important place. The individual differences were especially evident during Item 8, the medication management task. These findings support previous research highlighting the need to consider individual preferences for compensation and previous and current EMA use before designing patient-specific strategies (Lanzi et al., 2018, 2021; Scherer, 2005; Sohlberg et al., 2022; Sohlberg & Mateer, 1989, 2001). Patient-reported outcome measures and needs assessments can be used to examine the frequency of compensation use (Sohlberg & Turkstra, 2011). For example, the Everyday Compensation tool (Farias et al., 2020) is a questionnaire that examines the frequency with which older adults use various strategies to complete everyday tasks (e.g., “How often do you write appointments on your calendar to manage appointments?”—never, rarely, sometimes, frequently, always). The FEMAT can be used alongside such questionnaires that measure frequency to gain further insight into how patients complete tasks in their everyday lives (e.g., What type of EMA do they use, and how efficient are they in using it?).

Fourth, a surprising finding from the present study was that the FEMAT elicited discussion from many participants about their self-perceived memory and comprehension abilities. Memory self-efficacy is an important variable to consider for intervention with older adults with MCI (Beaudoin & Desrichard, 2011; Dux et al., 2008; Greenaway et al., 2008, 2013; Langer et al., 2019; Rebok & Balcerak, 1989). Research suggests that improving self-efficacy may reduce the impact of symptoms of depression on one's cognition, and self-efficacy may predict self-rated “capacity to live well” (O'Shea et al., 2016). Adults with MCI and their caregivers also ranked memory self-efficacy among the top important treatment outcomes (Barrios et al., 2016; Smith et al., 2018). Interestingly, a recent study reported that only four of 442 active clinical trials for MCI included self-efficacy as an outcome, which may be due, at least in part, to the limited psychometrically sound measures of memory self-efficacy for adults with MCI (Kurasz et al., 2021; Smith et al., 2018). Further studies are needed to examine the type of self-efficacy information that participants share during the FEMAT, but this preliminary work suggests that the FEMAT may help the clinician gain some insight into the patient's memory self-efficacy. We encourage clinicians to further evaluate a patient's memory self-efficacy using patient-reported outcome measures, such as the Self-Efficacy for Managing MCI scale (Kurasz et al., 2021).

Fifth, the FEMAT elicited functional information from participants regarding their personal life, health, or technology. Although this information can be considered off-topic to the FEMAT tasks, clinicians are encouraged to document the information shared as it may provide important insight into a patient's social activities, participation, and environmental factors, which is necessary for treatment planning. For example, the clinician may learn that the patient has grandchildren or that they are very uncomfortable using technology. Personal information shared during the FEMAT may complement the patient's case history or be a foundation for topics to discuss during the clinical interview.

Lastly, the FEMAT has been revised based on the findings from the present study (FEMAT Version 2.0; Lanzi et al., 2022). We have developed guidelines for the assessor to provide prompts to the participants that are outlined in the “Repetition of Items and Probing of Responses” section of the Lanzi et al. (2022) FEMAT manual. FEMAT Version 2.0 now includes five screening items to examine sensory and motoric abilities before administering the 11-item assessment. We have also developed videos and handouts to help clinicians interpret FEMAT performance for treatment planning. For more information about FEMAT Version 2.0, we direct readers to the manual (Lanzi et al., 2022) and FEMAT website (Lanzi, n.d.).

Limitations

The limitations of the current study are similar to those discussed in the companion study (Lanzi et al., 2020). The demographics of participants were predominantly non-Hispanic White women. A previous study found that the prevalence of MCI was significantly higher in their Hispanic participants, while the prevalence of dementia was higher in their Black participants (Manly et al., 2022). This lack of representativeness in the sample should be considered when interpreting findings. It may be that older adults of different racial/ethnic backgrounds and/or different gender identities implement EMAs differently or with a different frequency.

Future research should also consider collecting more comprehensive cognitive test data instead of using a screening tool. Although the MoCA has strong evidence for detecting mild impairments, the cutoff score used should consider education, age, race, and ethnicity (Carson et al., 2018; Milani et al., 2018). More comprehensive cognitive test data would support further participant classification by diagnosis and subtype to enhance the generalizability of findings. There may also be a relationship between individual cognitive profiles and FEMAT responses.

In addition, it is not known how well the self-reported/demonstrated strategies on the FEMAT translate into actual use of strategies in naturalistic environments. Future research could address this through observations of participants in real-life situations requiring the use of strategies and/or through semistructured interviews with older adults and their care partners. Use of the FEMAT along with standardized, decontextualized measures as a way to demonstrate improvement posttreatment and/or to demonstrate maintenance of independent living skills over time could also be informative.

Conclusions

The FEMAT was developed by Lanzi et al. (2020) to support clinicians in their design of person-centered compensatory interventions for adults with MCI. This tool was the first validated measure to score individuals' use of EMAs to complete simulated everyday tasks. Our findings suggest that the FEMAT provides valuable information about the person's strategy use and experiences with memory in their everyday lives. These data, when taken together, can provide the foundation for an effective person-centered compensatory treatment and may better support individuals with MCI in maintaining their independence.

Author Contributions

Alyssa M. Lanzi: Conceptualization, Data curation, Investigation, Methodology, Project administration, Data curation, Validation, Visualization, Writing – original draft, Writing – review & editing. Julia L. Mendez: Conceptualization, Investigation, Methodology, Project administration, Data curation, Validation, Visualization, Writing – review & editing. Megan Tobin: Conceptualization, Investigation, Validation, Supervision, Visualization, Writing – Review & editing. Melissa A. Johnson: Conceptualization, Investigation, Validation, Supervision, Visualization, Writing – review & editing.

Data Availability Statement

The transcript data analyzed during the current study are available from the corresponding author on reasonable request. The Functional External Memory Aid Tool (FEMAT) is available for access on the FEMAT website (A. M. Lanzi, n.d.).

Funding Statement

This work was supported by the National Institute of Aging of the National Institutes of Health under Grant K23AG070185-01 awarded to Alyssa M. Lanzi. This work was part of Julia Mendez's master's thesis at Nazareth College, and further analysis was supported by the National Institute of Aging of the National Institutes of Health under Grant K23AG070185-01 awarded to Alyssa M. Lanzi.