Abstract
We describe a smartphone/smartwatch system to evaluate anomia in individuals with aphasia by using audio-recording-based ecological momentary assessments. The system delivers object-naming assessments to a participant’s smartwatch, whereby a prompt signals the availability of images of these objects on the watch screen. Participants attempt to speak the names of the images that appear on the watch display out loud and into the watch as they go about their lives. We conducted a three-week feasibility study with six participants with mild to moderate aphasia. Participants were assigned to either a nine-item (four prompts per day with nine images) or single-item (36 prompts per day with one image each) ecological momentary assessment protocol. Compliance in recording an audio response to a prompt was approximately 80% for both protocols. Qualitative analysis of the participants’ interviews suggests that the participants felt capable of completing the protocol, but opinions about using a smartwatch were mixed. We review participant feedback and highlight the importance of considering a population’s specific cognitive or motor impairments when designing technology and training protocols.
Keywords: ecological momentary assessment, experience sampling, microinteraction, aphasia, anomia, smartwatch, health research, stroke, Human-centered computing, Empirical studies in ubiquitous and mobile computing, Sound-based input/output
1. INTRODUCTION
Aphasia is an acquired communication disorder that occurs in approximately one-third of stroke survivors [1–3]. People with aphasia can struggle to understand and produce language in either spoken or written forms. While aphasia is a heterogeneous disorder, anomia (i.e., impaired word retrieval) is ubiquitous across aphasia subtypes [4]. Anomia can be conceptualized as a more severe version of the "tip-of-the-tongue" state experienced by neurologically healthy adults when they have the concept of what they want to say but cannot retrieve the specific word. In people with aphasia, anomia severity can range from mild, such as delayed retrieval of low-frequency words (e.g., abacus), to severe, often manifesting as difficulty with retrieving the names of everyday, familiar objects (e.g., bed). Even mild anomia can disrupt conversations and lead people with aphasia to disengage from social situations, which can negatively affect their quality of life [5, 6].
2. BACKGROUND
In clinical practice, speech-language pathologists (SLPs) typically assess word retrieval through a single administration of a picture naming test (e.g., the Boston Naming Test [7]). Such tests are usually performed in a quiet clinic room devoid of distractions as the clinician asks the patient to name a series of pictures presented one at a time. Although considered the current standard of care, this traditional assessment format likely does not capture the extent of anomia people with aphasia experience during everyday communication. First, word retrieval in the real world typically occurs when an individual performs other daily tasks, often in distracting, noisy environments, that increase cognitive load. Second, post-stroke aphasia is considered a disorder of access, meaning that conceptual and linguistic representations remain intact, but individuals’ ability to access those representations is variable [8, 9]. Thus, with anomia, access impairments manifest as inconsistent retrieval across attempts for a given word, which necessitates assessing the same target multiple times to form a complete picture of an individual’s anomia. However, each item is only assessed once in the traditional SLP assessment format. Ideally, anomia assessment would occur repeatedly in various contexts experienced over days or weeks.
2.1. Ecological momentary assessment
Ecological momentary assessment (EMA) [10], also known as the experience sampling method [11], is a research methodology used to measure in-situ experiences via momentary self-report. The use of EMA with neurologically healthy populations has increased because the ubiquity of smartphones has made the method practical to deploy affordably with most adult populations [12]. EMA protocols commonly use smartphones to prompt with audio and vibration, typically several times per day; each time, a set of questions (usually multiple-choice) is presented that may take between a few seconds and a few minutes to answer. The sampling frequency in EMA varies widely based on study goals, and the annoyance from prompting and the time and mental effort required to answer questions introduces participant burden [13, 14]. Microinteraction ecological momentary assessment (μEMA) is a method shown to permit self-report at rates of up to four times per hour [15]. Microinteractions are 3–4 s actions; the goal of μEMA is to keep all prompts for information to microinteractions that can be answered ’at a glance.’ This simplicity is achieved by (1) running μEMA on an always-accessible device such as a smartwatch and (2) ensuring each prompt is always for a single, cognitively simple question. When these goals are achieved, μEMA has been shown in neurologically healthy persons to have high compliance at high temporal density [16, 17], even for deployment periods as long as a year [18].
Audio-based EMA leverages speech to collect verbal self-report from participants. It may retain most of the benefits of text-based, multiple-choice EMA while also being suitable for open-ended questions where the participants can respond with non-determined answers. One group of researchers developed a mobile device that samples 30 s of audio at 12-minute intervals throughout the day, which can be used to identify and code daily behaviors and interactions via audio [19, 20]. In another study, a group of older adults completed a one-week, audio-based self-reporting of their current activity using a smartwatch [21]. Overall, frequent, repeated sampling of audio for research remains relatively unexplored. A few researchers have examined different EMA paradigms and audio-based EMA, specifically among people with aphasia. In one case study, Likert-scale EMAs were administered to a single participant with aphasia four times per day for six weeks; with clinician reminders, the participant reached 100% compliance [22]. In another study, participants’ overall talk time duration with and without non-fluent aphasia was compared across 14 days, with six total hours of data per day [23]. Other researchers found that a head-worn computer with a custom interface facilitated faster access to vocabulary words than traditional word retrieval prompting for 20 participants with aphasia [24]. In this work, we explored the feasibility of audio-based EMA, including a single-question version that would allow μEMA-like interactions.
3. STUDY DESIGN
Participants were recruited through The Aphasia Network Lab at Northeastern University. Inclusion criteria were (1) a history of left hemisphere stroke at least six months prior to study enrollment that resulted in aphasia, (2) premorbid proficiency in English, and (3) normal or corrected-to-normal visual acuity. Two participants exhibited hemiparesis of the right upper extremity; they wore the smartwatch on their hemiparetic wrist and navigated the watch with their left hand. Participants included five men and one woman with a mean age of 61.3 years. Aphasia severity ranged from mild to moderate according to the Quick Aphasia Battery (QAB) [25]. Study protocols were approved by the Institutional Review Board at Northeastern University. All participants provided their written informed consent to study procedures.
3.1. Study protocol
Upon study enrollment, participants completed two two-hour sessions before starting EMA. During the first session, a research assistant administered neuropsychological assessments to ascertain each participant’s overall aphasia severity, naming abilities, and other cognitive skills. This battery also included a brief survey regarding participants’ comfort with technology, which was adapted from Sitren & Vililla-Rohter [26] and included a 7-inch visual analog scale that participants marked to indicate their comfort with smartwatch usage; lower scores (closer to a ‘sad’ face) indicated less comfort and higher scores (closer to a ‘happy’ face) indicated more comfort with smartwatch technology. Participants were also administered an object naming test of the 260 items comprising the Snodgrass and Vanderwart word list [27]. Images of these items were pulled from the Bank of Standardized Stimuli (BOSS) set [28] and are unique from the images used in the QAB. During this assessment, participants were asked to name each item aloud while audio was recorded, and the responses were subsequently scored as correct or incorrect. Based on the results of this assessment, 108 unique target items were selected for each participant for use during smartwatch-based EMA. When possible, half of the images were items named correctly during this evaluation, and the other half were incorrectly named items. Participants were pseudo-randomly assigned to the single-image EMA—approximating a μEMA interaction—or the nine-image EMA condition based on overall aphasia severity.
During the second session, researchers who had experience working with people with aphasia trained users on the devices and protocol. Verbal instructions were supplemented by a written manual with visuals and plain, aphasia-friendly language and demonstrations by the researcher. Training activities included four components: (1) smartwatch and smartphone orientation, (2) EMA protocol training, (3) safety instructions, and (4) EMA protocol simulation. During smartwatch and smartphone orientation, participants were familiarized with the devices, shown how to charge each device, and provided instructions on connecting the supplied phone to their home WiFi network. During protocol training, participants were instructed on the specific single-or nine-image EMA task, including how to tap on the screen and record a verbal response while holding the watch close to their mouth. Participants were instructed on when to ignore prompts (e.g., while driving) and when to remove the watch (e.g., when showering) to ensure their safety. At the end of the session, participants completed an EMA practice task that mirrored the EMAs they would complete during the three-week protocol. During this practice, participants donned the watch and attempted to name 18 items, either randomly interspersed with one trial per prompt (for those in the single-image condition) or split into two sets of nine trials per prompt (for those in the nine-image condition). The researcher provided cueing and feedback as needed so that the participant successfully learned how to operate the smartwatch and respond to prompts.
At the end of the orientation, the research team initiated the 21-day protocol to begin the following day. Participants were then prompted to name 108 unique images, seven times each over the subsequent three weeks. At the end of each week of EMA, a researcher scheduled a check-in with the participant over Zoom, during which participants completed a 16-item Qualtrics survey about their EMA experiences during the prior week and discussed any issues they experienced. Following the three-week EMA protocol, participants completed one final session with a researcher, during which they were again administered the Snodgrass and Vanderwart naming test and completed an exit interview. Throughout the study, staff trained in communicating with people with aphasia provided appropriate assistance with understanding surveys and other materials. Participants were compensated with a $100 Amazon gift card for completing the study.
3.2. Prompt technical design
We designed a mobile system to administer, capture, and analyze EMA recordings throughout the study period. A Fossil Gen 6 smartwatch was the primary tool used by participants; it was used only for the study. An app on the smartwatch delivered EMA prompts to the participants and recorded their responses. Participants also received a Motorola G Power smartphone and were instructed to connect it to WiFi once they returned home. A phone app served as a companion to our smartwatch app, permitting remote configuration of image files on the smartwatch and uploading of log and data files while the participants were at home.
3.3. Prompting protocol
Participants completed a nine-item EMA protocol consisting of four prompts per day with nine images per prompt or a single-item EMA protocol where they received a single image 36 times per day (3–4 prompts per hour). All prompts were delivered between 10 am and 8 pm. Nine-item prompts were separated by at least 150 minutes, while single-item prompts were separated by at least 10 minutes. To mitigate practice effects, the individual pictures (in the single-item condition) or picture sets (in the nine-item condition) were presented randomly without replacement until all 108 pictures were used before the picture cycle restarted. To reduce the technological burden of the equipment for participants, we instructed them to leave the smartphones plugged in at home during the study. We asked participants to wear their watches during the prompting window each day (10 hours) and to place the watch on the charger at all other times.
Prompts, along with a sound and vibration, were delivered to the watch as a full-screen notification (Fig. 1). The sound was a 4 s chime; the vibration pattern consisted of three one-second vibrations followed by eight 300 ms vibrations. On this initial screen, participants could tap ‘YES’ to proceed to the naming task or ‘NO’ to postpone the prompt for 5 min. The smartwatch screen would automatically turn of 30 s after the prompt if neither button was pressed. When participants received a reprompt after tapping ‘NO,’ tapping ‘NO’ a second time led to skipping the naming task (i.e., the trial) with no additional reprompts.
Figure 1:

Example of one EMA naming trial. Far left image shows the prompt notification screen. Center images show examples of images from the BOSS data set that a participant might see when completing naming trials. Far right image is the screen participants saw when they completed a naming trial.
When the naming task began, the smartwatch display changed to a full-screen, color picture of an object to name (Fig. 1) and the watch began recording audio. In the single-item condition, the recording stopped once the thank you screen (Fig. 1) was reached or after 10 s had passed. In the nine-item condition, participants had up to 10 s to name each image, for a cumulative total of 90 s. If participants named an image in less than 10 s, they could tap the screen to proceed (either to the ‘thank you’ screen or the subsequent image, depending on the condition). These images were shown in a participant-specific order, as described previously.
4. RESULTS
Six participants each completed 21 days of continuous data collection. Three completed the single-image EMA protocol, and three completed the nine-image EMA protocol.
4.1. Compliance and completion
Compliance and completion rates for all participants served as our primary measures of response behavior. Compliance is defined as the number of all trials (the sum of single-image naming trials or sets of nine-image naming trials depending on condition) responded to divided by the number of trials that were expected to be delivered, expressed as a percentage. The completion rate is defined as the number of trials responded to divided by the trials fully delivered to the participant, also expressed as a percentage. Completion could be higher than compliance if some prompts were never delivered to a participant due to issues such as the watch battery expiring during the prompting hours.
If users initiated a response by tapping ‘YES’ on the initial trial screen, we counted that as a successful trial response attempt. A trial was counted as delivered if the log files indicated that the initial prompt screen was rendered. Prompt screen rendering was tracked during reprompts but was not counted as a separate prompt delivery in our calculations.
For participants in the single-image condition, there were 756 total scheduled trials (36 prompts per day for 21 days, not considering reprompts). For participants in the nine-image condition, there were also 756 scheduled trials (organized into 84 sets of nine trials each and four prompts per day for 21 days, not considering reprompts). When calculating compliance and completion in the nine-image condition, each set of nine images was treated as one trial set because participants only indicated their willingness to complete a trial with a set of images, not each individual image. For the first nine-image EMA participant, [P02], we could not calculate completion due to a problem with log file transmission, but we could calculate their compliance based on timestamped audio files.
Among participants in the nine-image EMA condition, the average compliance rate was 80.1%, and the average compliance among single-image EMA participants was 80.0%. The average completion rate among nine-image EMA participants was 87.9%, and among the single-image EMA participants the average completion rate was 85.6%.
Among the five out of six participants for whom logs were available, they postponed (i.e., tapped ‘NO’ on the initial prompt) 1.5% of the time, and 52.9% of the subsequent reprompts resulted in initiated trials. Prompts timed out after the 30 s countdown 11.6% of the time due to participant inactivity.
4.2. Audio quality
To assess the quality of the audio samples, we randomly selected two days for each participant and analyzed all available audio files recorded on those days for a total of 193 recordings. For the nine-item condition, each participant’s nine answers in each trial are treated as individual samples. A research team member listened to each recording and labeled intelligibility (intelligible, unintelligible, and word naming missing) and type (if any) of background noise (sporadic, persistent, and no background noise). Sporadic background noise refers to intermittent noises from background conversations or music in the sampled dataset. Persistent background noise was present throughout the recording (e.g., traffic noise, fans, or a television). The noise sources in the recordings were conversations (0.04%; 8/193 clips), wind (0.01%; 3/193 clips), television (13.4%; 26/193 clips), and ringtones (47.2%; 91/193 clips). The background noise labels were not mutually exclusive since an audio clip could contain both sporadic and persistent background noises (e.g., persistent television noise and someone calling out to a participant). Each naming attempt was labeled intelligible if the research team member could understand the sounds within the spoken utterance, even if the response was a real-word or nonword paraphasia of the target word (e.g., ‘celery’ for asparagus; /tewəl/for heart).
Out of 377 responses, 258 responses were intelligible (68.4%), 66 responses were missing (17.5%), and 44 were unintelligible (11.6%) (See Table 2). Background noise impacted the intelligibility of the spoken words in only five instances (0.02%; 5/193). Ongoing vibrations were canceled when a user tapped ‘YES,’ but the chime prompt was not always canceled upon a tap due to a software error. Therefore, a partial chime was sometimes captured in the audio clip. However, this did not affect our understanding of the participants’ answers. One of the participants in the single-image EMA group [P04] with auditory comprehension impairments misunderstood the instructions and believed that they were not supposed to name the images aloud when alone; this was not discovered until a check-in a few days before the end of the study. This misunderstanding resulted in all the participant’s audio in the sample being marked as missing (14%; 54/377). In eight instances (0.02%; 8/377), the participants seemed to have accidentally dismissed the prompt before they finished their answer, affecting their understandability. In some instances, due to lag in the smartwatch operating system, the user interface would not immediately update upon a tap, and users would tap multiple times; these repeated taps could cause images to be dismissed too quickly.
Table 2:
Results of audio quality labeling for the single- and nine-image EMA conditions
| Intelligibility | Background Noises | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| Intelligible | Unintelligible | Missing | Sporadic | Persistent | None | |
| μEMA | 91 | 20 | 59 (5 **) | 88 | 48 | 54 |
| a EMA | 167 | 24 | 7 | 3 | 3 | 17 |
For the EMA condition, each response is rated separately in terms of intelligibility, but the entire clip is used to rate background noises
Removing the 54 samples missed due to P04, as described in the text below
4.3. Perceived burden
During the weekly check-in Qualtrics survey, participants were surveyed about their experiences with EMA the prior week. They were asked nine positive valence questions about the ease of smartwatch usage, their ability to follow the protocol, and the visual design of EMA features. The final three negative valence questions primarily reflected participants’ perceived burden of completing EMAs. (See Table 3).
Table 3:
Representative results from participant weekly Qualtrics surveys
| Burden Question | StronglyAgree | Agree | Neutral | Disagree | StronglyDisagree |
|---|---|---|---|---|---|
|
| |||||
| The smartwatch is easy to learn how to use | 78.6% | 21.4% | 0% | 0% | 0% |
| The smartwatch asked me to answer questions while I was doing things | 14.3% | 28.6% | 35.7% | 21.43% | 0% |
| The questions on the smartwatch annoyed me | 20% | 20% | 20% | 40% | 0% |
| The surveys on the smartwatch distracted me | 35.7% | 7.14% | 21.4% | 35.7% | 0% |
There were a total of 18 submitted weekly Qualtrics surveys, but responses from one participant were missing because he incorrectly clicked on the electronic Likert scale when responding, yielding a final sample size of 15 surveys. All participants ‘agreed’ or ‘strongly agreed’ that the app was ‘easy to learn how to use,’ but participants also ‘agreed’ or ‘strongly agreed’ that the smartwatch prompts were distracting (43%) and annoying (40%). Participants ‘agreed’ or ’strongly agreed’ that the smartwatch prompts arrived when they were ‘doing things’ (43%).
4.4. Qualitative user feedback
At the end of the 21-day protocol, each participant completed a semi-structured, 15 to 30 min interview led by a team member. Participants were asked about their general impressions of the study, what they liked and did not like about the experience, how easy they found the protocol to be, and what changes they would like to see. Interviews were recorded and transcribed.
Participants generally found the study manageable and often enjoyable. When asked about positive experiences with the protocol, one participant stated, "I really like the pictures" [P03]. Four of the six participants, however, faced some challenges when using the watches. Poor and unpredictable battery life was the most common issue raised by participants. For example, P02 explained that the battery would drain more quickly than the phone: "The other day I brought it [the smartwatch] home. It was at 1% . . . and you look at the watch [gestured towards the phone, "watch" was likely a semantic word error for "phone" here] and it would be at 90%.”
Participants were asked to recall times or locations when they had difficulty responding to the prompts. One participant took a nap in the afternoon, and another worked in the morning to mid-afternoon. In general, when responding, commuting (e.g., driving, riding a bike, or flying on a plane) presented a challenge for five participants. For example, one participant explained, "I can’t. Usually if I am biking sometimes I can stop at a stoplight or something, and then it comes on. . . And then I have big gloves, so it’s sometimes yes but 90%, 80 or 90% I can’t do that ’cause I am biking" [P04].
5. DISCUSSION
Traditionally, accurate measurement of post-stroke anomia requires repeated standardized assessment that is not feasible in clinical practice due to time and insurance coverage limitations. Audio-based EMA could enable longitudinal, in-situ data collection of symptoms; μEMA could also provide temporally dense measurement. The method appears feasible for people with aphasia, at least for three weeks of data collection. Our completion and compliance rates demonstrate participants’ ability to adhere to the protocol. Compliance rates of approximately 80% are consistent with EMA compliance rates of 79% in the broader literature [29]; people with aphasia responded well to the prompts. Participant interview responses also suggest that they found the protocol to be understandable and reasonable to complete. The audio recording quality was generally good. Eleven percent of the samples were unintelligible, but most of these cases (88.6%; 39/44) were due to participants’ low speaking volume or speech impairments. Prompts were sometimes dismissed prematurely by the participants, most often in the single-image condition, which resulted in the latter part of some words being excluded from recordings. In the exit interviews, none of the participants reported annoyance at the frequency of the prompts as a negative aspect of the study. A naming task of even a single image might have been cognitively challenging and require more than the 3–4 s microinteraction recommended when using μEMA [15]. The single-item EMA also resulted in substantially more frequent interruptions than the nine-item EMAs; however, both the nine-item and single-item EMA protocols appeared to be feasible.
5.1. Challenges and adjustments
Our early data support the feasibility of this study design, but we also faced challenges during the creation and rollout of this EMA system.
One of our biggest challenges was the poor battery life of the smartwatch. We sometimes noticed that the battery could last for the entire 10-hour study period with more than 50% of its charge remaining, but on the worst days the battery level could fall from fully charged to below 20% within three hours of wearing it. Some watches consistently overheated and abruptly lost their charge during testing; this was identified as a manufacturing defect, and devices were returned to the supplier when this problem occurred. Unfortunately, there was no reliable way to identify this problem proactively, and no watches were guaranteed to be free of it (including watches replaced by the manufacturer). This problem highlights the importance of continuous, real-time device use and performance monitoring throughout the study deployment period.
Even watches without the manufacturer defect had unpredictable battery drain. Despite extensive testing in our lab and software adjustments to reduce the number of data transmissions between the phone and watch, the battery drain rate remained unpredictable, and the cause(s) were unknown. We could verify that repeated disconnection and reconnection of the Bluetooth wireless with the phone could contribute to, but not fully explain, rapid drain. This highlights a dilemma: when a participant leaves the phone in one location in a home instead of carrying the phone, the rate of Bluetooth connect/disconnect events may increase; however, asking the participant to carry the phone would further burden them. In short, consumer mobile technology is not optimized for research tasks and requires careful data monitoring during the study deployment period.
Another limitation of consumer technology is that we had limited control over the timing of interface elements on the smartwatch. As mentioned above, the operating system sometimes updated the display slowly, causing participants to tap through images too quickly. Specifically, the transition between the prompt screen would sometimes lag after a "YES" tap, so we added a 1.5 s ’debouncing’ delay before registering a second tap. Subsequent image taps in the nine-image EMA condition were similarly restricted.
We also encountered some challenges with application installation on the smartwatch that impacted our testing flexibility and complicated the process of adjusting the app during feasibility testing. An Android Wear smartwatch application must be available on the Google Play app store to receive automatic updates without requiring any participant interaction. Unfortunately, as of late, the procedures for approving apps have become unpredictable. We had versions of the app repeatedly rejected for reasons ranging from legacy permissions to icon image quality to concerns about why the app would not function on other types of smartwatches not used in this study. Each rejection could take 1–2 weeks to understand and resolve; we ultimately decided that the process was too unreliable for the first several participants, and we used a ’sideloading’ application to install the app on the watches, meaning we directly installed the application file onto the watch and phone without using the Google Play store. Sideloading severely restricted our ability to make improvements while devices were in the field; for example, it ultimately contributed to log data file loss for one participant.
5.2. Lessons learned
First, it is important to consider the specific underlying cognitive or motor impairments a study population faces and design the technology or build in workarounds, as needed. For example, language is the primary impaired cognitive domain for people with aphasia, so creating an app that relies little on language is important. Despite the app only displaying two buttons (‘YES’ and ‘NO’) and a one-task speech interface, relatively extensive training was required. Even then, one participant misunderstood the instructions, leading to extensive data loss. Two of our participants (P01 and P06) had pronounced right upper limb hemiparesis; during training, we ensured they could don and doff the watch one-handed and navigate the app with their non-dominant hand.
Second, data collection tools like the one we developed need to be robust enough to work regardless of an individual’s experience using the technology, Internet access, or ability to assist with debugging. In the case of some participants who experienced difficulties with the phone and watch disconnecting or a lack of Internet connectivity, despite impromptu phone check-ins, it was difficult or impossible for our team to assess how well the protocol was being followed or if the EMA application itself was working for the three-week study duration. Sometimes a design decision, such as using the phone as a data transfer device to enable remote monitoring, can unintentionally create other problems (e.g., exacerbating battery drain on the watch).
Third, onboarding protocols can benefit from interactive training. We believe that the 45-minute practice sessions where participants received prompts and trials structured identically to those during the full protocol improved their understanding of, and ability to complete, the study. We also added several testing and debugging screens to the watch that could be used during setup to ensure features like audio recording, smartwatch speakers, and prompt scheduling were working correctly.
Fourth, issues can arise despite internal testing, ranging from participant misunderstandings to hardware or software issues. Systems to quickly identify anomalies—remotely—and protocols to address them are invaluable. Importantly, we recommend that researchers spend as much effort developing and feasibility testing the researcher-side remote data checking and visualization tools as they spend developing the user-facing application. Both outcome data and app logging data must be easy to review daily. Building such tools is often delayed until feasibility testing has begun, but doing so may lead to some data loss among early participants. We also recommend that researchers find workarounds that enable auto-updating so that when remote monitoring does uncover problems, they can be resolved efficiently without further burdening participants. Ultimately, after these participants had completed the study, we could update our code and documentation so that we could publish our app on a testing track in the Google Play store, which allows for remote updating that takes days instead of weeks.
6. CONCLUSIONS
We designed and implemented a two-version audio-based EMA system for in-situ measurement of post-stroke anomia. Initial pilot and interview data, though from a small sample, support this kind of study’s feasibility based on compliance and data quality, albeit with extensive participant training during onboarding. We expect that audio-based EMA can benefit other domains where vocal data and speech patterns provide valuable signals, including the detection of dementia. In a future paper, we will examine the relationship between compliance and subject-specific factors such as age and aphasia severity.
Supplementary Material
Table 1:
Participant demographics
| ID | Gender | Age | QAB Overall (out of 10) | Aphasia Type | S & V Naming Test (out of 260) | Right-Sided Hemiparesis | Comfort level watch technology (out of 7) | Condition |
|---|---|---|---|---|---|---|---|---|
|
| ||||||||
| P01 | M | 54 | 6.18 | Broca’s | 112 | Yes | 5.5 | Single-image EMA |
| P02 | M | 69 | 9.43 | Anomic | 245 | No | 2.3 | Nine-image EMA |
| P03 | M | 52 | 8.27 | Anomic | 242 | No | 6.5 | Single-image EMA |
| P04 | M | 69 | 8.00 | Transcortical Sensory | 234 | No | 0.5 | Single-image EMA |
| P05 | M | 53 | a 10.00 | aAnomic | 249 | No | 3.7 | Nine-image EMA |
| P06 | W | 71 | 6.47 | Broca’s | 212 | Yes | 7.0 | Nine-image EMA |
P05 scored as non-aphasic by the QAB but endorsed lingering difficulties with language production, particularly with word retrieval. ‘S&V’ indicates ‘Snodgrass and Vanderwart’
ACKNOWLEDGMENTS
This work was funded by the National Institutes of Health (NIH) and the National Center for Advancing Translational Sciences grant UL1 TR002544, awarded to the Tufts Clinical and Translational Science Institute with pilot funds awarded to Drs. Meier and Intille. Additional support was provided by NIH grant P2C HD101899, which is supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development and National Institute of Neurological Disorders and Stroke and awarded to the Shirley Ryan AbilityLab. We thank Leanna Ugent, Nathalie Mitchell, and members of The Aphasia Network Lab for their assistance with project development and data acquisition. We express our gratitude to the individuals with aphasia who participated in the study, without whom this work would not have been possible.
Footnotes
ACM Reference Format:
Jack Hester, Ha Le, Stephen Intille, and Erin Meier. 2023. A feasibility study on the use of audio-based ecological momentary assessment with persons with aphasia. In The 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23), October 22–25, 2023, New York, NY, USA. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3597638.3608419
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.
Contributor Information
Jack Hester, Khoury College of Computer Sciences and Bouvé College of Health Sciences, Northeastern University.
Ha Le, Khoury College of Computer Sciences, Northeastern University.
Stephen Intille, Khoury College of Computer Sciences and Bouvé College of Health Sciences, Northeastern University.
Erin Meier, Bouvé College of Health Sciences, Northeastern University.
REFERENCES
- [1].Flowers HL, Skoretz SA, Silver FL, Rochon E, Fang J, Flamand-Roze C and Martino R Poststroke aphasia frequency, recovery, and outcomes: A systematic review and meta-analysis. Archives of Physical Medicine and Rehabilitation, 97, 12 (2016), 2188–2201. [DOI] [PubMed] [Google Scholar]
- [2].Grönberg A, Henriksson I, Stenman M and Lindgren AG Incidence of aphasia in ischemic stroke. Neuroepidemiology, 56, 3 (2022), 174–182. [DOI] [PubMed] [Google Scholar]
- [3].Pedersen PM, Jørgensen HS, Nakayama H, Raaschou HO and Olsen TS Aphasia in acute stroke: Incidence, determinants, and recovery. Annals of Neurology, 38, 4 (1995), 659–666. [DOI] [PubMed] [Google Scholar]
- [4].Goodglass H and Wingfield A Word-finding deficits in aphasia: Brain-behavior relations and clinical symptomatology. Foundations of Neuropsychology, 1997. [Google Scholar]
- [5].Davidson B, Howe T, Worrall L, Hickson L and Togher L Social participation for older people with aphasia: The impact of communication disability on friendships. Topics in Stroke Rehabilitation, 15, 4 (2008), 325–340. [DOI] [PubMed] [Google Scholar]
- [6].Lam JMC and Wodchis WP The relationship of 60 disease diagnoses and 15 conditions to preference-based health-related quality of life in Ontario hospital-based long-term care residents. Medical Care, 48, 4 (2010), 380–387. [DOI] [PubMed] [Google Scholar]
- [7].Kaplan E, Goodglass H and Weintraub S Boston Naming Test-2 (BNT-2). Pro-Ed, 2001. [Google Scholar]
- [8].Britt AE, Ferrara C and Mirman D Distinct effects of lexical and semantic competition during picture naming in younger adults, older adults, and people with aphasia. Frontiers in Psychology, 7, 813 (2016), 1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Jefferies E and Ralph MAL Semantic impairment in stroke aphasia versus semantic dementia: A case-series comparison. Brain: A Journal of Neurology, 129, 8 (2006), 2132–2147. [DOI] [PubMed] [Google Scholar]
- [10].Stone AA and Shiffman S Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 3 (1994), 199–202. [Google Scholar]
- [11].Csikszentmihalyi M and Larson R Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 9 (1987), 526–536. [DOI] [PubMed] [Google Scholar]
- [12].van Berkel N, Ferreira D and Kostakos V The experience sampling method on mobile devices. ACM Computing Surveys, 50, 6 (2017), 1–40. [Google Scholar]
- [13].Fuller-Tyszkiewicz M, Skouteris H, Richardson B, Blore J, Holmes M and Mills J Does the burden of the experience sampling method undermine data quality in state body image research? Body image, 10, 4 (2013. 2013), 607–613. [DOI] [PubMed] [Google Scholar]
- [14].Courvoisier DS, Eid M and Lischetzke T Compliance to a cell phone-based ecological momentary assessment study: The effect of time and personality characteristics. Psychological Assessment, 24, 3 (2012), 713–720. [DOI] [PubMed] [Google Scholar]
- [15].Intille S, Haynes C, Maniar D, Ponnada A and Manjourides J μEMA: Microinteraction-based ecological momentary assessment (EMA) using a smart-watch. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2016), 1124–1128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16].Ponnada A, Haynes C, Maniar D, Manjourides J and Intille S Microinteraction ecological momentary assessment response rates: Effect of microinteractions or the smartwatch? Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1, 3 (2017), 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Ponnada A, Li J, Wang S, Wang W-L, Do B, Dunton GF and Intille SS Contextual biases in microinteraction ecological momentary assessment (μEMA) non-response. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6, 1 (2022), 1–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Ponnada A, Wang S, Chu D, Do B, Dunton G and Intille S Intensive longitudinal data collection using microinteraction ecological momentary assessment: pilot and preliminary results. JMIR Formative Research, 6, 2 (2022), e32772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Mehl MR The Electronically Activated Recorder (EAR): A method for the naturalistic observation of daily social behavior. Current Directions in Psychological Science, 26, 2 (2017), 184–190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].Mehl MR, Pennebaker JW, Crow DM, Dabbs J and Price JH The Electronically Activated Recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, & Comput, 33, 4 (2001), 517–523. [DOI] [PubMed] [Google Scholar]
- [21].Kim Y-H, Chou D, Lee B, Danilovich M, Lazar A, Conroy DE, Kacorri H and Choe EK MyMove: Facilitating older adults to collect in-situ activity labels on a smartwatch with speech. CHI Conference on Human Factors in Computing Systems (2022), 1–21. [Google Scholar]
- [22].Fitzgerald-DeJean DM, Rubin SS and Carson RL An application of the experience sampling method to the study of aphasia: A case report. Aphasiology, 26, 2 (2012), 234–251. [Google Scholar]
- [23].Brandenburg C, Worrall L, Copland D and Rodriguez A An exploratory investigation of the daily talk time of people with non-fluent aphasia and non-aphasic peers. International Journal of Speech Language Pathology, 19, 4 (2017), 418–429. [DOI] [PubMed] [Google Scholar]
- [24].Williams K, Mofatt K, Hong J, Faroqi-Shah Y and Findlater L The cost of turning heads: A comparison of a head-worn display to a smartphone for supporting persons with aphasia in conversation. Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (2016), 111–120. [Google Scholar]
- [25].Wilson SM, Eriksson DK, Schneck SM and Lucaine JM A quick aphasia battery for efficient, reliable, and multidimensional assessment of language function. PLOS One, 13, 2 (2018), e0199469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Sitren A and Vallila-Rohter S How well do we use our technology? Examining iPad navigation skills in individuals with aphasia and older adults. American Journal of Speech Language Pathology, 28, 4 (2019), 1523–1536. [DOI] [PubMed] [Google Scholar]
- [27].Snodgrass JG and Vanderwart M A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology, 6, 2 (1980), 174–215. [DOI] [PubMed] [Google Scholar]
- [28].Brodeur MB, Guerard K and Bouras M Bank of Standardized Stimuli (BOSS) phase II: 930 new normative photos. PLoS One, 9, 9 (2014), e106953. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29].Wrzus C and Neubauer AB Ecological momentary assessment: A meta-analysis on designs, samples, and compliance across research fields. Assessment, 30, 3 (2023), 825–846. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
