Can older people remember medication reminders presented using synthetic speech?

Abstract

Reminders are often part of interventions to help older people adhere to complicated medication regimes. Computer-generated (synthetic) speech is ideal for tailoring reminders to different medication regimes. Since synthetic speech may be less intelligible than human speech, in particular under difficult listening conditions, we assessed how well older people can recall synthetic speech reminders for medications. 44 participants aged 50–80 with no cognitive impairment recalled reminders for one or four medications after a short distraction. We varied background noise, speech quality, and message design. Reminders were presented using a human voice and two synthetic voices. Data were analyzed using generalized linear mixed models. Reminder recall was satisfactory if reminders were restricted to one familiar medication, regardless of the voice used. Repeating medication names supported recall of lists of medications. We conclude that spoken reminders should build on familiar information and be integrated with other adherence support measures.

BACKGROUND AND SIGNIFICANCE

As more people live longer, levels of multimorbidity have increased, leading to more complex medication regimes. Non-adherence can have serious consequences, including hospitalization and death.^1,2 Forgetting what to take when is a common cause of non-adherence.³ Reminders that help users remember those details are an important part of successful adherence interventions.^4–6

Spoken reminder messages can be presented through many channels such as interactive voice response systems, a digital TV, or electronic pill boxes. Using a computer-generated (synthetic) voice, spoken messages can be tailored quickly and cost effectively to different patients and medication regimes, which may make them more effective at changing medication behavior.⁷

However, despite major advances in speech synthesis technology, synthetic speech is still not as intelligible as human speech,⁸ and the perceived listening effort is significantly higher.⁹ In this study, we assess whether synthetic speech is sufficiently intelligible to allow older people (who are likely to have hearing loss) to recall medication names, which are difficult to remember.¹⁰

Hearing loss starts early. About one in four adults aged 50–59 has a hearing loss in the frequencies covered by speech sounds,¹¹ and incidence rises sharply with age.¹² As more cognitive and perceptual effort is required to process auditory stimuli, fewer cognitive resources may be available for understanding and remembering what was said.¹³

Older people find synthetic speech more difficult to understand than younger people.^14,15 However, most related work^14–19 has focused on older speech synthesis technology and may therefore overestimate intelligibility problems. Here, we used publicly available implementations of the two main current speech synthesis approaches, statistical parametric synthesis^20,21 and unit selection^22,23 (figure 1). Both methods start with a large database of speech. Given a message, unit selection systems search the database for pieces of speech that fit the message and concatenate them. Statistical parametric systems use the database to build a set of statistical models of all core speech units, which are then used to generate arbitrary new messages.

Figure 1:

One word, two ways of converting it to speech: unit selection (left) and statistical parametric synthesis (right).

Reminders are often heard against background noise, such as radio, TV, or road traffic, or they can be transmitted through noisy telephone lines, which may make them more difficult to understand. Listeners may also be distracted by other tasks or people after hearing a reminder. Therefore, we varied listening conditions in our study design and included a brief distraction before recall.

Better message phrasing and organization can counteract unfavorable listening conditions. In our study, we tested two options: adapting the message to how listeners think about medication, for example by linking each medication to the reason for taking it,^24,25 and repeating key information.²⁶

OBJECTIVES

This study was designed to determine if older people with a range of hearing ability (from clinically normal to some age-related hearing loss) find medication reminders more difficult to recall when these are presented using a synthetic as opposed to a human voice. We included two factors that might impair recall: memory load (one vs four medications per reminder) and listening conditions (difficult vs acceptable). Finally, we tested whether message design might be able to counteract recall problems. Two alternatives were tested: explaining medication indications (Explanation) and repeating medication names (Repetitions).

MATERIALS AND METHODS

Stimuli

Based on 12 frequently taken UK medications (six over-the-counter and six prescription), we created 12 one-medication reminders, 24 baseline four-medication reminders, 12 four-medication reminders where the medications were repeated, and 12 four-medication reminders where indications were explained (table 1).

Table 1:

Examples of the medication reminders used in the study

Reminder	Example
One medication
Baseline	Please remember to take the following medication: aspirin.
Four medications
Baseline	Please remember to take the following four medications: aspirin, Corsodyl, amoxicillin, and Dulcolax
Repetition	Please remember to take the following four medications: aspirin, Corsodyl, amoxicillin, and Dulcolax. I repeat: aspirin, Corsodyl, amoxicillin, and Dulcolax
Explanation	Please remember to take the following four medications: aspirin, to thin your blood, Corsodyl, for your mouth ulcer, amoxicillin, for your infection, and Dulcolax, for your constipation.

Auditory stimuli were generated using three male voices with a standard British English accent: a human voice, a statistical parametric synthesis voice,²¹ and a unit selection voice.²³

Participants heard stimuli in two levels of background noise (multi-speaker speech shaped babble,²⁷ signal-to-noise ratios 0 and + 10) and two levels of signal transmission quality (noisy and clear telephone line).

To reduce duration, we only tested the Explanation and Repetition message designs for high-memory load reminders with four medications. Participants heard both designs under two listening conditions, clear phone line/soft background noise (acceptable) and noisy phone line/loud background noise (difficult).

Participants

Ethics approval for this study was granted by the South East Scotland NHS Ethics Board, reference number 10/S1103/43. Participants were recruited from four family practices in Edinburgh through the Scottish Primary Care Research Network. We contacted native speakers of English aged 50 + with no hearing aids, no cognitive impairment, no neurodegenerative disorder, and sufficient mobility to attend testing. Of 56 participants who attended, 44 met the inclusion criteria.

Procedure

In a questionnaire, participants indicated whether they were familiar with the 12 medications used in the task. For each familiar medication, they were asked to describe its use.

Participants were screened for cognitive impairment using the Addenbrooke's Cognitive Examination-Revised.²⁸ Working memory capacity was assessed by a reading span test.²⁹ Working memory scores were recoded into three levels: first quartile, second/third quartile, and fourth quartile.

Hearing levels were measured using pure-tone audiometry,^30,31 and severity was categorized following Martini and Mazzoli.³² Participants were considered to have a hearing loss if they had at least a mild hearing loss in their poorer ear. We excluded people with a severe hearing loss in at least one ear or a conductive hearing loss (following ref. 33).

The intelligibility test consisted of a total of eight training and 72 test trials. Participants adjusted loudness to a comfortable level at the start of the experiment.

In each trial, participants heard a medication reminder, followed by a 2 s pause and a simple five-word distractor sentence.³⁴ After repeating the distractor, participants had to recall the medications in the reminder. Figure 2 shows a sample trial.

Figure 2:

Sample intelligibility trial.

To support recall, participants were shown a list of 24 possible medications and their key indication. Twelve targets that occurred in the reminders and 12 alternatives matched for indication. Responses were audio-recorded and transcribed verbatim by the experimenter. The reminder score was the number of correctly remembered medications.

All participants heard the same reminder/distractor pairs in the same, randomized order. The assignment of voices to sentences was balanced using a Latin square design.

Participants were debriefed in a semi-structured exit interview.

Statistical analysis

Intelligibility scores were analyzed using generalized linear mixed models (GLMM^35,36). Following established best practice for GLMMs,³⁶ the effect of each predictor is illustrated with a graph showing the 95% CI for that predictor. Fisher's exact test and pairwise t tests corrected for multiple comparisons as implemented in R (V.3.0.2) were used to illustrate effects of single predictors.

RESULTS

Demographic and baseline data are summarized in table 2. The decline in working memory capacity and the prevalence of hearing loss in the sample is consistent with normal ageing.^11,37

Table 2:

Demographics of participants

Gender
Female	N = 24	55%
Male	N = 20	45%
Age
Mean (SD), years	64 (9)
50–59	N = 15	34%
60–69	N = 14	32%
70 +	N = 14	32%
No information	N = 1	2%
Education (highest level achieved)
Postgraduate	N = 11	25%
Graduate	N = 9	20%
Vocational	N = 6	14%
Secondary school*	N = 14	32%
None	N = 1	2%
No information	N = 3	7%
Hearing loss (on worse ear)
None	N = 26	59%
Mild, monaural	N = 7	16%
Mild, binaural	N = 8	18%
Moderate on worse ear, mild on better ear	N = 2	4.5%
Moderate, binaural	N = 1	2%
Working memory capacity†
Mean (SD)	43 (13)
Age 50–59	51 (12)
Age 60–69	42 (15)
Age 70 +	38 (11)

*Secondary school qualifications include GCSE, Leaving Certificate, and A-Levels.

^†The score is the sum of all items recalled correctly in the correct order. Scores range from 0 to 70.

On average, participants knew five of the over-the-counter and two of the prescription medications.

Recall was perfect for 74.8% (790 of 1056) of the one-medication reminders, but only for 4.5% (95 of 2112) of the four-medication reminders. When participants made a mistake, they mostly forgot a medication name or confused the medication with another target medication.

Figure 3 shows the results of the GLMM analyses for one medication, four medications (baseline message design only), and four medications (message design varied).

Figure 3:

Graphical summary of the three generalized linear mixed models fitted for the main analysis. (A) Model for one medication only. (B) Model for four medications, no variation in message design. (C) Model for four medications, message design varied. The graphs show 95% CIs for the estimates of each individual level predictor. Group level predictors are not shown. In the text, predictors are referred to as having a significant effect if the 95% CI of the corresponding estimate does not include 0. The individual level corresponds to the variation among trials, while the group level covers variation among participants. We used a default intercept and the predictors presence of hearing loss (Hearing), working memory capacity level (Memory), signal quality (Signal), background noise level (Noise), medication familiarity (Known), and voice type (SPS: statistical parametric synthesis, USel: unit selection). For the Message Design model, we added predictors corresponding to message type, Explanation and Repetition. The Signal and Noise conditions were collapsed into one predictor, difficult listening condition (Difficult). On the participant level, we used a default intercept that captured inter-individual variation.

As expected, listeners with a hearing loss found it more difficult to recall the reminders than listeners with normal hearing. Working memory capacity does not have a significant effect after controlling for hearing. Signal quality did not affect recall; the effect of background noise varied depending on the number of medications and message design. Figure 3 illustrates the corresponding parameter values.

Looking at one-medication reminders, we find that known medications are typically recalled correctly (89.3% (484 of 541) recall for known, 59.5% (306 of 512) recall for unknown; p<0.001, OR 5.7, 95% CI 4.1 to 8.0).

Although voice type affected recall, this difference was almost entirely due to unknown medications. For known medications, participants recalled 89% (human voice), 90% (statistical parametric synthesis), and 88% (unit selection), while for unknown medications, they recalled 64.8% (human voice) and 52.2% (both synthetic voices).

We found similar patterns in the four-medication case. Listeners knew an average of 1.9 medications and remembered 1.6. The average number of medications recalled for the human voice was 1.86 (SD 1.1), 1.5 (SD 1.1) for statistical parametric synthesis, and 1.6 (SD 1) for the unit selection voice (figure 3B).

Message design improved recall. With the basic design, listeners recalled an average of 1.6 medications (SD 1.0), with explanations, 1.8 (SD 1.1), and with repetitions, 2 (SD 1.0). Only the effect of repeating medication names was significant after controlling for all other relevant variables (figure 3C).

Repeating medications particularly helped listeners understand the synthetic voices. In the difficult listening condition (loud noise, bad signal), listeners recalled an average of 1.7 medications (SD 1.1) in the human voice, but only 1.4 (SD 1.1) when medications were presented using statistical parametric synthesis, and 1.5 (SD 1.0) when they heard the unit selection voice. Using repetitions, recall increased to a mean of 2.1 (human, SD 1.0), 1.9 (statistical parametric synthesis, SD 1.1), and 2.1 (unit selection, SD 1.1). Unit selection performed as well as the human voice (pairwise comparisons using the t test with adjustments, p<0.6) and outperformed statistical parametric synthesis (p<0.02).

DISCUSSION AND CONCLUSION

When presenting short reminders for known medications, a high-quality synthetic voice works as well as a human voice. However, for longer lists of medication names or unknown medication names, recall is not satisfactory, regardless of voice.

Therefore, spoken medication reminders will only work if they build on what users know. Knowledge can come from users’ own experience of taking medications, from friends and family, or from the media. In practice, this means that spoken reminders have to be integrated into complex interventions that allow for patient input.

Speech synthesis method matters. While both synthetic voices performed similarly overall, unit selection, which preserves more acoustic information, was more robust when lists of medication names were repeated, and statistical parametric synthesis performed better for one-medication reminders. Reminders could also be sent directly to users’ hearing aids, bypassing much of the ambient distortion.

Repeating key information is significantly better at improving recall than explanations, even though in the exit interviews, participants thought that the explanations were particularly helpful. While repetition alone is not enough to help people recognize or recall information,^38,39 in this case, it provides a second chance to catch an auditory glimpse of the speech signal,⁴⁰ building a more robust percept that is in turn more likely to be recognized later.

Limitations

The distractor task was designed to prevent participants from rehearsing the medication name(s) subvocally, a common strategy for remembering auditory information.⁴¹ We need to examine the effect of more realistic distractors that use modalities other than hearing. We also did not probe why participants were familiar with a given medication.

It is not clear why working memory had little effect. This could be due to the high proportion of participants with relatively good working memory. Working memory might also be less important than other cognitive abilities, such as speed of processing.

In line with best practice in designing health information technology,⁴² our study was conducted in a laboratory environment. In deployment, reminders will need to be tailored to fit with a person's concepts of their medication and their routine, and to reduce the potential for unintended disclosure of illness.^43,44

The medication list provided an overview of possible medications similar to a list provided by a clinician, or to a medicine cabinet where people can see all their medications at once. However, real medicine cabinets (and medication lists) are often far more messy.^45,46 Reading level may also have affected participants’ ability to use the list.

Finally, we used actual medication names instead of phrases such as ‘morning medication’ that patients typically use to describe medications. However, such phrases require clinicians and pharmacists to cooperate with patients and carers. Otherwise, patients may not remember what pills are included in the ‘morning medication.’

CONCLUSION

In order that the usefulness and usability of computer-generated reminders in practice can be assessed, they need to be integrated into a multimodal medication management system that also provides tailored information about other aspects of the medication such as dosage and side effects.