Skip to main content
NCBI home page
AMIA Annual Symposium Proceedings logoLink to AMIA Annual Symposium Proceedings
. 2018 Dec 5;2018:185–194.

Using conversational agents to explain medication instructions to older adults

Renato F L Azevedo, Dan Morrow 1, James Graumlich 2, Ann Willemsen-Dunlap 2,3, Mark Hasegawa-Johnson 1, Thomas S Huang 1, Kuangxiao Gu 1, Suma Bhat 1, Tarek Sakakini 1, Victor Sadauskas 2, Donald J Halpin 2,3
PMCID: PMC6371340  PMID: 30815056

Abstract

In an effort to guide the development of a computer agent (CA)-based adviser system that presents patient-centered language to older adults (e.g., medication instructions in portal environments or smartphone apps), we evaluated 360 older and younger adults’ responses to medication information delivered by a set of CAs. We assessed patient memory for medication information, their affective responses to the information, their perception of the CA’s teaching effectiveness and expressiveness, and their perceived level of similarity with each CA. Each participant saw CAs varying in appearance and levels of realism (Photo-realistic vs Cartoon vs Emoji, as control condition). To investigate the impact of affective cues on patients, we varied CA message framing, with effects described either as gains of taking or losses of not taking the medication. Our results corroborate the idea that CAs can produce a significant effect on older adults’ learning in part by engendering social responses.

Introduction

The purpose of our research project is to improve understanding and memory for medication information (especially older adults) by leveraging progress in computer-agent communication. More than a quarter of all Americans and two out of three older adults have multiple chronic conditions. Treatment for this population accounts for 66% of the country’s health care budget.1,2 Age-related increase in chronic illness is accompanied by increasing demands on self-care such as taking medication. Medication adherence - the extent to which a person’s medication use corresponds to recommendations from a healthcare provider – is estimated to be around 50% for chronic conditions.1,2,3 Poor adherence causes about 33-69% of all medication-related hospital admissions in the U.S., can intensify disease or even cause death, and costs the healthcare system about $100 billion annually disease or even cause death, and costs the healthcare system about $100 billion annually4,5. Reasons for low medication adherence are manifold, including poor communication and inadequate health literacy, especially for older adults4. Typically, patients (especially older adults) rely on providers to help them understand directions. In face-to-face communication, providers can use both verbal and nonverbal cues (e.g. tone of voice and facial expressions) to indicate what information is most important, which help patients remember medical information6. Unfortunately, these strategies are not always used consistently, so patients leave office visits without enough information or remembering little of what was presented7.

To address low medication adherence, innovative methods – including: new technologies, audiovisual education materials, and others, are recommended4. However, older adults, who stand to gain the most from this technology to support their self-care needs, are often the least likely to use it. Older adults with lower health literacy are less likely than younger adults to use portals to Electronic Health Record (EHR) systems because of limited access to the internet and other resources, and increasingly because inadequate design makes health information delivered by these systems less cognitively accessible.8,9,10 Therefore, it is critical to design such technology with older adults’ needs and abilities in mind. An important goal of our project is design technology to support older adults’ medication management and to evaluate whether older as well as younger adults are open to using and will benefit from the technology. Our system consists of two integrated components. The first component is an automated machine translation (MT) system 11, which given the provider’s technical and complicated input, generates a patient-centered and simple output, while keeping the provider’s intended meaning. That component of our system identifies complicated words in EHR based on frequency in English Gigaword corpus56. For example, we expand ambiguous abbreviations (e.g., ‘RA’ becomes ‘Rheumatoid Arthritis’ or ‘Refractory Anemia’ depending on the context), as well as replace complicated terms (e.g., ‘QD’ becomes ‘daily’). Without the need for a suitable parallel corpus, the proposed machine translation framework utilizes the well-established medical vocabulary resource, Unified Medical Language System (UMLS)57, 58. Subsequently, it considers multiple UMLS concepts and relies on a language model on the English Gigaword corpus56 to choose the most accurate and simplest translation (for details see11).

Second, we are developing a Conversational Agent (CA), or ‘virtual provider’ to deliver this information in ways that engage older adults to perform self-care. In an effort to guide the development of this CA to deliver medication instructions in portal environments (or smartphone apps), we assessed patient memory for the medication information delivered by the CA, their affective responses to the information, their perception of the CA’s teaching effectiveness and expressiveness, and their perceived level of similarity with the CA. Preliminary evaluation found a photorealistic CA generated from a video provider was as effective as the video in supporting memory for and responses to the messages12. The next step, described in this paper, was to expand the capability of the CA-based system by developing and evaluating a broader set of CAs that varied along dimensions that can be important for patient/provider communication.

Ideally, the CA emulates best practices for face-to-face communication but is also generative, with capability of delivering a wide range of health information to diverse patients. CAs can be implemented to use nonverbal (e.g., facial expressions, prosody) as well as verbal cues to convey the affective and cognitive meaning of the medication information.13,14,15,16 Social and emotional functions become increasingly important across the lifespan, with older adults more likely than younger adults to notice and remember emotionally relevant information (especially positive information, aka positivity effect).17,18 Affective processes tend to be preserved relative to declining fluid mental ability in older adults.19 According to fuzzy-trace theory, emotion-based decisions may be most effective especially because gist comprehension (e.g., of risk for illness) is often organized in terms of evaluative relations26. In addition, affective and nonverbal cues in patient-provider communication are often associated with patient satisfaction27. Furthermore, a growing literature on health decision making suggests that older adults are more likely to use heuristic processing than are younger adults.28 The framing of health decisions in terms of potential gains versus losses has been shown to influence choice preferences. Moreover, gain frames tend to activate positive affect while loss frames activate negative affect29,30. There is evidence that older adults better remember gain-framed messages relatively to younger adults.29,30. However, the relative influence of gain- and loss-framed messages tends to be contingent on how people perceive the behavior promoted in health messages29,30,31. Moreover, in health care, older adults may better remember than younger adults because they generally know more about health topics.24,25 Often when older adults integrate information and decision options for decision tasks in which they have knowledge, motivation20,21, or knowledge of social and emotional relational relationships22,23 age differences in performance are inexistent or older adults may even outperform younger adults.

CAs may be well suited for explaining complex concepts to patients (e.g., older adults and/or patients with multiple chronic conditions) by using exemplary communication techniques without time constraints in patient portals or other contexts.12 Hence, CAs can complement face-to-face communication, potentially reducing the amount of time physicians must spend instructing each individual patient. As CAs increase in realism, people are more likely to treat them as social beings. There is some evidence that learning (e.g., about Science, Technology, Engineering and Math, (STEM) topics) is improved by more realistic vs cartoon agents, although the evidence is mixed, and the studies focus on students32,33. Moreover, beyond some point of realism, people may respond negatively to the CA (the ‘uncanny valley’34). In the health domain, CAs with realistic facial expressions, gestures, and other relational cues can improve learning, support a variety of patient goals, and help diverse patients follow self-care recommendations, as compared to text-based or lower fidelity interfaces12-15,49-52. CAs have been shown to be at least as effective as human professionals in explaining medical information to patients, and more effective than standard EHR formats of communication12.

To sum up, in this present study, we investigated the following questions related to the impact of CA characteristics on older and younger adults memory for medication information and their responses to the messages delivered by the CAs: 1) Are there age differences in message memory? Do these effects depend on CA age? 2) Are there age differences in affective response to CA-based communication? 3) Are gain-framed messages better remembered than loss-framed messages? 4) Does the level of realism of the CA influence memory and response to the messages? 5) How do older and younger adults evaluate the CAs’ teaching effectiveness and expressiveness, and their perceived level of similarity with the agents?

Method

Materials and Procedure

Using the CrazyTalk software35, we developed more naturalistic CAs than our previous version12. We expanded the options of CAs (now varied in gender and age) to investigate whether participants prefer CAs that match them on these categories. CA realism (photo-realistic or cartoon) was also varied. We also compared a highly stylized agent (emoji) to the photorealistic and cartoon versions of our CA as a control condition (Figure 1).

Figure 1.

Figure 1.

Open in a new tab

Conversational Agents (CAs) of varying age, gender and levels of realism

Participants were asked to complete a survey online using Qualtrics, and after filling some demographics questions, they watched short video messages about medications (~30 seconds each) delivered by CAs, and answered questions about the medication’s use and effects, their affective responses when receiving the medication messages, the CAs’ teaching effectiveness and expressiveness, and their perceived level of similarity with the CAs.

After two practice rounds (with neutral landscape and voice only), each participant viewed either older or younger CAs (between-subjects variable) explaining the uses and effects of six different medications (beta-blocker, antiarrhythmic, diabetes, asthma, antibiotic, hypothyroidism). In addition, to investigate the impact of affective cues on patients, we varied CA message framing, with effects described either as gains (“If you take your beta blocker you will strength your heart”) or loss (“If you do not take your beta blocker, you will weaken your heart”). CA speech emphasized important concepts (e.g., emphasized words; Table 1). In addition, CA speech and appearance was varied to convey affective information, as it is shown to support gist memory as well as motivate patients (Figure 2). Hence, participants were exposed to 12 counterbalanced scenarios. CAs were presented in a randomized order.

Table 1.

Example of medication scenario (gain- and loss-framed)

[Gain-framed] This prescription is for a medicine called Amiodarone. Amiodarone acts on your heart rhythm. If you take Amiodarone as directed by your doctor, your heart rhythm will be steady and regular. You will also have a heartbeat that is strong and steady. Your heart will pump enough blood so that you feel more energetic and can breathe more easily when you take Amiodarone.
[Loss-framed] This prescription is for a medicine called Amiodarone. Amiodarone acts on your heart rhythm. If you don’t take Amiodarone as directed by your doctor, your heart rhythm will be unsteady and irregular. You will also have a heartbeat that is weak and unsteady. Your heart may not be able to pump enough blood so that you might feel faint and short of breath if you do not take Amiodarone.
Open in a new tab

Figure 2.

Figure 2.

Open in a new tab

CA Example delivering gain-framed (left) and loss-framed (right) messages.

After viewing each CA message, participants answered two comprehension questions (“What is this message used for?” and “If you take/do not take this medication how can it help/harm you?”). Memory for the medication effects was measured in two ways: (1) as a correct proportion in comparison with the number of effects propositions stated; (2) as a correct proportion in comparison with the total number of effects propositions in the message.

For example, if a participant answered after seeing the Amiodarone CA- gain-framed message (see Table1), that “his/her heart will pump enough blood and he/she will feel more energetic”, the first correct proportion will be 2 correct ideas out of 2 propositions stated by the respondent (100%), whereas the second will account for 2 correct ideas out of 5 possible ideas present in the message (40%). Because responses to the first question (older = 96%; younger = 93%, correct) and the proportions out of effect propositions stated were very accurate (older = 91.4%; younger = 91.3%, correct), we focus on analyses of the correct proportion in comparison with the total number of effects propositions in the message , which had more variation. They also indicated the extent (9 pt. scale, 1= not at all; 9=very much) that they felt a list of positive and negative emotions while watching the messages31. Participants also rated each CA’s teaching effectiveness and expressiveness (e.g., agent was knowledgeable/was expressive) (5 pt. scale, Agent Persona Instrument, API Measure36), as well as perceived similarity to the agent (7 pt. scale37). At the end of the task we also asked participants to rank order which CA they would prefer to deliver their health messages. Results from this task will be presented in a future paper.

Sample

We evaluated response to medication information delivered by these CAs among a sample of older and younger adults. Participants (n=360, age range: 19-77) were recruited on Amazon Mechanical Turk (MTurk), an online service that allows human workers to complete small tasks for monetary compensation. Studies conducted via MTurk have been shown to be as reliable as those from traditional approaches38,39. We restricted the workers to being in the United States with 95% approval ratings on previous MTurk tasks. Figure 3 illustrates the geographic distribution of our sample. Age was associated with health literacy (r(358) = .28, p<.001), but not with self-rated health condition (r(358) = -.06, p >.10), nor education (r(358) = .01, p >.10). The total average session time was 27.3 minutes.

Figure 3.

Figure 3.

Open in a new tab

Sample Geographic Distribution

Results

Overview of analysis: A 2 (participant age: older>=60 and younger) x 2 (message frame: gain vs loss) x 3 (CA levels of realism: photorealistic vs cartoon vs emoji) ANOVA was conducted on message memory, affective response, CA evaluation (API measure), and perceived similarity of self to CA. Due to the unbalanced sample, we also ran analyzes in which we randomly selected a subset of younger adults to balance the number of older adults. The results yield the same conclusions; hence we reported the analysis of the full dataset.

Effects of Age

Older adults remembered the messages more accurately than younger adults (O=51%; Y=46% correct out of possible, (F(1,358 =7.9, p<.01, η2=.01)). Older adults may better remember than younger adults because they generally know more about health topics24,25, which in turn may reflect greater interest due to personal relevance.

Figure 4 shows older and younger adults affective responses to CA-based communication (average scores on a 9-point scale31). Older adults were overall more positive than younger adults (F(1,358) =8.2, p<.01, η2=.01). But they were no different for negative affective responses (F(1,358) = 0.04., p>.10, η2<001, ns). Similar results were found in a different study from our group42. These findings are consistent with socioemotional selectivity theory17,18,20, as individuals focus increasingly on emotional regulation with age. In particular, older adults may focus on positive rather than negative-related information (positivity effect) as a way to self-regulate emotion. Moreover, Figure 4 shows that gain frames tend to activate positive affect while loss frames activate negative affect, which will be further elaborated in this paper.

Figure 4.

Figure 4.

Open in a new tab

Affective responses

Previous studies have explored agent similarity effects (e.g., gender, age, instructional role, ethnicity) between the learner and the agents41,53,54,55. These studies often reported that participants who interacted with non-matching CAs were less accurate in their risk perceptions than those who interacted with concordant CAs41. To further explore this question, we conducted a 2 (participant age: older>=60 and younger) x 2 (CA Age: older vs younger) x 2 (message frame: gain vs loss) x 3 (CA levels of realism: photorealistic vs cartoon vs emoji) ANOVA on message memory. Differently, the interaction of participants age and CA age was not significant (F(1,356=0.01, p>.10, η2<001, ns).

When considering how participants evaluated the CA’s effectiveness, we found a marginally significant trend (F(1,356) =3.7, p<.10, η2=.01), showing an interaction between participants’ age and CA age. We predicted that older adults would rate an older CA more positively than a younger CA, consistent with the matching hypothesis. We conducted this planned comparison despite the marginally significant omnibus tests43. A Welch’s unequal variances t-test showed that older CAs were indeed rated higher by older adults relatively to younger adults (O=3.13; Y=2.88 API score; t(108.5) = 2.2, p<.05). Older adults and younger adults do not differ in rating younger CAs (O=3.01; Y=3.05 API score; t(130.15) = -0.5, p>. 10, ns).

Furthermore, older and younger adults do not differ in perceived similarity of self to CAs (F(1,358) =1.4, p<.10, η2=.003). Neither an interaction between participant’s age and CA age is significant (F(1,356) =0.3, p<.10, η2=.001).

Effects of Framing

The results of message framing indicate that messages emphasizing gains were better remembered than the messages delivered as loss-frames (gain = 48.5%; loss = 46.3%; F(1,358 =19.6, p<.001, η2=.003)). Participants overall rated the CA more positively (teaching effectiveness and expressiveness) when the CAs delivered gain versus loss message (gain: 3.1; losses: 2.9; F(1,358 =125.4, p<.001, η2=.02). Furthermore, participants even felt they were more similar to the CAs that delivered gains than losses (gain: 2.8; losses: 2.4, similarity; (F(1,358 =91.0, p<.001, η2=.02). These results are consistent with the framing literature.29,30 Figure 4 shows that gain frames tend to activate more positive affect (means positive: 6.38 > negative: 2.78) while loss frames activate more negative affect (means positive: 3.33 < negative: 5.66).29,30 Framing effects did not interact with age. Hence, these findings suggest that gain-framed messages have benefits for adults of all ages.

Effects of CA Realism

The effects of CA realism on message memory were not significant (Realistic: 47.8%, Cartoon: 47.4%, Standard: 47.1%; F(1.98,709.10) =0.53, p >.10, η2<.001). However, realistic and cartoon CAs were better evaluated than the control emoji (Realistic: 3.1, Cartoon: 3,1, Standard 2.8, API composite score; F(1.52,544.08)=66.1, p<.001, η2=.01)) and participants also perceived the CAs, as more similar to them relatively to the emoji; Realistic: 2.8; Cartoon: 2.8, Standard: 2.2; F(1.41,505.55)=88.3, p<.001, η2=.03). The results perhaps reflect the fact that participants felt more similar to either CA than to the emoji. Thus, realistic CAs did not fall into the ‘uncanny valley’. CA realism did not interact with age.

For the rank order CA comparisons, female CAs were preferred than male CAs (62.1%). We found no evidence of a matching hypothesis for gender (χ2= 4.3, df=2, p-value >. 10, ns.), as 55% of our male participants also preferred their health information to be delivered by a female CAs.

Conclusion

To address low medication adherence, worse outcomes, and increased cost of care, among patients with chronic disease, we are developing a novel interactive technology for patient medication self-management. This system leverages a fully-automated machine translation (MT) system to translate highly technical medication information from the EHR and deliver patient-friendly instructions via a conversational agent (CA). In an effort to guide the development of a novel CA to deliver medication instructions, we assessed patient comprehension, their affective responses when receiving medication instructions, their perception of CA teaching effectiveness and expressiveness, and their perceived level of similarity with a set of different CAs.

Older adults responded positively to the CAs (especially the older CAs), even remembering the messages more accurately than younger adults did. These results corroborate the idea that CAs can support older adults’ learning in part by engendering social responses44,32,33,13. The CA used nonverbal and verbal cues to convey the affective and cognitive meaning of the medication information. By integrating the MT system with the CA, our interactive technology for patient medication self-management will have the ability to take technical, patient-specific medication information directly from the EHR and output a patient-friendly audiovisual presentation via the CA, simply and accurately explaining medication instructions. Our findings also have broad implications for improving the use of technology to support older adults’ self-care. Studies have demonstrated that older adult beliefs about ease of and usefulness of technology tend to predict acceptance of and intention to use this technology44,47,48. Ultimately, improving cognitive accessibility of health information through CA-based systems should increase older adults’ self-care by boosting their acceptance of the technology. Moreover, our findings related to message framing suggests that CA-based systems can engage older adults’ affective as well as cognitive facets of self-care. As a caveat our participants were exposed to brief messages and minimal interaction with the CAs. Further studies should explore the uses of interactive CAs (e.g., teach back strategies)45,46, for extended periods of time44. Some currently studies are exploring the use of speech-recognition for promoting interactivity49,50,51,52.

Figure 5.

Figure 5.

Open in a new tab

Interaction Effects from CA Age and participants’ age.

Figure 6.

Figure 6.

Open in a new tab

CA evaluation (API scores; left) and perceived similarity ratings (right) across levels of CA realism

Table 2.

Demographics (N=360)

Age Count (%)
< 30 83 (0.231)
31-40 90 (0.250)
41-50 30 (0.083)
51-60 52 (0.144)
61-70 87 (0.242)
> 71 18 (0.050)
Gender
Female 228 (0.633)
Male 132 (0.367)
Race/Ethnicity (multiple answers allowed)
Asian Indian 4
Asian or Pacific Islander 17
Black / African American 29
Latino/Hispanic 19
Native American 4
White 297
More than one race 3
Education
Less than high school 2 (0.006)
High school 44 (0.122)
Associate’s degree 60 (0.167)
Some college credit 69 (0.192)
Bachelor’s degree 132 (0.367)
Masters 37 (0.103)
Doctorate 16 (0.044)
Help to read health materials40 (self-reported)
Always 6 (0.017)
Often 17 (0.047)
Sometimes 65 (0.181)
Occasionally 60 (0.167)
Never 212 (0.589)
Open in a new tab

Table 3.

Proportions of correct propositions (memory recall) (N=360)

Participants Older CAs Younger CAs Total
Older 50.4% 52.0% 51.2%
Younger 44.5% 46.7% 45.6%
Total 46.4% 48.5% 47.5%
Open in a new tab

Acknowledgements

Research reported in this publication was approved by the Institutional Review Board at the University of Illinois at Urbana-Champaign and supported by the Jump Applied Research for Community Health through Engineering and Simulation (ARCHES) program, UIUC/OSF Hospital, Peoria IL. The content is solely the responsibility of the authors and does not necessarily represent the official views of these institutions.

References

  • 1.Centers for Disease Control and Prevention. Atlanta, GA: Centers for Disease Control and Prevention, US Dept of Health and Human Services; 2013. The State of Aging and Health in America 2013. [Google Scholar]
  • 2.Groessl EJ, Kaplan RM, Rejeski WJ, et al. Health-Related Quality of Life in Older Adults at Risk for Disability. American journal of preventive medicine. 2007;33(3):214–218. doi: 10.1016/j.amepre.2007.04.031. doi:10.1016/j.amepre.2007.04.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Fischer MA, Stedman MR, Lii J, Vogeli C, Shrank WH, Brookhart MA, et al. Primary Medication Non-Adherence: Analysis of 195,930 Electronic Prescriptions. J Gen Intern Med. 2010;25(4):284–90. doi: 10.1007/s11606-010-1253-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Brown MT, Bussell JK. Medication adherence: WHO cares? Mayo Clinic Proceedings. 2011;86(4):304–314. doi: 10.4065/mcp.2010.0575. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3068890/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Senst BL, Achusim LE, Genest RP, et al. Practical approach to determining costs and frequency of adverse drug events in a health care network. American Journal of Health-System Pharmacy. 2001;58(12):1126–1132. doi: 10.1093/ajhp/58.12.1126. [DOI] [PubMed] [Google Scholar]
  • 6.Reyna VF, Nelson W, Han P, et al. How numeracy influences risk comprehension and medical decision making. Psych Bulletin. 2009;135(6):943–973. doi: 10.1037/a0017327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Kessels RP. Patients’ memory for medical information. Journal of the Royal Society of Med. 2003;96(5):219–222. doi: 10.1258/jrsm.96.5.219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Black H, Gonzalez R, Priolo C, Schapira MM, Sonnad SS, Hanson CW, et al. True “meaningful use”: Technology meets both patient and provider needs. The American Journal of Managed Care. 2015;21:e329–337. [PubMed] [Google Scholar]
  • 9.Sharit J, Idrees T, Andrade AD, Anam R, Karanam C, Valencia W, et al. Journal of Health Psychology. 2016. Use of an online personal health record’s track health function to promote positive lifestyle behaviors in Veterans with prediabetes; pp. 1–10. doi:10.1177/1359105316681065. [DOI] [PubMed] [Google Scholar]
  • 10.Sharit J, Lisigurski M, Andrade AD, Karanam C, Nazi KM, Lewis JR, et al. The roles of health literacy, numeracy, and graph literacy on the usability of the VA’s personal health record by veterans. Journal of Usability Studies. 2014;9:173–193. [Google Scholar]
  • 11.Sakakini T, Azevedo RFL, Sadauskas V, Gu K, Zhang Y, Bhat S, Morrow D, et al. Dr. Babel Fish: A Machine Translator to Simplify Providers’ Language. Washington, DC: American Medical Informatics Association; 2017. In AMIA Annual Symposium Proceedings. [Google Scholar]
  • 12.Azevedo RFL, Gu K, Zhang Y, Sadauskas V, Sakakini T, Morrow D, Hasegawa-Johnson M, Huang TS, et al. Using computer agents to explain clinical test results. Washington, DC: American Medical Informatics Association; 2017. In AMIA Annual Symposium Proceedings. [Google Scholar]
  • 13.Morrow D, Hasegawa-Johnson M, Huang T, Schuh W, Azevedo RFL, Gu K, et al. A multidisciplinary approach to design and evaluating electronic medical record portal messages that support patient self-care. Journal of Biomedical Informatics. 2017;69:63–74. doi: 10.1016/j.jbi.2017.03.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Bickmore TW, Pfeifer LM, Byron D, Forsythe S, Henault LE, Jack BW, et al. Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials. J Health Commun. 2010;15(S2):197–210. doi: 10.1080/10810730.2010.499991. [DOI] [PubMed] [Google Scholar]
  • 15.Bickmore TW, Pfeifer LM, Paasche-Orlow MK. Using Computer Agents to Explain Medical Documents to Patients with Low Health Literacy. Patient Educ. Couns. 2009;75(3):315–320. doi: 10.1016/j.pec.2009.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Wolfe CR, Reyna VF, Widmer CL, Cedillos EM, Fisher CR, Brust-Renck PG, et al. Efficacy of a Web-Based Intelligent Tutoring System for Communicating Genetic Risk of Breast Cancer: A Fuzzy-Trace Theory Approach. Med. Decis. Making. 2015;35(1):46–59. doi: 10.1177/0272989X14535983. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Carstensen LL, Isaacowitz DM, Charles ST. Taking time seriously: A theory of socioemotional selectivity. American psychologist. 1999;54(3):165–181. doi: 10.1037//0003-066x.54.3.165. [DOI] [PubMed] [Google Scholar]
  • 18.Reed AE, Carstensen LL. The theory behind the age-related positivity effect. Frontiers in Psychology. 2012;3(339):1–9. doi: 10.3389/fpsyg.2012.00339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Peters E, Hess TM, Vastfjall D, Auman C. Adult age differences in dual information processes: Implications for the role of affective and deliberative processes in older adults’ decision making. Perspectives on Psychological Science. 2007;2(1):1–23. doi: 10.1111/j.1745-6916.2007.00025.x. [DOI] [PubMed] [Google Scholar]
  • 20.Mikels JA, Löckenhoff CE, Maglio SJ, Carstensen LL, Goldstein MK, Garber A. Following your heart or your head: Focusing on emotions versus information differentially influences the decisions of younger and older adults. Journal of Experimental Psychology:Applied. 2010;16(1):87–95. doi: 10.1037/a0018500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Hess TM, Rosenberg DC, Waters SJ. Motivation and representational processes in adulthood: The effects of social accountability and information relevance. Psychology and Aging. 2001;16:629–642. [PubMed] [Google Scholar]
  • 22.Baltes PB, Staudinger UM, Lindenberger U. Lifespan psychology: Theory and application to intellectual functioning. Annual review of psychology. 1999;50(1):471–507. doi: 10.1146/annurev.psych.50.1.471. [DOI] [PubMed] [Google Scholar]
  • 23.Grossmann I, Na J, Varnum ME, Park DC, Kitayama S, Nisbett RE. Reasoning about social conflicts improves into old age. Proceedings of the National Academy of Sciences. 2010;107(16):7246–7250. doi: 10.1073/pnas.1001715107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Beier ME, Ackerman PL. Age, ability and the role of prior knowledge on the acquisition of new domain knowledge. Psychology & Aging. 2005;20:341–355. doi: 10.1037/0882-7974.20.2.341. [DOI] [PubMed] [Google Scholar]
  • 25.Taha J, Czaja S, Sharit J, Morrow D. Factors affecting the usage of a Personal Health Record (PHR) to manage health. Psychol. Aging. 2013;28(4):1124–1139. doi: 10.1037/a0033911. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Reyna V. Communicating Risks and Benefits: An Evidence-based User’s Guide. 111-120. Silver Springs, MD: Federal Drug Administration; 2011. Across the life span. [Google Scholar]
  • 27.Ambady N, Koo J, Rosenthal R, Winograd C. Physical therapists’ nonverbal communication predicts geriatric patients’ health outcomes. Psychol. Aging. 2002;17(3):443–452. doi: 10.1037/0882-7974.17.3.443. [DOI] [PubMed] [Google Scholar]
  • 28.Kim S, Goldstein D, Hasher L, Zacks RT. Framing effects in younger and older adults. J Gerontol B Psychol Sci Soc Sci. 2005;60(4):215–218. doi: 10.1093/geronb/60.4.p215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Rothman AJ, Salovey P. Shaping perceptions to motivate healthy behavior: The role of message framing. Psychological Bulletin. 1997;121:3–19. doi: 10.1037/0033-2909.121.1.3. [DOI] [PubMed] [Google Scholar]
  • 30.Rothman AJ, Bartels RD, Wlaschin J, Salovey P. The strategic use of gain- and loss-framed messages to promote healthy behavior: How theory can inform practice. Journal of Communication. 2006;56:S202–S220. [Google Scholar]
  • 31.Garcia-Retamero R, Cokely ET. Communicating health risks with visual aids. Curr. Direct. Psychol. Sci. 2013;22(5):392–399. [Google Scholar]
  • 32.Schroeder NL, Adesope OO, Gilbert RB. How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research. 2013;49(1):1–39. [Google Scholar]
  • 33.Moreno R. Multimedia learning with animated pedagogical agents. In: Mayer R, editor. editor. The Cambridge Handbook of Multimedia Learning. New York: Cambridge University Press; 2005. pp. 507–523. [Google Scholar]
  • 34.Mori M. Energy. 4. Vol. 7. in Japanese: 1970. The Uncanny Valley; pp. 33–35. [Google Scholar]
  • 35.Crazy Talk 8 Pro [computer program] Version 8: Reallusion 2016. Retrieved from: https://www.reallusion.com/crazytalk/
  • 36.Ryu J, Baylor AL. The psychometric structure of pedagogical agent persona. Technol. Instruct. Cogn. Learn. (TICL) 2005;2:291–315. [Google Scholar]
  • 37.Zhou S, Bickmore T, Paasche-Orlow M, Jack B. Bickmore T, Marsella S, Sidner C. Springer, Heidelberg (2014): Agent-user concordance and satisfaction with a virtual hospital discharge nurse. IVA 2014, LNCS, 8637. [Google Scholar]
  • 38.Buhrmester M, Kwang T, Gosling SD. Amazon’s Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science. 2011;6(1):3–5. doi: 10.1177/1745691610393980. [DOI] [PubMed] [Google Scholar]
  • 39.Paolacci G, Chandler J, Ipeirotis PG. Running experiments on Amazon Mechanical Turk. Judgement and decision making. 2010;5(5) [Google Scholar]
  • 40.Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Family Medicine. 2004;36(8):588–94. [PubMed] [Google Scholar]
  • 41.Persky S, Kaphingst KA, Allen VC, Jr, Senay I. Effects of patient-provider race concordance and smoking status on lung cancer risk perception accuracy among African Americans. Ann Behav. Med. 2013;45(3):308–317. doi: 10.1007/s12160-013-9475-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Malik T, Azevedo RFL, Morrow D, Hasegawa-Johnson M, Huang TS, Schuh W, Gu K. Garcia-Retamero RAge Differences in Responding to Graphically Conveyed Health Risk Information. In: Proceedings from Human Factors and Ergonomics Society (HFES) International Annual Meeting; Austin, TX: 2017. [Google Scholar]
  • 43.Keppel G, Wickens TD. Pearson Prentice Hall: Upper Saddle River, N.J; 2004. Design and Analysis: A Researcher’s Handbook. [Google Scholar]
  • 44.Ring O, Shi L, Totzke K, Bickmore T. Social support agents for older adults: longitudinal affective computing in the home. J Multimodal User Interfaces. 2015;9(1):79–88. [Google Scholar]
  • 45.Nagarandeh R, Mahmoodi H, Noktehdan H, Heshmat R, Shakiasadeh E. Teach back and pictorial image educational strategies on knowledge about diabetes and medication/dietary adherence among low health literate patients with type 2 diabetes. Primary Care Diabetes. 2013;7:111–118. doi: 10.1016/j.pcd.2012.11.001. [DOI] [PubMed] [Google Scholar]
  • 46.White M, Garbez R, Carroll M, Brinker E, Jill Howie-Esquivel J. Is “Teach-Back” Associated With Knowledge Retention and Hospital Readmission in Hospitalized Heart Failure Patients? Journal of Cardiovascular Nursing. 2013;28(2):137–146. doi: 10.1097/JCN.0b013e31824987bd. [DOI] [PubMed] [Google Scholar]
  • 47.Czaja SJ, Zarcadoolas C, Vaughon WL, Lee CC, Rockoff ML, Levy J. The usability of electronic personal health record systems for an underserved adult population. Human Factors. 2015;57(3):491–506. doi: 10.1177/0018720814549238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Or CK, Karsh BT, Severtson DJ, Burke LJ, Brown RL, Brennan PF. Factors affecting home care patient’s acceptance of a web-based interactive self-management technology. JAMIA. 2011;18(1):51–59. doi: 10.1136/jamia.2010.007336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Beveridge M, Fox J. Automatic generation of spoken dialogue from medical plans and ontologies. Journal of Biomedical Informatics. 2006;39(5):482–499. doi: 10.1016/j.jbi.2005.12.008. [DOI] [PubMed] [Google Scholar]
  • 50.Bickmore T, Giorgino T. Health dialog systems for patient and consumers. Journal of Biomedical Informatics. 2006;39(5):556–571. doi: 10.1016/j.jbi.2005.12.004. [DOI] [PubMed] [Google Scholar]
  • 51.Campillos L, Bouamor D, Bilinski E, Ligozat A-L, Zweigenbaum P, Rosset S. Integrating a dialogue system into a virtual patient consultation. Chicago, IL: American Medical Informatics Association; 2016. In AMIA Annual Symposium Proceedings. [Google Scholar]
  • 52.Talbot TB, Sagae K, John B, Rizzo AA, Playa Vista CA. Designing useful virtual standardized patient encounters; In Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Proceedings.2012. [Google Scholar]
  • 53.Rosenberg-Kima RB, Baylor AL, Plant EA, Doerr CE. Interface agents as social models for female students: The effects of agent visual presence and appearance on female students’ attitudes and beliefs. Computers in Human Behavior. 2008;24(6):2741–2756. [Google Scholar]
  • 54.Yanghee K. The Role of Agent Age and Gender for Middle-Grade Girls. Computers in the Schools. 2016;33(2):59–70. [Google Scholar]
  • 55.Baylor AL, Kim Y. Lester JC, Vicari RM, Paraguacu F. Pedagogical agent design: the impact of agent realism, gender, ethnicity, and instructional role. Springer, Berlin: 2004. Intelligent Tutoring Systems (ITS 2004) [Google Scholar]
  • 56.Parker R, Graff D, Kong J, Chen K, Maeda K. Linguistic Data Consortium. 2011. English Gigaword. [Google Scholar]
  • 57.Humphreys BL, Lindberg DAB, Schoolman HM, Barnett GO. The Unified Medical Language System: an informatics research collaboration. J Am Med Inform Assoc. 1998;5:1–11. doi: 10.1136/jamia.1998.0050001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Lindberg DAB, Humphreys BL, McCray AT. The Unified Medical Language System. Methods Inform Med. 1993;32:281–91. doi: 10.1055/s-0038-1634945. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES