Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Aug 1.
Published in final edited form as: Am J Geriatr Psychiatry. 2020 Nov 22;29(8):804–815. doi: 10.1016/j.jagp.2020.11.004

Aging and Engaging: A Pilot Randomized Controlled Trial of an Online Conversational Skills Coach for Older Adults

Rafayet Ali 1, Ehsan Hoque 1, Paul Duberstein 2, Lenhart Schubert 1, Seyedeh Zahra Razavi 1, Benjamin Kane 1, Caroline Silva 3, Jennifer S Daks 4, Meghan Huang 4, Kim Van Orden 3
PMCID: PMC8140061  NIHMSID: NIHMS1649352  PMID: 33308893

Abstract

Objectives:

Communication difficulties negatively impact relationship quality and are associated with social isolation and loneliness in later life. There is a need for accessible communication interventions offered outside specialty mental health settings.

Design:

Pilot randomized controlled trial.

Setting:

Assessments in the laboratory and intervention completed in-home.

Participants:

20 adults age 60 and older from the community and a geriatric psychiatry clinic.

Intervention:

A web-based communication coach that provides automated feedback on eye contact, facial expressivity, speaking volume, and negative content (Aging and Engaging Program, AEP), delivered with minimal assistance in the home (8 brief sessions over 4–6 weeks) or control (education and videos on communication).

Measurements:

System Usability Scale and Social Skills Performance Assessment (SSPA), an observer-rated assessment of social communication elicited through standardized role-plays.

Results:

90% of participants completed all AEP sessions and the System Usability Scale score of 68 was above the cut-off for acceptable usability. Participants randomized to AEP demonstrated statistically and clinically significant improvement in eye contact and facial expressivity.

Conclusions:

The AEP is acceptable and feasible for older adults with communication difficulties to complete at home and may improve eye contact and facial expressivity, warranting a larger RCT to confirm efficacy and explore potential applications to other populations, including individuals with autism and social anxiety.

Keywords: social communication, technology, intervention


Older adults who lack high quality social relationships are at increased risk for poor quality of life,1 impaired physical and cognitive functioning,2,3 medical and psychiatric morbidity,4,5 and premature mortality.68 Effective social communication is essential for developing, maintaining, and improving relationships.914 Communication difficulties negatively impact relationship quality11,15,16 and are associated with social isolation and loneliness.1520 Social communication is included in the National Institute of Mental Health’s framework for transdiagnostic processes in mental disorders21—the Research Domain Criteria (RDoC)—and represents a treatment target for improving mental health and functioning.

Difficulties with social communication in later life can take many forms. Individuals experiencing depressive episodes often display overly inhibited communication, including restricted affect, limited eye contact, downward gaze, slow rate of speech, low speech volume, and restricted range of voice modulation.22,23 Communication difficulties also manifest as disinhibition, including difficulties with anger, speaking over others, and inappropriate content.20,24 Difficulties with communication may be attributable to mental disorders;18,2527 neurological conditions (e.g., Parkinson’s Disease);19 cognitive changes (e.g., executive functioning difficulties, memory problems);16 or sensory impairments.28 Novel situations (e.g., communicating with an oncologist);29,30 shifting family dynamics;29 and changes in how others communicate with an older person (e.g., ‘elderspeak’)31 can also pose communication challenges. This study includes older individuals with communication difficulties of varying etiologies to explore the potential acceptability of a novel computer-mediated intervention.

There are few evidence-based interventions to improve communication for older adults and those that are available most often target relationship skills such as assertiveness and conflict management with less emphasis on nonverbal behaviors that underlie such skills and serve to regulate social interactions.3236 Those interventions that do address nonverbal behaviors are resource-intensive and offered in specialty mental health settings, 3236 and changes in nonverbal communication are typically not assessed as an outcome, but are presumed to lead to changes in functioning.32,33 Only one prior study examined change in social communication in older adults as an outcome of a behavioral intervention, a skills training program for middle-aged and older adults with chronic psychotic disorders,37 that lead to improvements in communication. However, the content of this program was geared towards very basic skill development in adults with severe mental illness that cannot directly address the issue of how best to improve communication among older adults without severe mental illness who may be seeking to manage loneliness and isolation. In addition, there is a need for more accessible, less burdensome interventions offered outside specialty mental health settings.

Our team of computer scientists and geriatric mental health professionals developed a web-based communication coach for older adults—Aging and Engaging Program (AEP; Figure 1)—that can be delivered with minimal assistance in the home. To our knowledge, no such intervention has been developed for older adults. Prior studies on technology with older adults focused on providing companionship (e.g., robotic pets)38,39 or connecting older adults via video calls or online chat functions.40,41 The AEP uses a virtual agent, but is a communication skills coaching program, rather than a companionship program. The AEP is grounded in the science of artificial intelligence and utilizes affective computing and natural language processing to engage older adults in conversations, extracts features from users' video and audio in real-time, and gives automated feedback on eye contact, facial expressivity, speech volume, and degree of negative speech content. The AEP was adapted from a program called LISSA (Live Interactive Social Skills Assistance), which has been shown to be effective at improving nonverbal communication in college students.42 The AEP was adapted using a participatory design method with input from a neuropsychologist, geriatrician, geriatric psychiatrist, gerontologist with expertise in sensory impairment, a UX designer (i.e., user experience for technology), and a focus group of ten older adults to address challenges in designing an automated system for older adults, including providing feedback in easy-to-understand formats. The AEP was iteratively refined in a pilot study with 25 older adults who completed a single session with the AEP in a laboratory in order to develop and test the artificial intelligence that drives the program and obtain feedback on acceptability.43

Figure 1. The Aging & Engaging Program.

Figure 1.

Note. The user first initiates a conversation with the virtual agent. After conversing on a topic, the system uploads audio and video to a cloud server where features are extracted. Using a hidden Markov model classifier, the system then generates feedback. The user then sees the feedback items one-by-one with audio and visual support. The system then takes the user to the conversation interface to start the next topic. After conversing on three topics the system shows a final integrative feedback. Participants provided written informed consent for collecting video and audio data. Participants were given information about procedures to protect the privacy of their data, including data storage and limited access by only study personnel.

The current paper describes a pilot randomized controlled trial with adults age 60 and older who reported difficulties with social interactions. Participants were randomized to use the AEP eight times in their home over a period of 4–6 weeks (as it would ideally be deployed outside a research context) or to a control condition that involved reading information about improving conversation skills provided on the web with videos (with no feedback or engagement with AEP). All participants completed standardized role plays to assess verbal and nonverbal communication (via observational ratings) at baseline and follow-up (post-intervention). Our primary aim was feasibility and acceptability of the AEP: based on our prior study, we hypothesized that at least 75% of participants would complete at least 6 out of 8 assigned AEP sessions. We also examined self-reported usability. Our secondary aim was to examine whether we could detect a signal for efficacy of the AEP to warrant further study: we hypothesized that participants assigned to AEP would demonstrate improvement on verbal and nonverbal communication at follow-up, as measured by the Social Skills Performance Assessment (SSPA), an observer-rated assessment of social communication elicited through standardized role-plays.44,45 We used a randomized design at this early stage of intervention development and testing to ensure that our control condition was acceptable and feasible and to ensure that any signal for efficacy would be due to the AEP and not practice effects from the role play.

Methods

Participants.

We enrolled 20 adults age 60 or older from community advertisements and an outpatient geriatric psychiatry clinic who reported at least mild difficulties on the social skills and communication sub-scale of the Autistic-Spectrum Quotient.35 Exclusion criteria were diagnosis of dementia (self-report) and lack of an email address and/or access to the Internet in a private location (requirements of the AEP).

Procedures.

Participants visited the lab to complete initial questionnaires, the SSPA, and the first intervention session (to ensure ability to complete AEP/control at home). Participants completed up to seven additional self-directed sessions (AEP/control) over 4–6 weeks. Study staff sent emails with links to the program every three days and made phone calls and sent emails if participants did not complete sessions on time to address technical difficulties. Participants returned to the lab for an exit visit that included another role-play session. The study was approved by the Institutional Review Board at the University of Rochester and participants provided written informed consent, including consent to record video and audio data.

Interventions.

Participants access the AEP via their web browser; a webcam and microphone is needed. Due to the smaller screen size and limits with volume compared to desktop/laptop computers, we did not allow participants to use smartphones. The AEP requires minimal training. Study staff spent 5 to 10 minutes training participants on the AEP, including the role of the webcam and computer microphone to ensure they would be able to complete their sessions at home. The virtual agent initiates an open-ended conversation with the user, with topics that range in emotional intensity (e.g., weather, pets, retirement, life goals, growing older, spirituality). The virtual agent’s questions and responses were written by study team’s geropsychologists and piloted in a prior study.43 The original questions and response drafts were subsequently expanded and automated using dialog schemas, instantiated in the course of a dialogue via pattern transduction techniques for interpreting and responding to user inputs. The AEP records audio and video using the computer’s microphone and webcam and uploads data to a server in real-time. On the server, the facial and prosodic features are extracted from the audio and video files, including smile intensity, pitch, volume, eye gaze direction, and speech sentiment (Figure 1). Each session involves three brief conversations (2–3 minutes each), with automated feedback on eye contact, smile, speaking volume, and conversation content provided after each conversation followed by a final integrative feedback. For sessions 2–8, the system reminds participants what they talked about during the previous session, what type of positive and negative feedback they received, and what improvements they should focus on during the current conversation. To generate feedback, the AEP uses a hidden Markov model-based technique56. Additional details on the feedback generation process and dialogue are available46 and described in Appendix A.

Control participants received an email every three days containing a link to educational materials on the web to improve conversation skills. Our research group used these materials in previous studies testing similar programs for younger adults.13 Materials were provided online to better match the conditions between the two groups and control for time spent on the computer.

Measures

Participants completed self-report assessments to characterize the sample: PROMIS computerized adaptive tests for depressive and anxiety symptoms,40 social support,41 and self-efficacy to manage social situations;41 the World Health Organization Disability Schedule 2.0 (WHODAS);43 self-perceived health; UCLA Loneliness scale;47 and the Montreal Cognitive Assessment (MoCA).48 Participants assigned to AEP completed the System Usability Scale,49,50 for which a single total score is computed via summed responses from all participants and multiplied by 2.5 to convert the range to 0–100. A single score is computed using data from all participants: scores above 68 are considered to indicate ‘good’ or ‘excellent’ usability.49,50

Participants completed the Social Skills Performance Assessment (SSPA), an observer-rated assessment of social communication elicited through standardized role-plays,44,45 with minor modifications to scenarios to increase relevance to an older person attempting to reduce social isolation (i.e., introducing oneself to a group at a senior center and interviewing for a volunteer position). Participants interacted with a doctoral-level clinical psychologist who used standardized prompts and responses. Two independent raters used standardized instructions to rate seven dimensions of verbal communication (speaking volume, tone of voice, pitch, vocal clarity, pace, speech disturbances, and negative content) and nine dimensions of non-verbal communication (proximity to the other person, orientation to the other person, appearance, facial expressiveness, eye contact, posture, communicative gestures, unrelated hand movements, and length of time speaking). Each dimension was rated from 0 (no impairment) to 4 (severe impairment). One rater was a clinical psychology doctoral student observing the role play (in the room, sitting off to the side) who conducted the other study assessments (and was not blinded to condition) and a licensed clinical psychologist who participated in the role play (sitting across from the participant) who was blind to all information gathered in the assessment as well as treatment condition. We examined inter-rater reliability for the 15 communication behaviors: Kappa ranged from 0.00 (for pitch, orientation, and body position that were rarely impaired in participants) to 0.70 or higher (for volume, posture, and gestures that were more commonly impaired in participants). We held a consensus meeting with the raters and the study PI to review and resolve discrepancies through videotape review, which revealed that ratings for behaviors with lower kappas were impacted by differences between raters regarding their position in the room relative to the study participant and whether the rater was observing or participating in the role play. All discrepancies in ratings were resolved through consensus and the consensus ratings were used to compute two composite scores for each participant, verbal impairments (sum of ratings for verbal items) and nonverbal impairments (sum of ratings for nonverbal items).

Data Analytic Plan

The primary outcome was feasibility/acceptability as evidenced by the number of participants who completed at least 6 sessions as well as usability ratings for the AEP. The secondary outcome was communication skill on the SSPA, tested with two linear regression models (for verbal impairment and nonverbal impairment), with the condition (treatment/control) as the primary predictor and baseline scores on the SSPA as a covariate. We also provide two examples of participants’ performance on the SSPA and experience with the AEP to illustrate the program.

Results

Participant characteristics are shown in Table 1, including average age (71 years) and gender (68.4% female). Fewer than half reported being married (42.1%), and close to half reported living alone (47.4%). Participants demonstrated significant variability in severity of depression symptoms (T-score range 34.20–65.80) and anxiety symptoms (T-score range 32.90–65.40), indicating our sample includes individuals with moderate symptomatology. The average WHODAS score (8.87) corresponds to a population-normed 70th percentile (indicating significant disability). The average score on the MoCA score was 25.63, consistent with mild impairment in cognition. On the SSPA, participants demonstrated an average of 1.21 (std 1.03) impairments in verbal communication (out of 7 possible impairments) and 2.21 (std 1.78) impairments in nonverbal communication (out of 9 possible impairments, data presented here only). The most common impairments were facial expressivity (n=10), eye contact (n=8), and lack of gesture (n=12). Normative data are not available for the SSPA (as social communication is a relatively understudied contributor to mental health); however, the characteristics of our sample suggest at least mild difficulties with social communication that may be responsive to intervention, but that effects of the intervention may be limited by floor effects given that some behaviors were rarely rated as impaired.

Table 1.

Participant Characteristics at Baseline

Total Sample AEP Control
Age 71.47 (7.51) 70.70 (8.83) 72.33 (6.12)
Sex (female) 13 (68.4%) 8 (80%) 5 (56%)
Race (White)a 16 (84.2%) 8 (80%) 8 (89%)
Marriedb 8 (42.1%) 3 (30%) 3 (33%)
Living alone 9 (47.4%) 6 (60%) 3 (33%)
Education (college or greater) 15 (79%) 8 (80%) 7 (78%)
Cognitive performance (MOCA) 25.63 (2.56) 24.90 (3.00) 26.44 (1.81)
Depressive symptoms (PROMIS)c 49.57 (8.87) 48.35 (9.54) 50.78 (8.53)
Anxiety symptoms (PROMIS) c 50.18 (10.23) 46.98 (10.61) 53.39 (9.30)
Disability/functioning (WHODAS) 8.87 (7.98) 7.45 (8.10) 10.44 (8.01)
Berkman Social Integration 19.57 (5.14) 20.00 (4.69) 19.11 (5.84)
Social self-efficacy (PROMIS) c,d 45.71 (7.77) 46.35 (8.34) 45.00 (7.52)
Emotional support (PROMIS) c 50.33 (7.66) 52.85 (8.88) 49.37 (6.94)
Loneliness (UCLA) 43.59 (11.82) 43.00 (12.00) 44.11 (12.36)

Note: Values represent mean (std) for continuous variables or n (%) for binary variables.

a

1 participant identified as black, 1 as multiracial, and 1 declined to report on race.

b

7 participants reported being divorced, 1 legally separated, 3 widowed, and 0 as never married.

c

PROMIS scales are computerized adaptive tests with population normed T-scores with a mean of 50 and standard deviation of 10.

d

The full title for the ‘social self-efficacy’ scale is ‘self-efficacy to manage social interactions.’

Figure 2 depicts the CONSORT diagram, indicating 10 participants were randomized to AEP and 9 to control. 1 participant was not randomized because she reported not having access to email, a requirement for completing the intervention. 1 participant assigned to AEP withdrew from the study after the baseline visit and declined follow-up because she was not interested in the study. All remaining AEP participants completed the follow-up study visit and all completed the intervention, with only a single subject missing one AEP session. All control participants completed all sessions and study visits. Thus, 90% of participants completed the AEP sessions, consistent with acceptability and feasibility. Participants demonstrated capability to complete the programs in the home, supporting feasibility of this mode of intervention delivery.

Figure 2.

Figure 2.

CONSORT Diagram

The system usability score for the AEP was 69.5, which is above the validated cut point of 68 for acceptable usability.51 Usability scale items (Table 2) assessing ease of use, integration of system functions, and confidence using the system received the highest usability scores. The lowest usability score was for the item indicating that participants believed they would “need the support of a technical person to be able to use this program.”

Table 2.

Usability of the Aging & Engaging Program

System Usability Scale Item Mean Rating
I think that I would like to use this program frequently. 2.00 (1.83)
I found the program unnecessarily complex. 3.75 (0.50)
I thought the program was easy to use. 3.75 (0.50)
I think that I would need the support of a technical person to be able to use this program. 2.50 (1.00)
I found the various functions in this program were well integrated. 3.00 (0.82)
I thought there was too much inconsistency in this program. 3.50 (0.58)
I would imagine that most people would learn to use this program very quickly. 3.00 (0.82)
I found the program very cumbersome to use. 3.25 (0.50)
I felt very confident using the program. 3.50 (0.58)
I needed to learn a lot of things before I could get going with this program. 3.50 (0.58)
Overall, I would rate the user-friendliness of this program as: awful (1), poor (2), okay (3), good (4), excellent (5), or best imaginable (6). 4.17 (0.68)

Note: The System Usability Scale includes 10 items that assess several dimensions of usability (i.e., users perceptions of utility and user-friendliness of a program/technology) that are used to compute a total score as well as a final item assessing overall usability. Each item is rated on a 5-point Likert scale from strongly disagree (1) to strongly agree (5), with odd numbered questions worded to express positive attitudes, and even items worded to express negative attitudes. We followed suggested scoring procedures from the scale developer49,50 by subtracting 1 from responses to odd statements, and subtracting corresponding values from 5 in the even-numbered statements, yielding a converted response scale ranging from 0 to 4, with four being the most positive. The final item (Overall rating) is not used in calculating the total score.

Results for the secondary aim (signal for efficacy) appear in Table 3. Results indicate that participants randomized to AEP demonstrated significantly fewer impairments in nonverbal communication at follow-up compared to control, while results were non-significant for verbal impairment.

Table 3.

Effect of condition (AEP vs. control) on social communication impairments at follow-up (controlling for baseline impairments)

Outcome AEP Mean (Std) Control Mean (Std) b (SE) for condition effect P value
Baseline (n=10) Follow-up (n=9) Baseline (n=9) Follow-up (n=9)
Verbal communication 1.44 (1.33) 1.11 (1.17) 1.67 (1.32) 1.33 (1.41) 0.15 (0.59) .80
Nonverbal communication 2.11 (1.27) 1.00 (0.87) 2.89 (3.14) 2.44 (1.59) 1.30 (0.59) .04

Note: The composite scores (verbal, nonverbal) are scored such that higher numbers reflect greater skill impairments. Regression results include data for all participants who provided follow-up data (n=18) regardless of how many intervention sessions they completed.

Participant A is a 69-year-old divorced, retired, white female who lives alone. She reported difficulties with social anxiety and starting and maintaining conversations. She spoke softly, displayed little expression in her face, exhibited frequent pauses as she spoke, and avoided eye contact. One rater noted, “Very awkward, closed, stiff, quiet, and uncomfortable socially.” She completed all 8 AEP sessions. She received feedback to increase eye contact eight times (out of 24) but received positive feedback for eye contact on her final session, suggesting improvement with practice. She received feedback to increase speaking volume 10 times. When she began receiving positive feedback on volume, her responses were shorter, suggesting she may have had trouble maintaining louder speaking volume for longer periods of time. She received feedback to smile 16 times. She did not receive feedback on conversation content. At follow-up, she continued to speak softly and to pause frequently, but her face was more expressive, including more frequent smiling, and she made an appropriate amount of eye contact. She reported that the feedback was useful, accurate, and consistent with feedback she had received from others. She reported that she became “more aware of [her] communication habits,” the questions asked by the virtual agent were, “relatable and answerable,” and the program was “simple and straightforward” to use. She disliked that the virtual agent was “unnatural” in her responses and that the agent would interrupt her during the conversations. She suggested that our program could be improved by “having humans instead of computers because it may be more motivating.”

Participant B is an 84-year old, married, retired, white man with a doctorate in education. He reported difficulties making new friends and reported being dissatisfied with his relationships. He spoke quickly, for extended amounts of time, and spoke over his conversation partner. He perseverated on topics with negative content, demonstrated difficulty focusing, and did not ask the other person questions. One rater noted, he was “a bit impulsive in speech - cut off interviewer a few times, spoke over interviewer, spoke a bit fast, and talks for too long at times and needed to be cut off.” He completed all 8 AEP sessions. He received feedback to increase eye contact 20 times (out of 24). He received feedback to speak more positively 18 times (out of 24). He was somewhat negative about many topics. For example, on the topic of pets he said, “In my family, there are a number of cat lovers... I think cats are really dangerous but most people donť know it because they [cats] carry a little parasite and when they scratch you it enters in your blood and eventually sometimes gets to people's brain.” At follow-up, he continued to speak very quickly and for an extended period of time, to interrupt, and speak about negative content. His rating improved minimally on eye contact while his conversation content rating declined, and ratings of speaking volume, and facial expression remained the same. He reported that the feedback he received regarding negative content was “helpful and maps onto [his] personal experience.” Regarding the other aspects that the program suggested he could improve, he reported that he “could’ve used some convincing of what needed to be improved” in those domains and suggested that we could improve the program by including inperson feedback with a therapist the first time someone uses the program “to be able to have some discussion around it or be able to review the tape.” He reported that the program could be improved by including “more human contact.”

Discussion

This pilot RCT examined feasibility and acceptability of a web-based, automated communication coach. Results for our primary outcome indicate high acceptability, feasibility, and usability of the AEP, as well as acceptability of a control condition with online education. Results for our secondary outcome (SSPA) provide a signal that the AEP may improve some aspects of nonverbal communication, such as eye contact and facial expressivity, thus warranting further study. There are no comparable prior studies to serve as benchmarks for judging clinical significance of the effects we observed, as there are no evidence-based interventions for improving social communication in older adults without psychotic disorders. However, examination of individual variability in responsiveness to the AEP (i.e., case examples), suggest that the AEP may be most useful for individuals with an inhibited profile of communication characterized by shyness, poor eye contact, and low expressiveness—versus those with behavioral disinhibition—talking over others, speaking loudly, inappropriate negative comments. Subsequent research should, therefore, consider the type—and perhaps etiology—of communication impairment in selecting older adults for whom future research with the AEP is warranted.

Results indicate that the AEP may have helped participants with nonverbal behaviors, such as eye contact and facial expressivity, which our prior work has demonstrated may be two of the most malleable behaviors,52 while it was less successful in improving verbal communication, including speaking volume and negative speech content. Future research could examine strategies the program could offer to improve more challenging behaviors in addition to increasing awareness. For example, for the conversation topic, 'friends and family,' some participants discussed not being able to see their family (negative content). However, it is possible to discuss even challenging topics in a more positive manner, which may be especially effective when meeting new people. One participant who received feedback about negative content in a prior topic, described feeling disappointed that she does not have any grandchildren (negative content), but she then turned the conversation positive by making a joke that she is fortunate to have a 'grand-dog' living nearby that she enjoys taking for a walk. It may be useful to give participants examples such as this one so they can better understand how to shift negative statements to more positive ones that may increase positive connection with others. Regarding speech volume, it may be that the program could provide limited real-time (i.e., while the participant is speaking) feedback on volume (e.g., a flashing icon on the screen) to indicate when a participant successfully increased his/her speaking volume to increase awareness and provide stronger reinforcers for skilled behavior.

Our results should be placed in the context of several limitations. Effective behaviors are context dependent: behaviors can be effective in one context (e.g., a doctor's office) but less effective in another (e.g., meeting a new friend). Our program and assessment focused on communication in the context of increasing social connectedness through meeting new people, so results may not generalize to other contexts that could be appropriate for communication coaching. Further, individuals may have a harder time with communication in some settings versus others. Identifying contexts most appropriate for testing efficacy of the AEP and that are most relevant for practice is needed. The AI that drives the feedback module was developed with a primarily white, highly educated sample of older adults; thus, the algorithms will need to be tested in more diverse samples to ensure accuracy of the feedback. Assessing social communication is resource-intensive, thus our sample size was relatively small and also heterogenous with regards to commonly occurring comorbidities of communication difficulties, including psychopathology and functioning. Given promising results, future work should test the program with a larger, potentially more homogenous sample with regards to etiologies for communication difficulties. Third, we did not find improvements in verbal communication. These null results may be due to limited difficulties with these behaviors at baseline, but may also indicate a need to improve the program (discussed above). Another possible modification, suggested by our subjects, could be including a discussion with a human coach or therapist at the beginning of the program about what behaviors to improve and how to do so in order to increase motivation for using the program and maximize benefit. In this way, the AEP could offered in the context of mental health treatment, with the initial sessions supported by therapists or care managers and remaining sessions completed for “homework,” potentially supplemented with phone check-ins for technical support, encouragement, and suggestions to help participants maximally benefit. Finally, measuring social communication is resource intensive and there is scant data regarding norms or clinical significance,53 which limits the interpretability of findings at this early stage, but supports the need for continued study.

Improving social communication could have numerous mental health benefits in later life. For older adults who have difficulty communicating, it is challenging to maintain positive relationships and build new ones, which can lead to social isolation and declines in mental health. Some older adults may have struggled over their lifetime, while others may struggle in later life when confronted with new challenges (e.g., retirement, bereavement, moving to a senior living community). Some older adults may have strong communication skills in some domains, but difficulty in emotionally challenging situations, such as end-of-life discussions where both physicians and patients must maintain eye contact at certain moments to establish trust. Another pertinent example is Parkinson’s disease (PD), for which current social communication interventions involve face-to-face coaching on increasing expressivity.54 Offering web-based coaching would increase accessibility. Improving communication could also improve social connectedness and thereby reduce suicide risk, as medically serious suicide attempts among older adults are associated with difficulties communicating emotions.55 Problems with eye contact and facial expressivity may inhibit efforts to seek support and also make it more difficult for others to detect that the individual is in need of help.56

Our study has several strengths, including a focus on a relatively understudied contributor to mental health in later life—social communication—and development and testing of a novel AI-based intervention for improving social communication that can be delivered in the home with minimal training and interventionist support. Prior technological innovations for older adults focused on virtual companions and connecting online with peers. The AEP provides skill coaching online and is unique in that it directly targets specific communication behaviors and uses artificial intelligence to provide an automated yet flexible and engaging conversation experience in a practice environment that is private, accessible, and reduces stigma associated with seeking mental health care. The utility of in-home interventions such as the AEP is especially salient during the COVID pandemic due to the need for physical distancing, particularly for older adults who are most susceptible to severe illness. Future research is needed to determine the best setting for providing online communication coaching, including whether it should be ‘prescribed’ by mental health professionals with monitoring of engagement and progress, or offered through community-based programs, such as senior centers. These decisions must be considered in the context of guidance on the ethical provision of AI-driven conversational agents such as the AEP, to ensure that older adults receive the necessary support to promote safety, security of their data, and positive experiences with the technology.57,58 Our preliminary results suggest that addressing these challenges and continuing development of AEP is warranted and that online communication coaching holds promise as a tool for improving communication and thereby improving social function, mental health, and well-being in later life.

Supplementary Material

1

Highlights.

  • What is the primary question addressed by this study?—Is a self-directed, automated, web-based communication skills coaching program acceptable and feasible for older adults?

  • What is the main finding of this study?—The Aging and Engaging Program is acceptable and feasible for older adults with communication difficulties and may improve eye contact and facial expressivity.

  • What is the meaning of the finding?—A larger RCT of the AEP is warranted to confirm efficacy and explore potential applications to other populations, including individuals with autism and social anxiety.

Acknowledgments

This work was supported by funding from the National Center for Advancing Translational Sciences (NCATS) and the University of Rochester CTSI (UL1TR002001); the National Institute of Mental Health (NIMH; K23MH096936); NSF EAGER Grant IIS-1908595; and a University Research Award from the University of Rochester.

The authors would like to acknowledge the clinicians and researchers in our Advisory Group who assisted in the development of this work—Tom Caprio, MD, Mia Weber, PhD, Silvia Sörensen, PhD, and Yeates Conwell, MD—as well as Viet-Duy Nguyen, a research assistant in the Department of Computer Science.

Footnotes

Conflict of Interest Statement

None of the authors have conflicts of interest to disclose.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Golden J, Conroy RM, Bruce I,et al. Loneliness, social support networks, mood and wellbeing in community-dwelling elderly. Int J Geriatr Psychiatry. 2009;24(7):694–700. 10.1002/gps.2181 [DOI] [PubMed] [Google Scholar]
  • 2.Perissinotto CM, Stijacic Cenzer I, Covinsky KE. Loneliness in Older Persons: A Predictor of Functional Decline and Death. Archives of Internal Medicine. 2012:1–7. 10.1001/archinternmed.2012.1993 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wilson RS, Krueger KR, Arnold SE, et al. Loneliness and risk of Alzheimer disease. Arch Gen Psychiatry. 2007;64(2):234–240. 10.1001/archpsyc.64.2.234 [DOI] [PubMed] [Google Scholar]
  • 4.Tomaka J, Thompson S, Palacios R. The relation of social isolation, loneliness, and social support to disease outcomes among the elderly. J Aging Health. 2006;18(3):359–384.. 18/3/359 [pii] 10.1177/0898264305280993 [DOI] [PubMed] [Google Scholar]
  • 5.Cacioppo JT, Hawkley LC, Thisted RA. Perceived social isolation makes me sad: 5-year cross-lagged analyses of loneliness and depressive symptomatology in the Chicago Health, Aging, and Social Relations Study. Psychology and aging. 2010;25(2):453–463. 2010–11857-019 [pii] 10.1037/a0017216 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Luo Y, Hawkley LC, Waite LJ, Cacioppo JT. Loneliness, health, and mortality in old age: a national longitudinal study. Social Science & Medicine. 2012;74(6):907–914. 10.1016/j.socscimed.2011.11.0283303190. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Steptoe A, Shankar A, Demakakos P, Wardle J. Social isolation, loneliness, and all-cause mortality in older men and women. Proceedings of the National Academy of Sciences of the United States of America. 2013;110(15):5797–5801. 10.1073/pnas.12196861103625264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mortality risk: a meta-analytic review. PLoS Medicine. 2010;7(7):e1000316. 10.1371/journal.pmed.10003162910600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Burgoon JK, Guerrero LK, Manusov V. Nonverbal signals. In: Knapp ML, Daly JA, eds. The Sage Handbook of Interpersonal Communication. Fourth Edition ed. Thousand Oaks, California: Sage Publications; 2011:239–280. [Google Scholar]
  • 10.Burgoon JK, Bacue AE. Nonverbalcommunication skills. In:Greene JO, Burleson BR, eds. Handbook of Communication and Social Interaction Skills. New York: Routledge; 2003:179–219. [Google Scholar]
  • 11.Keating CF. The developmental arc of nonverbal communication:Capacity and consequence for human social bonds. APA handbook of nonverbal communication. 2016:103–138. http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc13&NEWS=N&AN=2015-02851-005. [Google Scholar]
  • 12.Mehrabian A, Wiener M. Decoding of inconsistent communications. J Pers Soc Psychol. 1967;6(1):109–114. https://www.ncbi.nlm.nih.gov/pubmed/6032751. [DOI] [PubMed] [Google Scholar]
  • 13.Mehrabian A, Ferris SR. Inference of attitudes from nonverbal communication in two channels. J Consult Psychol. 1967;31(3):248–252. https://www.ncbi.nlm.nih.gov/pubmed/6046577. [DOI] [PubMed] [Google Scholar]
  • 14.Depaulo BM. Nonverbal Behavior and Self-Presentation. Psychological Bulletin. 1992;111(2):203–243. Doi 10.1037/0033-2909.111.2.203 [DOI] [PubMed] [Google Scholar]
  • 15.Schwartz R, Pell MD. When emotion and expression diverge: The social costs of Parkinson's disease. J Clin Exp Neuropsychol. 2017;39(3):211–230. 10.1080/13803395.2016.1216090 [DOI] [PubMed] [Google Scholar]
  • 16.Struchen MA, Pappadis MR, Sander AM, Burrows CS, Myszka KA. Examining the contribution of social communication abilities and affective/behavioral functioning to social integration outcomes for adults with traumatic brain injury. J Head Trauma Rehabil. 2011;26(1):30–42. 10.1097/HTR.0b013e3182048f7c [DOI] [PubMed] [Google Scholar]
  • 17.Segrin C. Indirect Effects of Social Skills on Health Through Stress and Loneliness. Health Commun. 2019;34(1):118–124. 10.1080/10410236.2017.1384434 [DOI] [PubMed] [Google Scholar]
  • 18.Geurts HM, Stek M, Comijs H. Autism Characteristics in Older Adults with Depressive Disorders. The American journal of geriatric psychiatry: official journal of the American Association for Geriatric Psychiatry. 2015. 10.1016/j.jagp.2015.08.003 [DOI] [PubMed]
  • 19.Hemmesch AR. The detrimental effects of atypical nonverbal behavior on older adults' first impressions of individuals with Parkinson's disease. Psychology and aging. 2014;29(3):521–527. 10.1037/a0036637 PMC4176769. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Struchen MA, Clark AN, Sander AM, Mills MR, Evans G, Kurtz D. Relation of executive functioning and social communication measures to functional outcomes following traumatic brain injury. NeuroRehabilitation. 2008;23(2):185–198. https://www.ncbi.nlm.nih.gov/pubmed/18525140. [PubMed] [Google Scholar]
  • 21.Insel TR. The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. Am J Psychiatry. 2014;171(4):395–397. 10.1176/appi.ajp.2014.14020138 [DOI] [PubMed] [Google Scholar]
  • 22.Segrin C. Social skills deficits associated with depression. Clin Psychol Rev. 2000;20(3):379–403. http://www.ncbi.nlm.nih.gov/pubmed/10779900 http://ac.els-cdn.com/S0272735898001044/1-s2.0-S0272735898001044main.pdf?_tid=7c0a7236-6e90-11e4-897c00000aab0f6b&acdnat=1416252936_bb09f1882fbc7c1cbd145a7f25997598. [DOI] [PubMed] [Google Scholar]
  • 23.Hames JL, Hagan CR, Joiner TE. Interpersonal Processes in Depression. Annual Review of Clinical Psychology. 2013. 10.1146/annurev-clinpsy-050212-185553 [DOI] [PubMed] [Google Scholar]
  • 24.Sendroy-Terrill M, Whiteneck GG, Brooks CA. Aging with traumatic brain injury: cross-sectional follow-up of people receiving inpatient rehabilitation over more than 3 decades. Arch Phys Med Rehabil. 2010;91(3):489–497. 10.1016/j.apmr.2009.11.011 [DOI] [PubMed] [Google Scholar]
  • 25.Chou KL. Social anxiety disorder in older adults: evidence from the National Epidemiologic Survey on alcohol and related conditions. J Affect Disord. 2009;119(13):76–83. 10.1016/j.jad.2009.04.002 [DOI] [PubMed] [Google Scholar]
  • 26.Girard JM, Cohn JF, Mahoor MH, Mavadati SM, Hammal Z, Rosenwald DP. Nonverbal Social Withdrawal in Depression: Evidence from manual and automatic analysis. Image Vis Comput. 2014;32(10):641–647. 10.1016/j.imavis.2013.12.007 PMC4217695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Depp CA, Mausbach BT, Harvey PD, et al. Social competence and observer-rated social functioning in bipolar disorder. Bipolar disorders. 2010;12(8):843–850. 10.1111/j.13995618.2010.00880.x3639014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Feldman RS, Tyler JM. Factoring in Age: Nonverbal Communication Across the Life Span. The Sage handbook of nonverbal communication. 2006:191–199. http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc5&NEWS=N&AN=2006-11026-010. [Google Scholar]
  • 29.Manusov V, Keeley MP. When family talk is difficult:Making sense of nonverbal communication at the end-of-life. Journal of Family Communication. 2015;.15(4):pp. 10.1080/15267431.2015.1076424 [DOI] [Google Scholar]
  • 30.Wengryn MI, Hester EJ. Pragmatic skills used by older adults in social communication and health care contexts: Precursors to health literacy. Contemporary issues in commuication science and disorders. 2011;38:41–52. [Google Scholar]
  • 31.Kemper S, Harden T. Experimentally disentangling whaťs beneficial about elder speak from whaťs not. Psychology and aging. 1999;14(4):656–670. https://www.ncbi.nlm.nih.gov/pubmed/10632152. [DOI] [PubMed] [Google Scholar]
  • 32.Granholm E, Holden J, Link PC, McQuaid JR. Randomized clinical trial of cognitive behavioral social skills training for schizophrenia: improvement in functioning and experiential negative symptoms. J Consult Clin Psychol. 2014;82(6):1173–1185. 10.1037/a00370984244255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Fernandez-Ballesteros R, Izal M, Diaz P, Gonzalez JL, Souto E. Training of conversational skills with institutionalized elderly: a preliminary study. Perceptual and Motor Skills. 1988;66(3):923–926. 10.2466/pms.1988.66.3.923 [DOI] [PubMed] [Google Scholar]
  • 34.Praderas K, MacDonald ML. Telephone conversational skills training with socially isolated, impaired nursing home residents. J Appl Behav Anal. 1986;19(4):337–348. 10.1901/jaba.1986.19-337 PMC1308083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Thompson LW, Gallagher D, Nies G, Epstein D. Evaluation of the effectiveness of professionals and nonprofessionals as instructors of "coping with depression" classes for elders. Gerontologist. 1983;23(4):390–396. 10.1093/geront/23.4.390 [DOI] [PubMed] [Google Scholar]
  • 36.Foxx RM, McMorrow MJ, Schloss CN. Stacking the deck: teaching social skills to retarded adults with a modified table game. J Appl Behav Anal. 1983;16(2):157–170. 10.1901/jaba.1983.16-157 PMC1307873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Patterson TL, Mausbach BT, McKibbin C,Goldman S, Bucardo J, Jeste DV. Functional adaptation skills training (FAST): a randomized trial of a psychosocial intervention for middle-aged and older patients with chronic psychotic disorders. Schizophr Res. 2006;86(1–3):291–299. 10.1016/j.schres.2006.05.017 [DOI] [PubMed] [Google Scholar]
  • 38.Vardoulakis LP, Ring L,Barry B, Sidner CL, Bickmore T. Designing relational agents as long term social companions for older adults. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).2012.
  • 39.Ring L, Barry B, Totzke K, Bickmore T. Addressing loneliness and isolation in older adults: Proactive affective agents provide better support.. Proceedings - 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013. 10.1109/ACII.2013.17 [DOI] [Google Scholar]
  • 40.Garattini C, Wherton J, Prendergast D. Linking the lonely: An exploration of a communication technology designed to support social interaction among older adults. Univers Access Inf Soc 2012. 10.1007/s10209-011-0235-y [DOI]
  • 41.Czaja SJ, Boot WR, Charness N, Rogers WA, Sharit J. Improving Social Support for Older Adults Through Technology: Findings From the PRISM Randomized Controlled Trial. Gerontologist. 2017. 10.1093/geront/gnw249 [DOI] [PMC free article] [PubMed]
  • 42.Hoque M, Picard RW. Rich Nonverbal Sensing Technology for Automated Social Skills Training. Computer. 2014;47(4):28–35. <Go to ISI>://WOS:000335394400014. [Google Scholar]
  • 43.Ali MR, VanOrden KA, Parkhurst KA, al. e. Aging and engaging: A social conversational skills training program for older adults. International Conference on Intelligent User Interfaces, Proceedings IUI. 2018. 10.1145/3172944.3172958 [DOI]
  • 44.Patterson TL, Mausbach BT. Measurement of functional capacity: a new approach to understanding functional differences and real-world behavioral adaptation in those with mental illness. Annu Rev Clin Psychol. 2010;6:139–154. 10.1146/annurev.clinpsy.121208.131339 PMC3160788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Patterson TL, Moscona S, McKibbin CL, Davidson K, Jeste DV. Social skills performance assessment among older patients with schizophrenia. Schizophr Res. 2001;48(2–3):351–360. https://www.ncbi.nlm.nih.gov/pubmed/11295387. [DOI] [PubMed] [Google Scholar]
  • 46.Razavi SZ, Schubert LK, Van Orden KA, Ali MR. Discourse Behavior of Older Adults Interacting With a Dialogue Agent Competent in Multiple Topics. arXiv Prepr 2019. arXiv:1907.06279
  • 47.Russell DW. UCLA Loneliness Scale (Version3): Reliability, Validity, and Factor Structure. Journal of Personality Assessment. 1996;66(1):20–40. [DOI] [PubMed] [Google Scholar]
  • 48.Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society. 2005;53(4):695–699. 10.1111/j.1532-5415.2005.53221.x [DOI] [PubMed] [Google Scholar]
  • 49.Brooke J.SUS: A Retrospective. Journal of Usability Studies. 2013;8(2):29–40. [Google Scholar]
  • 50.Brooke J. System Usability Scale—A quick and dirty usability scale. In: Jordan PW, Thomas B, McLelland I, Weerdmeester BA, eds. Usability Evaluation in Industry. London: Taylor and Francis; 1996:189–194. [Google Scholar]
  • 51.Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies. 2009;4(3):114–123 [Google Scholar]
  • 52.Ali MR, Crasta D, Jin L, et al. LISSA — Live Interactive Social Skill Assistance. Paper presented at: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII); 21–24 Sept. 2015, 2015.
  • 53.National Advisory Mental Health Council. Behavioral Assessment Methods for RDoC Constructs August 2016: A Report by the National Advisory Mental Health Council Workgroup on Tasks and Measures for Research Domain Criteria (RDoC). Bethesda, MD: National Institutes of Health;2016. [Google Scholar]
  • 54.Dumer AI, Oster H, McCabe D, et al. Effects of the Lee Silverman Voice Treatment (LSVT(R) LOUD) on hypomimia in Parkinson's disease. Journal of the International Neuropsychological Society: JINS. 2014;20(3):302–312. 10.1017/S1355617714000046 [DOI] [PubMed] [Google Scholar]
  • 55.Szanto K, Dombrovski AYMD, Sahakian BJ, et al. Social Emotion Recognition, Social Functioning, and Attempted Suicide in Late-Life Depression. American Journal of Geriatric Psychiatry. 2012;20(3):257–265. 10.1097/JGP.0b013e31820eea0c [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Duberstein PR, Ma Y, Chapman BP, et al. Detection of depression in older adults by family and friends: distinguishing mood disorder signals from the noise of personality and everyday life. Int Psychogeriatr. 2011;23(4):634–643. 10.1017/S1041610210001808 PMC3032027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.McGreevey JD, 3rd, Hanson CW, 3rd, Koppel R. Clinical, Legal, and Ethical Aspectsof Artificial Intelligence-Assisted Conversational Agents in Health Care. JAMA. 2020. 10.1001/jama.2020.2724 [DOI] [PubMed] [Google Scholar]
  • 58.Matheny ME, Whicher D, Thadaney Israni S. Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. JAMA. 2019. 10.1001/jama.2019.21579 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES