NegotiAge: Development and Pilot Testing of an Artificial Intelligence-based Family Caregiver Negotiation Program

Abstract

Background:

Family caregivers of people with Alzheimer’s disease experience conflicts as they navigate healthcare but lack training to resolve these disputes. We sought to develop and pilot test an artificial-intelligence negotiation training program, NegotiAge, for family caregivers.

Methods:

We convened negotiation experts, a geriatrician, a social worker, and community-based family caregivers. Content matter experts created short videos to teach negotiation skills. Caregivers generated dialogue surrounding conflicts. Computer scientists utilized the dialogue with the Interactive Arbitration Guide Online (IAGO) platform to develop avatar-based agents (e.g., sibling, older adult, physician) for caregivers to practice negotiating. Pilot testing was conducted with family caregivers to assess usability (USE) and satisfaction (open-ended questions with thematic analysis).

Results:

Development. With NegotiAge, caregivers progress through didactic material, then receive scenarios to negotiate (e.g., physician recommends gastric tube, sibling disagrees with home support, older adult refusing support). Caregivers negotiate in real-time with avatars who are designed to act like humans, including emotional tactics and irrational behaviors. Caregivers send/receive offers, using tactics until either mutual agreement or time expires. Immediate feedback is generated for the user to improve skills training. Pilot Testing. Family caregivers (n=12) completed the program and survey. USE questionnaire (Likert scale 1–7) subset scores revealed: 1.)Useful–Mean 5.69(SD0.76);2.)Ease–Mean5.24(SD0.96); 3.)Learn–Mean5.69(SD0.74);4.)Satisfy–Mean 5.62(SD1.10). Items that received over 80% agreements were: It helps me be more effective, It helps me be more productive, It is useful, It gives me more control over the activities in my life, It makes the things I want to accomplish easier to get done. Participants were highly satisfied and found NegotiAge fun to use (91.7%), with 100% who would recommend it to a friend.

Conclusion:

NegotiAge is an Artificial-Intelligent Caregiver Negotiation Program, that is usable and feasible for family caregivers to become familiar with negotiating conflicts commonly seen in healthcare.

Introduction:

Over 44 million family caregivers provide unpaid assistance and support to older people and adults with disabilities who are living in the community.^1–4 Family caregivers by definition include spouses, children, parents, relatives, or loved ones, who provide assistance that is above what is typical in that relationship.⁵ Evidence shows that most caregivers are ill-prepared for their role and provide care with little or no support.^6–10 Caregivers often experience burden from their perspective roles, which is associated with decline in caregivers’ physical health, mental health, and health-related quality of life.^11–14 Caregiver burden also has an effect on patient healthcare; increased burden has been shown to negatively impact patients’ emotional affect, interfere with how well caregivers assess patient pain and other symptoms, and influence long-term medical decision making for the patient.^15–18 Contributing to their burden, caregivers frequently have to interact with the healthcare system.^19,20

The need to negotiate care is recognized as a problem between families and older adult care recipients, health providers, and home and community-based service agencies.^21–23 Research shows that caregivers commonly play the role of “patient advocate” in negotiating the optimal healthcare of their loved ones.^24,25 Caregivers of older adults awaiting hospital discharge reported needing to intervene and “negotiate with the system” to ensure their loved ones were placed in an appropriate long term care facility and that their care needs were met.^26,27 A similar theme was echoed from caregivers of patients with mental illness who needed to fight and negotiate for access to services and learn how to make themselves heard by providers who tried to wave away their concerns.²⁸ Our research further supports this specifically on family caregiver conflicts with the healthcare system and the need to negotiate.²⁹ Moreover, family caregivers often lack formal training in conflict resolution.

Business schools have trained executives on negotiations and dispute resolution for over 50 years. Negotiation is an effective, proven method by which people settle differences through processes by which compromise, or agreement, is reached while avoiding argument and dispute.^30,31 Effective negotiators have the interpersonal skills to maintain a good working relationship with those involved in the negotiations. Extensive research has shown the effectiveness of Negotiation Dispute Resolution (NDR) training, specifically showing that lay people, trained in NDR, can persuade others without using manipulations, can maintain a positive atmosphere during a difficult negotiation, and can resolve conflicts successfully. NDR training provides individuals with the possibility to even the power dynamic.

With NDR training, caregivers can potentially learn skills to resolve disputes and negotiate agreements pertaining to their older adult. Prior research has shown that family caregivers are adept and willing to be trained in educational interventions.^32–34 As family caregivers are busy and unable to participate in-person in coursework, we sought to develop an online AI-based negotiation curriculum targeted to family caregivers of older adults.

Methods:

Convening a Panel for Development of the Negotiations Program.

To develop the negotiation program, we convened two panels: 1.) Family Caregiver Content Panel, which consisted of family caregivers (5), a social worker, nurses (2), and a geriatrician. 2.) Negotiation Expertise Content Panel, which consisted of business school negotiation professors (3), human-computer interface experts (3), and a geriatrician who taught negotiations in the medical school. Research staff attended each of the panel meetings.

Identifying the Context and Dialogue for Family Caregiver Conflicts

From our prior research, we elicited contexts (family caregiver – provider – older adult) where conflicts occur. To ensure completeness of contexts, we convened and queried a panel of family caregivers about the conflicts that they experience pertaining to their older adult care recipient (e.g., refusing assistance in the home, refusing gastric tube placement, sibling responsibilities and duties). We then asked the family caregiver panel to provide examples of real-world dialogue that they have encountered in conflicts (e.g., You can’t even keep the house clean – You need to hire someone to help you!” “You’re not sending me to a nursing home – I’ll write you out of the will!” “I’m the father, you’re the child. I don’t obey you!”). This generated an extensive set of contexts and dialogue where conflicts occur for family caregivers.

Completing the Negotiation Dialogue between Provider-Caregiver-Older Adult.

Building on the conflict scenarios and drafted dialogue, we coded each statement into stem and response, with each stem having multiple response choices. The dialogue stems were presented to our panelists, who were asked to provide feedback on how they perceive the statements (e.g., argumentative, refusals). The goal of each negotiation activity was to move the dialogue towards a pre-programmed agreeable option for both parties.

Framing of the Caregiver Negotiations Program.

Following recommendations from our business school professor consultants, we leveraged the Interest-Rights-Power (IRP) negotiation strategy which is commonly taught in business schools.

The Interest-Rights-Power framework is used to identify three types of statements made in dispute resolution negotiations. Interest statements are statements that show a person’s reasoning or position in a negotiation. They answer the why, why not question and are used to increase understanding of the dispute and find creative, respectful solutions to conflicts. Rights statements are arguments using standards of fairness, past practice, even law. They provide justifications for making concessions. Power statements are arguments that show a person’s authority over the other person in a negotiation. They often appear as threats and are used to try to get people to make concessions in order to avoid the threatened negative action. The goal of this framework is to identify when interest statements are made and move both parties of the negotiation towards interest statements, which is where most negotiation and conflict resolutions occur. The IRP framework is easy to remember and fits well with the family caregiver – older adult - provider relationships and the conflict they experience. Members of the research team coded the qualitative dialogue to fit the IRP framework (e.g., “I’m the boss” was coded as a power statement) and refined the dialogue responses to support the AI-based programming.³⁵

The AI-based Interactive Arbitration Guide Online (IAGO) Platform.

Since family caregivers are often stretched for time and may find it difficult to attend in-person courses, the negotiation training was created virtually using the Interactive Arbitration Guide Online (IAGO). IAGO is a platform for designing computer agents (AIs) that can negotiate with humans and is used to enable easy access to negotiation learning, activities, and remote negotiations with computer simulation.³⁶ With this negotiation-specific platform, we are able to build/adapt cases, assign negotiator roles, make pairings, and integrate surveys. IAGO supports a wide variety of features that have been shown to be critical to realistic human-agent negotiation. These include partial offers, preference elicitation statements, a customizable set of natural language argumentation phrases, and an expressive virtual human agent.³⁷ The interface also features a full conversational history, the ability to feature non-linear utility structures, and a customizable graphic interface. IAGO is also, notably, fully asynchronous. Humans or agents may both choose to take any action at any time and these decisions will be represented in real-time through the interface.³⁸ A core set of events is listed below:

1. Expression: allows for a user to detect, send, or receive an emotion (e.g., anger, surprise, happiness).

2. Message: allows users to send and receive natural language strings that will be displayed in the chat log.

3. Make Offer: the core of any negotiation, this allows a user to send or receive an offer (partial or full).

4. Timing Prompt: allows users to easily record periods of idleness from their partner, waking after a specified amount of time to take another action (such as reiterating a message, or relenting on a previous offer).

5. Feedback: allows users to obtain feedback instantly after completing the exercise.

For the IAGO platform, avatars were developed to reflect the participants (e.g. older adult, caregiver, provider) with whom family caregivers would negotiate. Multiple versions were created using real persons of different ethnicity/race.

IAGO AI-programming was based on the Interest-Rights-Power negotiation framework. The IAGO AI starts by considering its own options or utility for the issues at hand. For example, agreement to finding in-home care. The utilities are actually represented as “dis-utilities” as the issues are perceived at first as “bad” rather than “good”. This puts the AI-issues diametrically opposed to the interests of the caregiver. However, by focusing on the “interests” of the other party, IAGO revises its internal AI recognition of these utilities, adding in a factor based on others by gradually weighting the utilities of the issues to the other party. Other factors beyond dialogue choices in the IRP framework are weighted, including the use of repeated offers strategically, but all actions affecting this weighting factor of the other party’s utilities. Gradually, the positive utility of the other’s preferences comes to outweigh the inherent disutility, leading to agreement.

NegotiAge usability study design

Following the development of NegotiAge, we completed a usability study (n=12). Inclusion criteria for recruitment were: 1.) Individuals over the age of 21 years, 2.) Family caregivers of people aged 65 or older who have cognitive loss (2 or greater on the AD8: Eight-item Informant Interview to Differentiate Aging and Dementia Test)³⁹ for at least one hour per week, 3.) Have internet and computer access, 4.) Able to read and speak English 5.) Are involved in decision-making related to the healthcare and support of the adult over the age of 65. Participants were compensated for their time.

Usability is defined as the ease in which the users can interact with the technology.^40,41 The objective of this usability study was to assess whether family caregivers could appropriately use and interact with the NegotiAge curriculum from a functionality perspective.

Usability sessions were conducted online, via the Zoom video conferencing platform. Research staff obtained informed consent from the participant and completed a demographic survey with the participant. Participants were provided with the secured link to the NegotiAge online tool and asked to share their screen with research staff, minimizing the visual of the research staff member. Using a structured walk-through of the NegotiAge, research staff asked the participants to talk-aloud, or to verbalize their thoughts while using the tool and recorded how participants utilized the online tool, specifically noting where participants experienced barriers, questions, or challenges. Usability sessions were audio- and video-recorded for subsequent review. The Northwestern University Institutional Review Board approved this study.

Mixed methodology data were collected from participants following the session. Usability was determined using the System Usability Scale (SUS), where items are read (e.g., “It helps me be more effective.”) and participants respond with a 1–7 Likert Scale as follows: 1.Strongly Disagree, 2. Disagree, 3. Somewhat Disagree, 4. Neither Agree Nor Disagree, 5. Somewhat Agree, 6. Agree, 7. Strongly Agree. A higher score on an individual panel item indicates greater agreement with the provided statement. There are four sub-panels - Ease of Use, Usefulness, Satisfaction, and Learning. Sub-panels are aggregated and scored to produce a sub-panel average (arithmetic mean).^42,43 Open-ended questions were utilized to better delineate participant recommendations and targeted potential challenges. Deductive thematic analysis was conducted on the responses by two reviewers and a third as tie-break.

Results:

We created an Artificial Intelligence-based Caregiver Negotiation Program to facilitate teaching negotiation skills to caregivers of older adults with AD. (Figure 1) Since it is web-based and provides real-time feedback, busy caregivers can complete the negotiation program when their schedule permits. Negotiation basics resources (e.g., videos) last between 3–7 minutes and each IAGO exercise had a time limit of 7 minutes. Caregiver subjects could spend as much time as needed using the resources. Two waves of pilot testing with family caregivers of older adults were completed (n=12 in first wave, n=8 in second wave) with revisions completed following each wave. In response to the qualitative question: “Do you find this negotiation-training program easy to understand?”, the majority of responses were positive (e.g., Yes, easy to understand and would be easy for other caregivers. Overall, well made.) Thematic analysis is presented in Table 1. Results of the usability questionnaire (Likert scale 1–7), revealed the following aggregate subpanel means: 1.) Useful – Mean 5.69 (SD 0.76; Range 4.25–6.75); 2.) Ease – Mean 5.24 (SD 0.96; Range 3.0–6.55); 3.) Learn – Mean 5.69 (SD 0.74; Range 4.0–7.0); 4.) Satisfy – Mean 5.62 (SD 1.10; Range 3.0–7.0). Individual question responses are available in Table 2. Of note, specific items that received over 80% agreements were that the intervention helped caregiver subjects be more effective, more productive, have more control over the activities in their lives, and made things wanted to accomplish easier to get done. Participants were highly satisfied (n=11, 91%) and found the negotiation training program fun to use, with 100% (n=12) who would recommend it to a friend.

Figure 1:

Example Pages of NegotiAge

Table 1:

Thematic Analysis from Responses of Pilot Testing with Caregivers.

Question	Theme	Example Quote
What are the positive aspect(s) of this program?	Negotiation Skills	This let me think about real life scenarios and how to argue more effectively with others to reach an agreement that’s phrased in a mutual interest.
	Negotiation Training	The tool itself is easy and intuitive to use.
	Resources	The resources provided are helpful for planning how to broach a difficult topic with a sibling, physician, parent. The content is simplified and accessible.
	Enjoyable	Fun. it reinforced creativity.
What are the negative aspect(s) of this program?	Chat speed	Some of the responses came too fast. Things pop up fast, lots to see. I got better as I practiced using the program.
	Unclear Feedback	The feedback should be more detailed. It was a bit frustrating not understanding how to achieve the perfect score.
	Limitation	You are limited to the exercise topic. It is limited to a few possible responses when there could be limitless responses
	Responses	The characters can feel a bit hostile at times. Sometimes the other participant sends double responses which can quickly change in demeanor.
	Tutorial	The introductory instructions are relatively long. It needs to be laid out better.
What part(s) of the program were confusing?	Offers	It didn’t make sense to have this conversation and close it with an offer.
	Glitches	The glitches made it confusing… Sometimes it would jump back and forth.
	Responses	The more I played it the less confusing it got to be but I thought that responses that were available were just so canned and in the time of chat gpt if they could be more creative I suppose.
	Unclear Feedback	Feedback you get is confusing. Could not get full points. Would like more specific feedback on how to get full points.
Anything to add?	Helpful	No overall it was a great scenario and user friendly and I enjoyed it and helped me strategize.
	More scenarios	It was innovative and works in other parts of society. I’m just curious you could build something that would take more scenarios that were sent in by user community to expand it.
	Diversity	Perhaps people might reflect a different culture.
	Disseminate	This could be a great tool but my concern is how will you let people know it exists? Would be good to promote through Alzheimer’s groups. (12)

Table 2:

Usability Results from Pilot Testing with Caregiver Subjects

Variable	Somewhat Disagree/ Disagree/ Strongly Disagree	Neither Agree or Disagree	Somewhat Agree/ Agree/ Strongly Agree
Useful – Mean 5.69 (SD 0.76; Range 4.25–6.75)
It helps me be more effective	0	1 (8.3%)	11 (91.7%)
It helps me be more productive	0	2 (16.7%)	10 (83.4%)
It is useful	0	0	12 (100%)
It gives me more control over the activities in my life	0	2 (16.7%)	10 (83.4%)
It makes the things I want to accomplish easier to get done	0	1 (8.3%)	11 (91.7%)
It saves me time when I use it	0	5 (41.7%)	7 (58.3%)
It meets my needs	0	1 (8.3%)	11 (91.7%)
It does everything I would expect it to do	1 (8.3%)	1 (8.3%)	10 (83.4%)
Ease – Mean 5.24 (SD 0.96; Range 3.0–6.55)
It is easy to use	2 (16.7%)	0	10 (83.4%)
It is simple to use	2 (16.7%)	0	10 (83.4%)
It is user friendly	2 (16.7%)	0	10 (83.4%)
It requires the fewest steps possible to accomplish what I want to do with it	1 (8.3%)	0	11 (91.7%)
It is flexible	5 (41.7%)	4 (33.3%)	7 (58.3%)
Using it is effortless	4 (33.3%)	0	8 (66.6%)
I can use it without written instructions	9 (74.9%)	0	3 (25.0%)
I didn’t notice any inconsistencies as I use it	2 (16.7%)	1 (8.3%)	9 (74.9%)
Both occasional and regular users would like it	0	0	12 (100%)
I can recover from mistakes quickly and easily	1 (8.3%)	1 (8.3%)	10 (83.4%)
I can use it successfully every time	1 (8.3%)	1 (8.3%)	10 (83.4%)
Learn – Mean 5.69 (SD 0.74; Range 4.0–7.0)
I learned to use it quickly	1 (8.3%)	0	11 (91.7%)
I easily remember how to use it	0	0	12 (100%)
It is easy to learn to use it	2 (16.7%)	0	10 (83.4%)
I quickly became skillful with it	0	1 (8.3%)	11 (91.7%)
Satisfy – Mean 5.62 (SD 1.10; Range 3.0–7.0)
I am satisfied with it	3 (25.0%)	0	9 (74.9%)
I would recommend it to a friend	0	0	12 (100%)
It is fun to use	1 (8.3%)	0	11 (91.7%)
It works the way I want it to work	3 (25.0%)	0	9 (74.9%)
It is wonderful	1 (8.3%)	2 (16.67%)	9 (74.9%)
I feel I need to have it	2 (16.7%)	2 (16.67%)	8 (66.6%)
It is pleasant to use	1 (8.3%)	0	11 (91.7%)

Discussion:

NegotiAge is an artificial intelligence-based caregiver negotiation program that has been successfully developed through a panel of family caregivers, negotiation content experts, and human-computer interface experts. The goal of NegotiAge is to provide family caregivers with negotiation educational materials and real-world opportunities to practice how to negotiate in a safe artificial intelligent setting. A key tenant of learning how to negotiate is that more practice leads to better skill (e.g., the more familiar, the easier it becomes). With the AI-version, family caregivers can learn, relearn, and individualize practice on a flexible timeline, and as much or as little time dedicated to it as needed.

The results from our usability sessions suggest that family caregivers are capable, willing, and comfortable with using an AI-based negotiation online program, which can offer an entirely new approach for resolving conflicts. The User-centered design approach enabled participants to visualize the software application and provide feedback on the overall design flow of NegotiAge. Incorporating family caregivers, including as end-users, at all stages of the development process of the tool is critical to the development of AI-based technology. Eliciting feedback from family caregivers ensures that each aspect of NegotiAge meets their needs and improves the usability of the program. Subsequently, NegotiAge is a usable resource for family caregivers to learn how to resolve common conflicts between older adults with AD, providers, and other caregivers.

While still in the early stages of NegotiAge’s development there have been some limitations, participants encountered several “computer glitches” such as: 1.) Avatar emotions would not match the avatar responses; 2.) Avatars would deliver multiple responses or switch responses quickly; 3.) Avatar would occasionally freeze at the end of the negotiation. Troubleshooting occurred in follow-up renditions. Interestingly, since IAGO was built for general negotiation education, the negotiation terminology (e.g., offers) needed to be explained more. In addition, the speed of the negotiation was challenging for caregivers and the pace needed to be slowed down. Participants reported wanting more response options and more scenarios to practice. This is an important point. There is a need to determine the optimal amount of AI-based training and resources needed for family caregivers to reach negotiation competence. While some caregivers may want more practice, others may have limited time and may not be interested in spending additional hours online. To overcome this development limitation, we are planning a further Multi-phase Optimization Study (MOST). Another limitation is that currently, NegotiAge is only available in the English language, as our funding did not enable tailoring to alternate cultures. We anticipate future cultural adaptations if further trials show promise. For the time-being, we utilized a range of diverse avatars from different cultures. Although resources are available to print, the IAGO platform does not facilitate printing. Therefore, this negotiation program is only available to people who have access to computers. NegotiAge is also available in a mobile-smart phone version. As it is, we feel that the online accessibility may be a strength for future multi-site trial recruitment and subsequent dissemination to family caregivers nationally. Since this was a pilot usability study, there was no comparison to a control group and there are no measures of long-term effects on other caregiver measures, such as caregiver strain. We anticipate future studies to further explore these effects in a randomized controlled trial.

In conclusion, NegotiAge is an innovative AI-based online negotiation curriculum that was developed to support family caregivers. Our multi-disciplinary team provided expertise in family caregiving, social work, negotiations, artificial intelligence, human-computer interfaces, geriatrics, and research methodology. Pilot usability testing has shown that NegotiAge is a viable product with strong interest from family caregivers. We look forward to further optimization and testing of the efficacy of NegotiAge in our quest to improve the lives of family caregivers and their older adult loved ones.

Key Points:

• Family Caregivers encounter conflicts when working with and advocating for their older adult care recipients – which lead to unnecessary burden, stress, strain, and emotional duress. Although business schools teach negotiation tactics, family caregivers do not have the opportunity or time to take in-person coursework and may be unprepared to resolve geriatric disputes.

• With a team of business school negotiation experts, human-computer interface experts, geriatricians, family caregivers, and social worker, we developed an artificial intelligence (AI) based online program, NegotiAge.

• NegotiAge teaches family caregivers the basics of how to negotiate, then enables them to practice their negotiation skills with avatars providing real time responses, and feedback on how to improve their skills.

Why Does This Matter:

Family caregivers are unprepared to handle conflicts that occur when working with and advocating for their older adult loved ones. An online AI-based negotiation program, based on the theory of dispute resolution, was developed to teach family caregivers how to resolve disputes. In teaching caregivers how to negotiate, we hope to reduce their stress, burden, and emotional duress, while managing conflict thereby improving their abilities to care for their older adults.