Evaluating a negotiation training program for family caregivers of older people using a Multiphase Optimization Strategy (MOST) design and protocol

Abstract

Traditional clinical trial designs such as the isolated two-arm randomized controlled trial (RCT) do not offer robust solutions for evaluating and optimizing delivery of complex, multi-component behavioral interventions. A recent alternative design, the Multiphase Optimization Strategy (MOST), addresses many shortcomings of the isolated two-arm RCT. The MOST framework for trial design provides researchers opportunities to perform independent evaluations of intervention content, dosage levels, delivery formats, and potential intra-intervention interactions. Results from factorial trials which implement MOST frameworks are used to optimize ongoing interventions.

Herein, we describe the protocol for a MOST RCT which evaluates NegotiAge, an artificial intelligence-based negotiation and dispute resolution training program for family caregivers of older adults. Many family caregivers experience conflicts as they support older adult care recipients. Teaching negotiation skills to family caregivers has potential to improve communication and resolve conflicts more efficiently. The trial evaluation of NegotiAge eschews traditional two-arm RCT design and instead employs the MOST framework. Our MOST trial tests eight treatment combination packages against one another and evaluates associations between specific treatment combinations and user-centered outcomes.

This research is the first to apply the MOST framework in geriatrics and family caregiving. Our use of the MOST framework to evaluate and optimize NegotiAge enables us to identify which components are most effective for family caregivers and isolate the interactional effects of each component. The protocol and eventual results of our MOST trial will demonstrate how to optimize an intervention to be efficient and potent for busy family caregivers of older adults.

Trial registration ID

NCT04837937.

1. Background/aims

Over 44 million individuals in the United States are family caregivers-spouses, children, parents, or other relatives who provide unpaid support to older adults and adults with disabilities [[1], [2], [3]]. Most family caregivers are unprepared for the demands of family caregiving, and perform their responsibilities with little or no support [[4], [5], [6], [7], [8]]. Being a family caregiver is associated with declines in caregiver physical health, emotional well-being, and health-related quality of life, as well as perception of increased burden. Additionally, caregiver health and well-being impacts the emotional and physical health of care recipients [[9], [10], [11], [12], [13], [14], [15]].

The broad scope of challenges impacting family caregivers underscores the need to design accessible, effective, and efficient interventions to improve their lives. Family caregivers have limited time and resources which can pose barriers to accessing behavioral interventions. It is thus of critical importance that scaled interventions addressing family caregivers are optimized for efficiency. Traditional clinical trial designs such as the isolated two-arm Randomized Controlled Trial (RCT) do not offer robust solutions for evaluating or optimizing the delivery of complex, multi-component behavioral interventions. The Multiphase Optimization Strategy (MOST), a novel and flexible methodology for evaluating behavioral interventions, can better accommodate these challenges.

MOST is a three-phase evaluative framework, initially developed in systems engineering, which researchers have used to successfully optimize e-health interventions in a variety of domains (e.g. smoking cessation, weight loss, physical activity). With MOST, intervention components are first defined and pilot tested in a preparation phase; components are then deployed and evaluated in an optimization phase and an ideal package of components is identified; finally, that optimized package is evaluated for effectiveness in an “evaluation” phase. The overarching goal is to design the intervention with eventual implementation in mind, taking steps to assemble and evaluate a treatment package that balances effectiveness with affordability, scalability, and efficiency [16].

Here, we describe a trial protocol which leverages the MOST framework's capabilities to optimize an online, AI-powered negotiation training program, NegotiAge, for family caregivers of older adults living with Alzheimer's Dementia. NegotiAge is a proprietary tool which trains family caregivers in negotiation and dispute resolution techniques using interactive negotiations with AI-driven avatars simulating real-world scenarios (e.g. negotiations between the caregiver and the care recipient, between the caregiver and another family caregiver, between the caregiver and medical providers). The MOST framework allows us to evaluate these scenarios independently and in combination for their effect on caregiver positive affect and well-being and identify an optimal treatment package for future implementation.

2. Methods

Study Overview.

• 2.1 Intervention Design Overview

• 2.2 Preparation Phase, including:

  • 2.2.1 Identifying candidate intervention components
• 2.3 Optimization Phase, including:

  • 2.3.1 Factorial Trial Design
  • 2.3.2 Data management and analysis

2.1. Intervention design overview

NegotiAge represents the first two phases (preparation and optimization) of MOST, with the final phase (evaluation) to follow in a separate trial. The content of the intervention components that we selected is based on the theoretical model of Negotiation Dispute Resolution (NDR) and is tailored to reflect real-world negotiation scenarios that family caregivers of older adults encounter. We tested the selected intervention components in a feasibility pilot study. We used feedback from this pilot study to improve usability of the NegotiAge tool, concluding the preparation phase. We then conduct a factorial trial in the optimization phase. This factorial trial tests the effects of intervention components and their interactions on our outcome of interest (positive affect and wellbeing) to identify the active components to consider for inclusion in our final intervention package.

Our intervention adapts NDR training, a gold-standard negotiation training method from business management literature, to train family caregivers on successful dispute resolution. In brief, the model theorizes statements in a multi-party negotiation as “interests” statements, “rights” statements, or “power” statements. Interests statements address the individual interests of the negotiating parties, rights statements assert the rights of the negotiating parties, and power statements seek to establish a hierarchy of influence vs other negotiating parties. The NDR model posits that idealized negotiations will focus on addressing the individual interests of both parties first and foremost [17].

The NegotiAge tool is comprised of three elements which educate participants on negotiation and conflict training: videos, print materials, and the AI-based conflict negotiation platform. Throughout intervention, participants freely access instructional/educational materials on Negotiage.com. Video topics include teaching caregivers how to resolve conflicts using the interests-rights-power framework, tips on how to move between interests, rights, and power statements, and advice from a geriatrician on strategies for negotiating with older adults. Print materials include a document to help family caregivers plan for a negotiation by setting goals and identifying interests, rights, and power for each person involved in the negotiation.

After receiving the introductory training resources, family caregivers are provided with an AI-based avatar to practice negotiating through Interactive Arbitration Guide Online (IAGO). IAGO allows caregivers to negotiate in real-time with responsive avatars who are designed to act like humans. The avatars may employ emotionally manipulative tactics or engage in irrational behaviors, and their responses can change contextually based on user input. Our customized instance of IAGO features avatars and negotiation scenarios customized to be familiar to family caregivers of older adults. In our deployment of IAGO, users employ strategies from the training content to interact with the AI avatars. Participants send and receive offers, using strategies from the instructional material to negotiate with the AI avatar until either a mutual agreement is reached, or time expires. Immediate feedback is generated based on the results of the exercise for the user to review and use to improve their negotiation skills [18].

2.2. Preparation phase

2.2.1. Identifying candidate intervention components

To identify candidate intervention components, we collected qualitative information from a panel of 15 family caregivers to identify common conflict scenarios and conflict agents that family caregivers encounter as part of caregiving activity. Trained assessors performed thematic analysis on the qualitative information collected and extracted common negotiation scenarios/agents to test in the optimization phase. We also convened a community advisory board consisting of three community-based family caregivers in Florida, New York, and Illinois.

From the results of this thematic extraction, we developed four scenarios to test in the optimization phase: a universal/constant exercise and three additional “complex” negotiation scenarios. The universal/constant exercise simulates a negotiation with an older adult care recipient. The three additional exercises simulate negotiations with a “difficult” or less cooperative older adult care recipient, a physician, and another family caregiver, respectively. Each negotiating avatar (OA, Physician, Family Caregiver) has differing interests and levels of cooperation/receptivity to subject statements in the negotiation exercise, and so participants must identify each avatar's specific interests and choose responses which reflect an understanding of mutual and shared interests in order to achieve a successful negotiation (in the context of the NegotiAge tool, a successful negotiation is one where a participant makes an offer to the AI avatar which is accepted by the avatar).

The responses collected from family caregivers engaged in the preparation phase reinforced our existing belief that any intervention for this population must be either free or low-cost and must also be minimally invasive in terms of time and effort demanded. Our selection of the NegotiAge tool as an intervention reflects this philosophy: NegotiAge is free and easily accessible via web and mobile devices, and individual exercises can be completed in no more than 7 min. Because our tool has no cost to participants and because it is designed to be as time efficient as possible, we do not test time or cost as optimization objectives.

Following the conclusion of qualitative data collection and input from our community advisory board, we asked our 15 participants to perform usability testing on the NegotiAge tool in an three-round process. The first two rounds comprised collection of qualitative and quantitative data. Participants were asked to describe their reactions while using the NDR tool and proactively assisted in the identification of bugs/glitches/technical errors. The third, final cycle of feedback comprised a qualitative assessment of the.

NDR tool: the 15 participants navigated through the entire tool on their own, completed a brief online survey, and provided additional feedback via a semi-structured phone interview.

2.3. Optimization phase

2.3.1. Factorial trial design

Our trial is a randomized full factorial trial which compares the levels of the three experimental treatment components: the Difficult Older Adult negotiation, the Physician negotiation, and the Sister negotiation. We test 2 levels of each component: yes (the participant receives the intervention component) or no (the participant does not receive the intervention component). All participants receive the constant component, a universal “caregiver-patient” negotiation exercise. A table of treatment combinations is shown in Table 1.

Table 1.

MOST randomized controlled trial design, by experimental conditions.

Experimental Condition	Negotiation Activities, By Relationship Type Conflict
	Caregiver-Older Adult [Constant]	Caregiver – Sister	Caregiver-Physician	Caregiver- Older Adult*
1	Yes	No	No	No
2	Yes	No	No	Yes
3	Yes	No	Yes	No
4	Yes	No	Yes	Yes
5	Yes	Yes	No	No
6	Yes	Yes	No	Yes
7	Yes	Yes	Yes	No
8	Yes	Yes	Yes	Yes

The full factorial design allows us to separate the individual components as discrete effects and thus enables us to evaluate both the main effect of a given treatment and the interactions between components. This design is preferable to alternative strategies not only for the separation of components, but also for its efficiency. Factorial trials require fewer participants to achieve equivalent statistical power compared to individual trials of each component. The MOST factorial design thus allows us to both optimize delivery and minimize administrative costs, while producing equally robust statistical estimates at analysis time.

In traditional 2-arm randomized controlled trials, components assigned to individual conditions would be mutually exclusive, which precludes thorough investigation and optimization of multi-component treatment packages. In contrast, MOST designs such as our trial allow researchers to test all crossings of component effects and identify an optimal treatment package, rather than addressing treatment components in isolation. In our trial, the training and educational materials themselves do not vary-all participants receive the same training and have access to the same materials- but the negotiation scenarios do vary, although they each simulate real-world caregiver negotiations. The Interests-Rights-Power theoretical model which undergirds NegotiAge posits that successful negotiations first and foremost address the interests of the individual negotiating parties, although the interests of those parties vary from negotiation to negotiation. Though negotiating parties can and do differ in terms of their conception of individual interests, individual rights, and individual power within a given scenario, the identification of these domains and the application of the framework is generalizable between negotiations. In NegotiAge, the AI avatars in the assigned negotiation scenarios and their individual interests differ by scenario-that is, the interests of the “Older Adult” avatar differ from those of the “Sister” or the “Physician” avatars. Successful resolution of a given negotiation scenario thus depends, as it would in a “real world” scenario, on a participant's ability to identify and respond to the differing interests of differing interlocutors. By isolating these components and testing all combinations of component delivery, we can test and identify any component package which has an effect on our outcomes of interest.

Study Setting: This study received approval from the Northwestern University Institutional Review Board and is registered on clinicaltrials.gov (NCT04837937). Research staff secured signed, informed e-consent forms for each participant. Participants are recruited and take part in the intervention entirely via a remote, internet-delivered program. Recruitment took place nationally within the United States. Recruitment methods included the online ResearchMatch platform, partnerships with community health providers at two geriatric clinics affiliated with Northwestern, paper flyers, and word-of-mouth. While recruitment was not limited to the geriatric clinics, our existing linkages with these clinics suggested to us that there would be many potentially qualifying participants who act as family caregivers to patients of these clinics. All participants were screened for eligibility by study staff prior to consent. The eligibility screenings took place via short phone calls with a structured script and checklist for study staff. All data was collected and housed in the secure REDCap research data platform. Screening materials were not made available to participants ahead of time.

Eligibility Criteria:

• a.At least 21 years old;
• b.Ability to read and speak English;
• c.Currently provide care-giving support (e.g. emotional, social, physical, task-related) to an adult over the age of 65 (older adult);
• d.Provide care-giving support at least 1 h per week to the older adult;
• e.Involved in decision-making related to the healthcare and support of the older adult;
• f.Older adult has cognitive loss of 2 or greater on the AD8: Eight-item Informant Interview to Differentiate Aging and Dementia Test
• g.Internet and computer access; and
• h.Have a valid email address or the willingness to create one to access during the study.

Randomization: Participants are randomized using permuted block-randomization to one of eight possible treatment combinations. Each block is a list of eight randomly ordered conditions. Permuted block-randomization ensures that sample sizes are balanced across all eight conditions. The resulting number and type of negotiation scenarios received by participants is dictated by the randomization assignment associated with the unique link a participant receives. Researchers assessing the outcomes are blinded to the randomization assignments.

Intervention: Randomized participants receive the NDR intervention, consisting of an assigned minimum number and type of negotiation training exercises, based on the participant's randomization condition. All participants complete a tutorial session to become familiar with the web-based intervention. Once participants satisfactorily complete the tutorial, they are granted access to their assigned exercises and supplementary educational material. All participants then receive a “Universal” exercise to complete. Depending on randomization condition (as shown in Table 1), a participant can receive up to three additional assigned negotiations to complete. Individual negotiation exercises are limited to 7 min in length. Participants are required to complete their minimum assigned exercises but can complete additional exercises after completing their required assignment.

Outcome Measures: Measures collected by timepoint are displayed in Table 2. The primary outcome is Positive affect and well-being score of participants at one-month post-intervention, as measured with the Neuro-QoL Positive Affect and Well Being- Short Form panel. The outcome measure is converted to a t-score and paired t-tests are conducted to evaluate the change in score between baseline (T1) and one month (T3) post-intervention. We also evaluate a variety of secondary outcomes, including caregiver anxiety, burden, positive and negative affect, social satisfaction, emotional support, and general self-efficacy, assessed at T2, T3, and/or T4 as indicated in Table 2. Acceptability, usability, negotiation knowledge, negotiation utilization, and qualitative post-intervention reflections are also assessed at indicated timepoints. Qualitative data is coded by trained assessors to identify common themes in open-ended questions, although our protocol does not specifically iterate any qualitative primary or secondary outcomes.

Table 2.

Measures table, by timepoint.

Constructs	Measure	Timepoint
		T1	T2	T3	T4
Patient Centered Outcomes
Positive Affect and Well-being	Neuro-QoL – Positive Affect and Well-Being*	X		X	X
	*Primary Outcome
Anxiety	PROMIS- Anxiety Short Form-4	X		X	X
Caregiver Anxiety	Neuro-QoL Caregiver v2.0 – TBI-CareQoL Caregiver-Specific Anxiety – Short Form 6a	X		X	X
Caregiver Burden	Zarit Burden Interview	X		X	X
Fatigue	PROMIS - Fatigue - Short Form 7a	X		X	X
Self-Efficacy	PROMIS - General Self-Efficacy	X		X	X
Emotional Support	PROMIS - Emotional Support	X
Social Satisfaction	PROMIS - Satisfaction with Social Roles and Activities	X		X	X
Socio-demographics	Self-reported sociodemographic information	X
Intervention-Specific Outcomes
Acceptability	Usefulness, Satisfaction, and Ease of Use (USE)		X
Usability	System Usability Scale (SUS)		X
Negotiation Outcomes	Dutch Test for Conflict Handling (DUTCH)	X	X	X	X
	Negotiation Knowledge: Survey measuring caregiver knowledge of negotiation strategies	X	X	X	X
	Negotiation Utilization: Survey measuring utilization of study website, negotiation strategies			X	X
	Positive and Negative Affect Schedule (PANAS)	X	X	X	X
	IAGO Post-Assessment open-ended question re: negotiation success		X

Study timepoints: We assess participants at four time points in this trial. Eligible, consenting participants receive a pre-intervention survey at baseline (T1). Participants who complete the T1 survey are provided with a randomization hyperlink, which assigns each participant to their condition. Participants who complete their assigned exercises immediately receive a post-intervention assessment (T2). Participants are assessed again with online surveys at one month (T3) and three months (T4) following intervention. A study flow diagram is provided in Fig. 1.

Fig. 1.

Study flow diagram for NegotiAge MOST Factorial RCT.

Treatment Package Decision Making: Following the primary analysis, we will consider the inclusion of intervention components in the final treatment package based on our optimization objective of including only active components. To that end, we will identify components that significantly contribute to our main outcome first, followed by identification of any significant interaction effects that would influence our inclusion. For example, if two components both contribute as main effects, but when included together do not significantly improve our main outcome, we would not include both components in the same treatment package.

Sample size calculation: The goal of this research is to follow the MOST framework and optimize NegotiAge for a larger scale implementation RCT. We will achieve that goal by evaluating feasibility and generating effect size estimates of each component on changes in outcomes. The estimates inform which components to retain in the optimized intervention. With 120 subjects total and 40 subjects in each component group, we can detect effect sizes of 0.65 with 80 % power using a two-sided t-test at a type I error rate of 5 %. For the primary outcome of Neuro-QoL Positive Affect and Well-Being, this equates to a mean difference of 3.9, assuming a standard deviation of 6. We perform sample size calculations following Dahmen and Rochon 2004, implementing these methods in the SAS GEESIZE Macro v 3.1 [19].

2.3.2. Data management and analysis

Recruitment and post-randomization assessment data is collected using the REDCap data collection platform. Data is stored on a HIPAA-compliant server with secure access for approved study staff. Analyses are conducted in SAS 9.4. The main effects of each component and component interaction in Table 1 are estimated using Generalized Linear Mixed Models (GLMM) with PROC GLIMMIX in SAS 9.4. Data is effect-coded for analysis. Primary analyses are conducted using intent-to-treat GLMMs accounting for data collected at multiple assessment time points nested within individual participants. The GLMM structure accommodates missingness in outcome data without requiring the introduction of potential bias through imputation of missing values. For each component, we test differences in change in outcomes across time, with baseline values as the reference. Thus, effects are modeled as component × time interactions. We calculate Cohen's d by dividing the mixed effects model derived intervention effect estimate by the pooled standard deviation of the outcome. We include all randomized participants in the analysis, including those who only receive the universal exercise. Alpha levels for significance are set at 0.05.

Data monitoring: NegotiAge is non-invasive, so no stopping rules are in place to terminate the study. Subjects are free to withdraw if study conditions become unacceptable or untenable at any point. Data safety monitoring reports are produced at regular intervals and reviewed by the study data safety officer. The content of these reports is specified fully in the approved data safety monitoring plan, and the reports include participant study status, descriptive statistics, screen failure rates, safety information, protocol deviations, and study quality. Additional data fidelity checks are performed internally to clean and verify data, evaluate data capture quality, monitor study retention, and assess study activity completion at critical timepoints.

Confidentiality: Identification numbers are used to maintain participant confidentiality during data analysis. Multiple independent unique identifiers are used to de-identify participant data and link data between collection platforms when necessary.

Ancillary and post-trial care: We do not anticipate any harms or expected adverse outcomes resulting from the intervention, due to the intervention's nature as a training and education tool.

Dissemination policy: The results of this trial will be made available through international, peer-reviewed journals and conferences. No professional writers have been engaged in the creation of this manuscript. All authors listed above fulfill the authorship criteria as recommended by the ICMJE. The full protocol for this study, as well as de-identified data and/or statistical, will be made available upon reasonable request from the principal investigator(s).

3. Conclusion

We describe a protocol utilizing a novel research methodology, the MOST framework, to develop and optimize an efficient, scalable behavioral intervention leveraging AI to train family caregivers on negotiation strategies. Existing literature in geriatrics and in the field of family caregiving provides scant substantive background on how to design optimized interventions for this population and offers little suggestion on the selection of effective components for such interventions. Studies which use multi-component interventions rarely assess the efficacy of these components themselves or in interaction. This research is, to our knowledge, the first to apply the MOST framework in the field of geriatrics and family caregiving. It is our intent that by applying the MOST concept to our intervention, we will isolate any component-specific or interactional effect of our treatment package and then identify the optimal treatment package to deploy in a standalone evaluation trial.

CRediT authorship contribution statement

Charlie Olvera: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis, Data curation. Vanessa Ramirez-Zohfeld: Writing – review & editing, Visualization, Supervision, Project administration, Investigation, Formal analysis, Conceptualization. Alaine Murawski: Writing – review & editing, Validation, Project administration, Investigation, Formal analysis, Data curation, Conceptualization. Angela Fidler Pfammatter: Writing – review & editing, Supervision, Methodology, Funding acquisition, Formal analysis. Lee A. Lindquist: Writing – review & editing, Supervision, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.

Sponsor

The Northwestern University.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.