Ease of Use, Feasibility, and Inter-rater Reliability of the Refined Cue Utilization and Engagement in Dementia (CUED) Mealtime Video-coding Scheme

Abstract

Aims:

To refine the Cue Utilization and Engagement in Dementia mealtime video-coding scheme and examine its ease of use, feasibility and inter-rater reliability in assessing the food intake process and dyadic verbal and nonverbal interactions.

Design:

This study was a secondary analysis of 110 videotaped observations of mealtime interactions collected under usual care conditions from a dementia communication trial during 2011–2014.

Methods:

The videos involved 29 staff and 25 residents with dementia (42 unique staff-resident dyads) in nine nursing homes. Data coding and analysis was performed in 2018–2019. Logs of coding challenges with matched solutions and coding time were collected. Inter-rater reliability was examined through rating of randomly selected 22 videos across four trained coders.

Results:

It took a mean of 10.81 hours to code a one-hour video using the refined coding scheme. Coding challenges, including identification of key intake process characteristics and differentiation of similar verbal or nonverbal behaviors, were identified with appropriate solutions. The refined coding scheme had good inter-rater reliability (Cohen’s Kappa range = 0.93 – 0.99, 95% CI = 0.92 – 0.99).

Conclusion:

Findings supported preliminary evidence on feasibility, usability and inter-rater reliability of the refined coding scheme. Future psychometric testing is needed in diverse populations with dementia across different care settings.

Impact:

Existing tools assessing the food intake process and dyadic interactions are few and have limited feasibility and/or reliability and fail to capture the complexity and dynamics of mealtime care. The refined coding scheme showed preliminary feasibility, usability and inter-rater reliability. In consideration of the balance between time intensity and the richness of data obtained, the tool may be appropriate and useful in addressing certain research inquires (e.g., characterizing and clustering dyadic behaviors, temporal relationship between behaviors and intake) pertaining older adults with or without dementia and their formal or informal caregivers.

INTRODUCTION

Nursing home (NH) residents with dementia commonly experience functional, cognitive and behavioral symptoms that inference with mealtime activities, leading to functional and nutritional consequences that decrease quality of life and increase morbidity and mortality (Bell et al., 2015; Chang & Roberts, 2011; Hanson et al., 2013). Among the personal and environmental factors that influence resident eating performance and intake (Liu et al., 2016; Liu et al., 2018; Liu, Jao, et al., 2019), quality of mealtime engagement and interactions between NH direct care staff (staff) and residents with dementia (residents) are important intervenable factors (Liu et al., 2017; Liu, Williams, et al., 2019).

Person-centered care is defined as a philosophy of care when individuals’ values and preferences are elicited and, once expressed, guide all aspects of their health care, supporting their realistic health and life goals (Care et al., 2016; Fazio et al., 2018). Person-centered care is built around the needs of the individual and contingent on knowing the person through an interpersonal dyadic relationship (Care et al., 2016; Fazio et al., 2018) and is highly recommended for mealtime. It challenges current mealtime care that primarily focus on eating tasks and staff preferences and features optimal mealtime care through knowing and acknowledging the individual’s capabilities and preferences, providing choices and supporting independence through positive and respectful interactions (Edvardsson et al., 2008; Lann-Wolcott et al., 2011; Liu et al., 2018; Reimer & Keller, 2009). As dementia progresses, residents rely on skilled NH staff who are knowledgeable and experienced in using person-centered care approaches to provide high quality mealtime care.

BACKGROUND

Review of Assessments for Staff Behaviors, Resident Behaviors and Staff-Resident (Dyadic) Behaviors during Mealtime

While the importance of quality dyadic mealtime interactions is acknowledged as part of person-centered mealtime care, few tools are available to measure the dynamic and complex dyadic interactions, limiting the development of effective interventions. There are many assessments for staff behaviors, resident behaviors and dyadic behaviors during mealtime care and four limitations may be considered in using them to assess dyadic mealtime interactions (Table 1).

Table 1.

Assessments for Staff Behaviors, Resident Behaviors, and Staff-Resident (Dyadic) Behaviors during Mealtime, and Related Limitations

Existing Assessments	Limitations
Staff Mealtime Behaviors
• Person-Centered Behavior Inventory (PCBI) (Coleman & Medvene, 2013) • Task-Centered Behavior Inventory (TCBI) (Lann-Wolcott et al., 2011) • Relational Behavioral Scale (RBS) (McGilton et al., 2012) • Mealtime Engagement Scale (Liu et al., 2019) • Feeding Skills Checklist (Batchelor-Murphy et al., 2015) • Mealtime Scan for Long-Term Care (Keller et al., 2017)	- The measures assess staff mealtime behaviors and/or social dining environment only - The measures have limited evidence of reliability and validity and warrants further testing - Some measures (PCBI, TCBI, RBS) were originally developed to evaluate staff behaviors during routine daily care activities, and then used in mealtime-care activities.
Resident Mealtime Behaviors
• The Level of Eating Independence Scale (Coyne & Hoskins, 1997) • Eating Behavior Scale (Tully, 1997) • Feeding Abilities Assessment (LeClerc et al., 2004) • Self-feeding Assessment Tool (Osborn & Marshall, 1993) • McGill Ingestive Skills Assessment (Lambert, 2003; Lambert et al., 2006) • Feeding Dependency Scale (Riviere et al., 2002) • Assessment of Eating Performance (Jacobsson et al., 1997; Jacobsson et al., 2000) • Eating Disabilities Assessment Scale (McLaren & Dickerson, 2000) • The Minimal-Eating Observation Form-version II (Westergren et al., 2009); • The Edinburgh Feeding Evaluation in Dementia (EdFED) scale (Liu et al., 2014; Watson, 1994) • Feeding Behaviors Inventory (Durnbaugh et al., 1996) • Aversive Feeding Behavior Inventory (Riviere et al., 2002) • Meal Assistance Screening tool (Brush et al., 2002) • The Chinese • Feeding Difficulty Index (Liu et al., 2015)	- The measures focused on assessing one aspect of resident behaviors, such as eating function or ability, and resistive or disruptive behaviors. - Most of the measures do not have adequate psychometric properties, except for the EdFED. - None of these measures capture an inclusive list of resident functional, cognitive, and behavioral symptoms that interfere with the intake process. - None of the measures assess resident positive/neutral behaviors that could occur during mealtime interaction.
Dyadic Mealtime Behaviors
• The Feeding Traceline Technique (FTLT) • (Phillips & Van Ort, 1993) • Self-Feeding Assessment tool for people with Dementia (SFD) and a Feeding Cycle Recording (FCR) sheet (Edahiro et al., 2012). • A behavioral coding scheme for Caregiver person-centeredness and resident agitation during mealtime interactions (Gilmore-Bykovskyi, 2015). • Mealtime Social Interaction Measure for Long-Term Care (Keller et al., 2013) assessing the impact of social dinning environment on interaction • Structured Meal Observation (Reed et al., 2005) assessing dyadic behaviors and environment considerations associated with intake • The Eating Behavior Observation Scale (Kline & Sexton, 1996) assessing resident dependency and staff physical touch and engagement	- FTLT breaks down the intake process into three simplified static segments (i.e., acceptance of food, preparation of next bite, and relaxation), failing to capture the complexity and dynamics of dyadic interaction process. - FLTL lacks feasibility in that one hour is needed to code a three-minute video (coding time: video length = 20:1). - SFD and FCR were developed to record the number of feeding cycles completed by staff and residents by direct meal observation assess eating independence, failing to capture the complex dyadic mealtime interaction. - In the behavioral coding scheme, codes for caregiver person-centeredness and resident agitation were not initially derived from mealtime care scenarios. - The behavioral coding scheme did not include codes for resident positive and negative verbal behaviors as well as positive nonverbal behaviors during mealtime. - All the tools capture limited characteristics of the complex dyadic mealtime interaction and warrant further psychometric testing.

First, most tools focus on mealtime behaviors of either staff or residents and fail to capture the interactive nature of dyadic mealtime care that involve engagement from both staff and residents. A few tools are developed to evaluate dyadic mealtime behaviors but have simplified the complexity and dynamics of the interactive process. Notably, there are a larger number of measures that assess resident behaviors compared with those that assess staff behaviors and dyadic behaviors during mealtime. The focus of mealtime care research has been resident behaviors for decades and recent research has also focused on measuring staff behaviors and dyadic behaviors.

Second, the measures that assess resident behaviors focus on either eating ability or resistive behaviors and fail to capture an inclusive list of resident functional, cognitive and behavioral symptoms in relation to the intake process. In addition, none of the measures assess resident positive/neutral behaviors that could occur during mealtime interactions and the intake process.

Third, most tools are developed for real-time on-site observations, resulting in combined total scores for use. Data obtained from this type of scoring, rather than second-by-second behavioral coding, does not allow sequential analysis to examine temporal relationships between behaviors and intake. Only a few tools are developed for videotaped observations, including Feeding Traceline Technique that captures limited dyadic behaviors in the intake process (Phillips & Van Ort, 1993) and the behavioral coding scheme for caregiver person-centeredness and resident agitation during mealtime that does not capture resident verbal behaviors and positive/neutral nonverbal behaviors (Gilmore-Bykovskyi, 2015).

Lastly, most of the measures have limited evidence of feasibility, reliability and/or validity, except for Mealtime Engagement Scale with preliminary reliability and validity to assess staff behavioral engagement level (Liu, Batchelor-Murphy, et al., 2019) and Edinburgh Feeding Evaluation in Dementia scale that has been extensively tested to assess resident resistive behaviors (Aselage, 2010; Roger Watson, 1994).

Needs of a Behavioral Video Coding Scheme for Dyadic Mealtime Interactions

The use of behavioral coding schemes to analyze videotaped observations has become an emerging and innovative methodology for assessing the complex and dynamic mealtime care interactions, as it allows for repeated viewing and coding of multiple factors, more precise measurement and deeper levels of analysis (e.g., sequential analysis) not achievable with direct observations (Gilmore-Bykovskyi et al., 2015; Riley & Manias, 2004). Currently, none of existing tools assess mealtime-specific behaviors of both staff and residents in relation to resident intake process using videotaped observations. A feasible, easy to use and reliable video coding scheme is needed to capture dyadic verbal and nonverbal behaviors in relation to the intake process specifically in the context of the complex, dynamic mealtime care interactions. Such a tool will facilitate understanding of characteristics of intake process and dyadic mealtime interactions, help address certain research inquiries including clustering of behaviors and temporal relationships among behaviors, as well as guide the development and evaluation of innovative person-centered mealtime care interventions to optimize function and nutrition in residents with dementia.

The Cue Utilization and Engagement in Dementia (CUED) Mealtime Video-Coding Scheme

The CUED coding scheme was developed to assess resident intake process and dyadic verbal and nonverbal behaviors based on multiple established observational tools. Specifically, codes for characteristics of resident intake process were developed based on Self-Feeding Assessment tool for people with Dementia (SFD) and a Feeding Cycle Recording (FCR) sheet (Edahiro et al., 2012). Each mealtime has one or more intake episodes, defined as the process of getting a bite of solid food or a drink of liquid food from the plate/tray/cup, putting it into the mouth and chewing and swallowing it (Liu, Williams, et al., 2019). Codes for staff and resident verbal behaviors include eight positive behaviors (e.g., “giving choices”) and four negative behaviors (e.g., “interrupting”). Codes for staff nonverbal behaviors include 10 positive behaviors in three categories (i.e., modifications of resident ability, care approaches and dining environment). Codes for resident nonverbal behaviors include 14 nonverbal behaviors in three categories (i.e., chewing/swallowing difficulties, functional impairment, resistive behaviors). These verbal and nonverbal behavioral codes were identified from three sources: 1) Person-Centered Behavior Inventory (Coleman & Medvene, 2013) and Task-Centered Behavior Inventory (Lann-Wolcott et al., 2011), 2) multiple established observational measures that assess eating and mealtime behaviors in residents with dementia (Aselage, 2010) and 3) an observational case study of staff and resident behaviors during mealtime.

The CUED was initially tested using 18 videotaped mealtime observations and showed good inter-rater reliability (r=.80) and evidence of feasibility in that 6 hours were needed to code a one-hour video (coding time: video length=6:1). While the CUED was developed as a comprehensive tool to assess resident intake process and dyadic verbal and nonverbal mealtime behaviors, it has three limitations: 1) it includes a limited number of nonverbal behaviors for both staff and residents; 2) it does not include staff negative nonverbal behaviors or resident positive/neutral nonverbal behaviors; and 3) it has not been psychometrically tested through second-by-second behavioral coding of videotaped observations. These limitations need to be addressed to capture the complexity of dyadic mealtime interactions in greater depth.

THE STUDY

Aims

The purpose of this study was to refine the CUED and examine its ease of use, feasibility and inter-rater reliability in assessing the food intake process and dyadic verbal and nonverbal interactions.

Refinement of the CUED

To address the limitations of the original CUED, we added additional nonverbal behaviors for staff and residents from multiple relevant sources: 1) a literature review of resident mealtime difficulties and targeted staff behavioral strategies (Rediehs. & Liu, 2019), 2) a review of measures that assess mealtime caregiving behaviors and dyadic mealtime interactions (Liu et al., 2020) and 3) a qualitative study that examined staff’s perspectives of multilevel barriers and facilitators to engaging residents in eating (Liu et al., 2018). In the refined CUED, the person-centered mealtime care philosophy guided the identification of additional nonverbal behaviors from staff and residents from both positive/neutral and negative perspectives.

Specifically, 24 new staff nonverbal behaviors were added, including 8 behaviors addressing modification of resident ability (e.g., positioning resident appropriately), 5 behaviors addressing modification of care approaches (e.g., appropriate use of affectionate touch), 3 behaviors addressing modification of the dining environment (e.g., arranging/mixing edible items for easy access) and 8 behaviors grouped in a newly developed category - negative behaviors (e.g., physically controlling). Also, 14 resident nonverbal behaviors were added, including 3 behaviors addressing functional impairment (e.g., difficulty transporting food to the mouth), 6 behaviors addressing resistive behaviors (e.g., biting the utensil when food is offered) and 5 behaviors grouped in a newly developed category - positive/neutral behaviors (e.g., wiping away oral spillage or drool). Definitions and coding examples for all added codes were developed using relevant sources as described above and incorporated into the standard coding manual of the refined CUED.

Testing of the Refined CUED

Design

This study was a secondary analysis using archived videotaped observations of mealtime interactions collected from a clinical trial conducted during 2011–2014 in Kansas, United States. The parent study evaluated the efficacy of a staff training program to improve communication and decrease resistiveness to care among residents with dementia (Williams et al., 2016).

Sample/Participants

In the parent study, residents who had a diagnosis of dementia, long stay status, staff-reported resistiveness to care and capacity to hear staff communication were enrolled. Staff who were 18 years or older, spoke English, worked as a permanent employee in the nursing home study site and provided direct care for a resident participant ≥ 2 times/week in the previous month were enrolled. Morning care sessions (e.g., mealtime, dressing, oral care) representing the most concentrated period of interactions between residents and staff were videotaped (Sloane et al., 2007).

In this study, baseline mealtime videos that captured one-on-one interactions, lasted 1 minute or longer (to ensure adequate details to code at least one intake episode) and had good quality with observable verbal and/or nonverbal behaviors were selected from the archived video inventory of the parent study. Videos that captured the resident taking medication, being transferred to or from the dining location, or presenting in the dining location but not eating the meal were excluded. Among the 1125 baseline videos that were screened, we excluded 974 videos that did not capture mealtime activities, 30 videos that lasted less than 1 minute, 10 videos that involved two staff and/or two residents and one video that was too dark to observe nonverbal behaviors, leaving 110 videos eligible for this study. The duration of the 110 videos (totaling 497.2 mins) varied from 1 minute to 23.8 minutes (mean=4.5, SD 4.0).

The 110 videos involved 42 unique staff-resident dyads in 9 NHs, including 25 residents with moderately severe to severe dementia and 29 staff. Residents had a mean age of 84.6 years old (range: 64–96). All residents were White, and most were female (60%) and non-Hispanic (92%). Staff had a mean age of 34.9 years old (range=19–79), worked as a caregiver for an average of 9 years (range=0.5–30) and worked at the current NH for 4 years (range=.5–13). Most staff were female (83%), non-Hispanic (79%), White (72%) and had or were receiving college education (72%). All staff were Certified Nursing Assistants (CNAs) and 14% had additional roles (e.g., activity assistant, medication or rehabilitation aide).

Instrument (The Refined CUED)

In the refined CUED (Table 2), codes for the resident’s intake process address four characteristics of an intake episode: 1) eating technique indicating the person who initiates and completes an episode (resident-completed, staff-facilitated); 2) the type of food being consumed (solid, liquid); 3) the duration; and 4) the outcome (intake, no intake). Mealtime duration is operationalized as the time period the resident is engaged in eating activities from the beginning of the first intake episode to the end of the last intake episode. Codes for staff and resident verbal behaviors include eight positive behaviors and four negative behaviors. Codes for staff nonverbal behaviors include eight negative behaviors and 26 positive behaviors grouped into three categories: 1) modification of resident ability represented by 12 behaviors; 2) modification of care approaches represented by seven behaviors; and 3) modification of dining environment represented by seven behaviors. Codes for resident nonverbal behaviors include five positive/neutral behaviors and 22 negative behaviors grouped into three categories: 1) chewing/swallowing difficulties represented by four behaviors; 2) functional impairment represented by six behaviors; and 3) resistive behaviors represented by 12 behaviors.

Table 2.

The Refined Cue Utilization and Engagement in Dementia (CUED) Mealtime Video Coding Scheme

I. Resident Intake Process	II. Caregiver & Resident Behaviors (Verbal)	III-1. Caregiver Behaviors (Nonverbal)	III-2. Resident Behaviors (Nonverbal)
A. Structure of Intake episodes 1. Eating technique (resident-completed, staff-facilitated) (Edahiro et al., 2012) 2. Type of food (solid, liquid) 3. Duration 4. Intake outcome (intake, no intake) B. Structure of Mealtime 1. Mealtime 2. No mealtime 3. Mealtime duration	A. Positive Behaviors 1. Asking for help/cooperation (Coleman & Medvene, 2013) - Question 2. Assessing for comfort/condition (Coleman & Medvene, 2013) - Question 3. Giving choices (Coleman & Medvene, 2013) - Question 4. Orientation/Giving Instructions (Coleman & Medvene, 2013) - Statement 5. Showing Approval/agreement (Coleman & Medvene, 2013) - Statement 6. Showing interest (Coleman & Medvene, 2013) 7. Gaining Attention Verbally (Coleman & Medvene, 2013) 8. Unsure – positive B. Negative Behaviors 1. Interrupting/changing topic (Lann-Wolcott et al., 2011) 2. Verbal refusal/Disagreement (Lann-Wolcott et al., 2011) 3. Controlling Voice (Lann-Wolcott et al., 2011) 4. Unsure – negative	A. Modification of Resident Functional/Cognitive Ability 1. Positioning resident appropriately* (Liu et al., 2019; Wen Liu et al., 2018) 2. Adjusting to resident’s pace (Coleman & Medvene, 2013) 3. Offering different type of food (Liu et al., 2019; Wen Liu et al., 2018) 4. Offering beverage (Liu et al., 2019; Wen Liu et al., 2018) 5. Offering finger food* (Liu et al., 2019; Wen Liu et al., 2018) 6. Offering condiments* (Liu et al., 2019; Wen Liu et al., 2018) 7. Putting item in container/utensil resident can manage (Liu et al., 2019; Wen Liu et al., 2018) 8. Giving a bite of appropriate size* (Liu et al., 2019; Wen Liu et al., 2018) 9. Guiding resident’s hand to pick up food* (Liu et al., 2019; Wen Liu et al., 2018) 10. Putting food/utensil into resident hand* (Liu et al., 2019; Wen Liu et al., 2018) 11. Holding resident hand to get food into the mouth* (Liu et al., 2019; Wen Liu et al., 2018) 12. Wiping away oral spillage or drool* (Liu et al., 2019; Wen Liu et al., 2018) 13. Other (describe the action in comment box) B. Modification of Care Approaches 1. Adjusting proximity (Coleman & Medvene, 2013) 2. Attempting to gain attention (Coleman & Medvene, 2013) 3. Positive gestures/facial expressions* (Coleman & Medvene, 2013) 4. Appropriate use of affectionate touch* (Coleman & Medvene, 2013) 5. Assessing comfort* (Coleman & Medvene, 2013; Liu et al., 2019; Wen Liu et al., 2018) 6. Affirmative nodding* (Coleman & Medvene, 2013) 7. Resident-directed eye gaze* (Coleman & Medvene, 2013) 8. Other (describe the action in comment box) C. Modification of Dining Environment 1. Modifying traffic (Liu et al. 2019; Wen Liu et al. 2018) 2. Modifying noise level (Liu et al. 2019; Wen Liu et al. 2018) 3. Reducing clutter (Liu et al. 2019; Wen Liu et al. 2018) 4. Limiting distractions (Liu et al. 2019; Wen Liu et al. 2018) 5. Arranging/mixing edible items for easy access* (Liu et al. 2019; Wen Liu et al. 2018) 6. Cutting food into manageable size* (Liu et al. 2019; Wen Liu et al. 2018) 7. Adjusting providing or taking away assistive devices/items (hearing aid glasses bib napkins etc) * (Liu et al. 2019; Wen Liu et al. 2018) 8. Other (describe the action in comment box) D. Negative Behaviors 1. Ignoring/lack of interactions* (Lann-Wolcott et al., 2011) 2. Physically controlling* (Lann-Wolcott et al., 2011) 3. Inappropriate touch* (Lann-Wolcott et al., 2011) 4. Outpacing* (Lann-Wolcott et al., 2011) 5. Discouraging/taking over resident self-eating attempt* (Liu et al., 2019; Wen Liu et al., 2018) 6. Mixing ALL food up* (Liu et al., 2019; Wen Liu et al., 2018) 7. Leaving the table/resident* (Liu et al., 2019; Liu et al., 2018) 8. Non-affirmative nodding* (Cipriani et al., 2016) 9. Other - negative (describe the action in comment box)	A. Chewing or Swallowing Difficulty 1. Leaving mouth open allowing food falls out of mouth (passive) (Watson, 1994) 2. Prolonged/continuous chewing/sipping (Cipriani et al., 2016) 3. Doesn’t chew/swallow (holds food in mouth) (Cipriani et al., 2016) 4. Coughing, choking, or gaging on food (Liu et al., 2015) 5. Other (describe the action in comment box) B. Functional Impairment 1. Difficulty using utensil properly, (Coyne & Hoskins, 1997; Tully, 1997) 2. difficulty transporting food to mouth* (Cipriani et al., 2016) 3. Contracture/Limited Range of Motion (ROM) (Cipriani et al., 2016) 4. Taking empty utensil/container to mouth* (Cipriani et al., 2016) 5. Spillage (Cipriani et al., 2016) 6. Drooling* (Watson, 1994) 7. Other (describe the action in comment box) C. Resistive Behaviors 1. Doesn’t open mouth (when mouth is empty) (Watson, 1994) 2. Biting the utensil (when food is offered)* (Liu et al., 2015) 3. Turning head away/tilts head backward (Liu et al., 2015; Watson, 1994) 4. Leaning backward* (Cipriani et al., 2016) 5. Pushing away help/food (Watson, 1994) 6. Spitting out food (Watson, 1994) (two modifiers)  • Doesn’t seem to like food/texture – (passive)  • Intentionally spitting food at the feeding assistant – (aggressive) 7. Non-affirmative nodding (Cipriani et al., 2016) 8. Disengaging from meal (four modifiers) (Cipriani et al. 2016; Liu et al. 2019; Liu et al. 2018)  • Closing eyes  • Distracted  • Falling asleep/becoming drowsy  • Other 9. Playing with food/utensil* (Liu et al., 2015) 10. Taking food from others* (Cipriani et al., 2016) 11. Attempting to eat inedible object* (Cipriani et al., 2016) 12. Attempting to leave the table* (Liu et al., 2015) 13. Other-resistive behavior (describe the action in comment box) D. Positive/Neutral Behaviors 1. Using hands to eat/taking over attempts to eat or drink* (Liu et al., 2015) 2. Leaning forward* (Cipriani et al., 2016) 3. Affirmative nodding* (Coleman & Medvene, 2013) 4. Wiping away oral spillage or drool* (Lann-Wolcott et al., 2011) 5. Indicating end of meal (waving hand no; taking off bib, etc) * (Lann-Wolcott et al., 2011) 6. Other-resident behavior (describe the action in comment box)

Note.

^*New codes added to the original version during the refinement of CUED.

Data Coding

Videos were coded using a three-step procedure. First, we coded characteristics of the resident intake process (Part I). Second, staff and resident verbal behaviors (Part II) were transcribed and coded. Third, staff and resident nonverbal behaviors (Part III) were coded. In each step, all videos were coded second-by-second using Noldus Observer® 14.0 (Noldus Information Technology Inc., Leesburg, VA, USA). Four research assistants were trained as coders by the first author through coding gold standard videos following a standard coding manual developed based on the refined CUED. After training, research assistants coded randomly selected videos from the study sample on their own and then met as a group with the first author to discuss coding challenges and came up with appropriate solutions. Multiple rounds of separate coding and group meetings were done to establish inter-rater reliability before trained coders independently coded the sample.

Data Collection

Ease of use of the refined CUED was evaluated by collecting data on coding challenges that coders encountered during the coding process along with appropriate solutions based on weekly discussion and agreement among team members. Feasibility of the refined CUED was evaluated by collecting data on time used by coders to complete the coding of each video. The inter-rater reliability of the refined CUED was established by having four trained research assistants code 22 randomly selected videos (20% of the 110 videos) on their own. Prior studies have used at least 10% of the study sample to establish inter-rater reliability of behavioral coding schemes using videotaped observations (Mahoney et al., 1999; Williams et al., 2018). Because our video sample was mostly video clips with varied durations rather than whole mealtime videos, we used 20% of the sample to establish inter-rater reliability.

Ethical Considerations

Ethical approvals were obtained through Institutional Review Boards of universities where the parent study and this study were conducted. In the parent study, staff were enrolled through written consent. Residents were enrolled through written assent and proxy consent from surrogate decision makers.

Data Analysis

Descriptive content analysis was used to categorize coding challenges and solutions based on similarities and differences (Sandelowski, 2010; Silverman, 2013). Time required for coding videos was analyzed using descriptive statistics in SPSS 25.0 (SPSS, Chicago, IL). Inter-rater reliability was assessed using percent agreement and Cohen’s Kappa (± 1s tolerance) (McHugh, 2012) by comparing all the codes in each of the 22 selected videos across coders using both frequency/sequence and duration/sequence comparison methods in Noldus Observer® 14.0. When estimating inter-rater reliability, both the type and timing of the coded behavior were compared across coders in Noldus Observer® 14.0. A tolerance window of ± 1 second was used in this study, indicating that the timing of coded behavior across coders should be within 2-second difference to be considered consistent. Adequate inter-rater reliability is indicated by percent agreement ≥ 90% and Cohen’s Kappa ≥ 0.80 (McHugh, 2012).

RESULTS/FINDINGS

Ease of Use

For Part I (intake process), four coding challenges were identified with matched solutions (Table 3): 1) when exactly an intake episode began and ended was based on the movement of the utensil or hand; 2) when the staff and resident were both involved, whoever dominated the eating or drinking movement was coded as the person who initiated/completed the episode; 3) observation of complete details was needed to code an intake episode; and 4) when an intake episode involved the movement of an empty utensil, or food fell out of the utensil or the resident’s mouth, no intake was coded.

Table 3.

Major Coding Challenges and Solutions for Part I (intake process)

Challenges	Solutions	No. of videos with the issue
1. When exactly does an intake episode start or end?	An intake episode should begin with the longest pause of the utensil or hand before intake, that is, code the starting point of an intake episode as soon as the longest pause ends (whether utensil or hand is resting on plate or in the air). An intake episode should end when utensil or hand was completely removed from the resident mouth area, unless it stays on or within millimeters away from the mouth while the resident is attempting to get food in. When the resident sip from cup or straw for liquid, code each sipping act as an intake episode. In the case that there is a long period of sipping where the cup or straw is barely moved away from the mouth, code this sipping act as one intake episode.	12
2. The view of the subjects was sometimes blocked due to camera movement or people being in the way	Code if enough details for a complete intake episode (starting point, ending point, who initiates, type of food involved, intake or not) can be observed. Can’t guess what is going on behind blocked camera if the movement of hand or utensil is not observed.	11
3. Video footage begins or ends while an intake episode is ongoing.	If the video begins with the resident chewing or food is already in the mouth, no episode will be coded. If the video begins when food is moved toward or placed right outside the mouth, and an accurate starting point of an episode is observed, one episode is coded. If the video ends with the resident chewing or food is already in the mouth, code as one episode; if the video ends when the food is moved toward the mouth and still outside the mouth, no episode is coded.	10
4. When the staff and resident were both involved in an intake episode, who should be coded as the subject initiating/completing the episode?	When the staff and resident are both involved in an episode, whoever dominates the eating or drinking movement or makes the attempt successful should be coded as the subject. For example, if the staff is the one who puts the food in the resident’s mouth, or holds the cup for the resident to drink (whether the resident holds the cup/straw or not), it is the staff who makes the intake episode happen and successful, and the staff is coded as the subject initiating/completing the episode.	9
5. The resident brings his/her hand or utensil to his/her mouth as if she/he is eating, but no food is involved.	Code the movement of an empty hand or utensil as an intake episode with no intake.	6
6. Food doesn’t make it completely to residents’ mouth, falls out of the utensil, or is spit out by the resident.	If the resident brings food to mouth and doesn’t make it to the mouth or food falls out, code as an intake episode with no intake.	6

For Part II (verbal behaviors), three transcription challenges were identified with matched solutions (Table 4): 1) when personal identifiers were mentioned in videos, they were replaced with non-identifiable participant IDs and site IDs; 2) when transcription of multiple sentences were put in one line, the transcription were split into separate lines with only one full sentence in each line so as to assign a separate code for each full sentence; and 3) indistinguishable or unclear verbal utterance was literally transcribed as “unsure”.

Table 4.

Major Transcription and Coding Challenges and Solutions for Part II (Verbal behaviors)

Transcription Challenges	Solutions	No. of videos with the issue
1. Personal identifiers are mentioned in the video	Personal identifiers (i.e., person’s real name, name of NH) usually expose who someone is or where he/she lives or works. These identifiers are replaced with the staff or resident participant’s ID or the study site ID that was created for the study.	20
2. Wrong transcription format	Each transcription is one full sentence with a period (.) or question mark (?), taking up one line in the coding screen in the Noldus software, and is assigned with one code. Sometimes transcription of multiple sentences was put in one line. In this case, the transcription needs to be split into separate lines following the rule above, in order to have a separate code for each full sentence.	18
3. Transcription starting point and ending point	Verbal utterances of the whole video from the beginning to the end of the video footage should be transcribed.	5
4. Could not hear clearly what the subject was saying	Transcribe as “unsure” literally.	4
Coding Challenges
1. Clarification between “giving choices” vs. “assessing comfort/condition”	For a question-related verbal utterance to be coded as “giving choices”, a specific item such as a bite of certain food, a sip of certain liquid, or other meal-related items (e.g., utensils, napkins) must be involved. Additionally, giving the resident multiple options to choose from is also coded as “giving choices”. When the staff repeated the resident’s request for clarification purpose, code as “assessing comfort/condition”.	37
2. Distinguishin g between codes for meal-related and non-meal-related verbal utterances	Both meal-related and non-meal-related verbal utterances should be coded following the coding scheme. It is possible that general conversation that is non-meal-related are mostly coded as “showing interest” (e.g., laughter, small-talk), when other codes are not appropriate. In many cases, conversations contain specific item or substance that make the verbal utterances qualified for other more descriptive codes. It is recommended to always attempt to find the most descriptive code for a verbal utterance.	31
3. Clarification between “showing approval/agreement “ vs. “showing interest”.	The verbal utterance is coded as “showing approval/agreement” if the subject is encouraging, praising, or agreeing with the person he/she is conversing with. Often there is a friendly general utterance that keeps the conversation going back and forth, but does not warrant a “showing approval/agreement” code, code as “showing interest” in this case.	27
4. Clarification between “asking for help/cooperation”	The first distinction is often made upon whether there is a question mark at the end of the transcription or not (whether the verbal utterance is a question or a statement). If the verbal vs. “assessing for comfort/condition” vs. “showing interest” utterance is a question, attempting to elicit an action, code as “asking for help/cooperation”, rather than “assessing for comfort/condition”. If the verbal utterance is a question, attempting to check on the subject’s status, code as “assessing for comfort/condition”. A few exceptions existed. If the verbal utterance is a question, but the subject is asking for help to understand something or to solicit more information related to something or someone, code as “showing interest”.	25
5. The use of “unsure” codes	The codes “unsure - positive” and “unsure - negative” are mostly used when it is unclear what the subject is saying, with few exceptions. Meaningful verbal utterances are usually assigned to more descriptive codes if possible, with the two “unsure” codes being the least descriptive options.	20
6. Clarification between “orientation/giving instructions” vs. “controlling voice” vs. “verbal refusal”	When giving instructions, if the staff uses a voice that clearly sounds harsh or negative, code as “controlling voice”, rather than “orientation/giving instructions” based on tone of voice. However, the staff or resident participant can disagree or speak with a strong and loud tone throughout a conversation without warranting the coding of “verbal refusal” or “controlling voice”.	18

For Part II, six coding challenges were identified with agreed solutions (Table 3): 1) the clarification between “giving choices” vs. “assessing comfort/condition” was based on whether the staff was giving the resident one or more options to choose from (“giving choices”), or the staff was repeating the resident’s request for clarification (“assessing comfort/condition”); 2) both meal-related and non-meal-related utterances were coded; 3) the clarification between “showing approval/agreement” vs. “showing interest” was based on whether the person was encouraging, praising, or agreeing with the other person he/she is conversing with (“showing approval/agreement”), or whether it was just a friendly general conversation that went back and forth among the subjects but did not indicate any approval or agreement (“showing interest”); 4) the clarification between “asking for help/cooperation” vs. “assessing for comfort/condition” vs. “showing interest” was based on whether the utterance was attempting to elicit an action (“asking for help/cooperation”), check on the subject’s status (“assessing for comfort/condition”), or solicit more information to understand something or someone (“showing interest”); 5) the “unsure-positive” and “unsure-negative” codes were least descriptive and were used only when the utterance was indistinguishable or unclear or was transcribed as “unsure” literally; 6) the clarification between “orientation/giving instructions” vs. “controlling voice” vs. “verbal refusal” was based on tone of voice. While “controlling voice” was usually used when there was a harsh or negative voice, the use of a strong and loud voice did not always justify the use of “verbal refusal” or “controlling voice”.

For Part III (nonverbal behaviors), coding challenges and solutions were identified for both residents and staff (Table 5). There were three coding challenges with solutions for residents’ nonverbal behaviors: 1) the selection of one of the four actions to appropriately describe “disengagement from the meal” was based on observation of the video; 2) the code “prolonged chewing” was used when the chewing activity occurred without interruption for 10 seconds or more; and 3) the clarification between “leaning forward/backward” vs. “turning head away” was based on whether the movement involved the whole upper body (“leaning forward/backward”) or just the head (“turning head away”).

Table 5.

Major Coding Challenges and Solutions for Part III (Non-verbal behaviors)

Coding Challenges for resident non-verbal behaviors	Solutions	No. of videos with the issue
1. Clarification of different modifiers of “disengagement from the meal”.	There are four modifiers (closing eyes, distraction, falling asleep/becoming drowsy, others) underneath this code. Choose the most appropriate modifier based on observation of the action. If the action doesn’t qualify for any of the first three modifiers, code as “others” and make note of (describe) the action literally in the comment section.	5
2. When to code “prolonged chewing”, and how frequently can this code be used?	When there is food in the resident’s mouth, code “prolonged chewing” after 10 straight seconds counting of the continuous chewing activity. If the chewing activity is interrupted by another bite, drink, verbal communication, or other interruptive activities, start counting over from 1. If there is a short break of chewing (2–3 seconds) without any interruptive activities, don’t count during the short break and continue counting after the short break. This code can be used as many times as it occurs following the definition.	5
3. Clarification between “leaning forward/backward” and “turning head away”	“Leaning forward/backward” usually includes the entire upper body moving into a direction, whereas “turning head away” usually involves the movement of only the head, rather than the whole upper body.	2
Coding Challenges for staff non-verbal behaviors
1. Clarification between “offering different types of food/beverage/finger food/condiments” and characteristics of intake episodes coded in Part I	The codes “offering different types of food”, “offering beverage”, “offering finger food”, and “offering condiments” are general actions that describe situations that the staff is attempting to offer resident something. These actions are not necessarily followed up with an actual drink or bite, and are distinct from any codes from phase 1.	9
2. How to capture specific details of a situation in nonverbal behavior coding?	If a code is not specific enough to represent the nonverbal behavior, descriptive notes can be added to the comment section of that code to specify certain action or situation. For example: if the code “affirmative nodding” is used, it can be specified as to who they are directed towards by describing it in the comment section of this code in the Noldus software. Data in the comment sections can be exported to Excel worksheet for descriptive analysis.	6
3. How to code a specific instance that appears in the videos but seems not represented by any nonverbal behavior code?	Code the nonverbal behavior as “other” under one of the four categories as appropriate, and describe the behavior in details in the comment section in the Noldus software. Data on “other” codes and related comments can be exported to Excel worksheet and described/categorized for common themes as appropriate. The purpose is to see if all staff nonverbal behaviors can be represented using available codes, or if new nonverbal behavior codes can be developed to add to the coding scheme.	5
4. Clarification between “positive gestures/facial expressions”, “appropriate use of affectionate touch”, “affirmative nodding”, and “resident-directed eye gaze”	While “appropriate use of affectionate touch”, “affirmative nodding”, and “resident-directed eye gaze” may fall under the code “positive gestures/facial expressions”, these three codes describe specific gestures or facial expressions and should be used when specific instances are observed. “Positive gestures/facial expressions” is used as a general description of appropriate situations when none of the three specific behavioral codes can be applied, such as laughter, waving, blowing a kiss, and smiling.	4

In addition, there were four coding challenges and solutions for staff nonverbal behaviors: 1) the codes “offering different types of food/beverage/finger food/condiments” described general actions or situations that staff was attempting to offer resident something, which may not necessarily be followed with an actual drink or bite by the resident; 2) if a code was not specific enough to represent the nonverbal behavior, descriptive notes can be added in the comment section following that code to specify certain situation in the Noldus software; 3) if a specific instance was observed but not represented by any code, we will code the nonverbal behavior as “other” under one of the four nonverbal behavior categories as appropriate and descriptive notes can be added to the comment section in the Noldus software; and 4) the clarification between “positive gestures/facial expressions”, “appropriate use of affectionate touch”, “affirmative nodding” and “resident-directed eye gaze” was based on whether the nonverbal behavior was a specific instance of positive gestures or facial expressions (“appropriate use of affectionate touch”, “affirmative nodding” and “resident-directed eye gaze”), or just a general instance that none of the specific codes can be applied (“positive gestures/facial expressions”).

Other general challenges identified were technical challenges related to the use of Noldus Observer® 14.0 and were addressed by reference to software user manuals and consultation of Noldus technical support. Trained coders reported fewer challenges toward the middle and end of each coding phase as they gained more experiences in behavior coding following the refined CUED coding manual.

Feasibility

The average time to code one single video clip was 11.5 mins (range = 2–60, SD 10.35), 19.0 mins (range: 2–120, SD 15.61) and 18.86 mins (range: 5–130, SD =18.39) for Part I, II and III, respectively. On average, it took 2.52 hours (Part I), 4.16 hours (Part II, excluding transcription time) and 4.13 hours (Part III) to code a one-hour video. Altogether, it took an average of 10.81 hours (excluding transcription time) to code a one-hour video using the three parts of the refined CUED (coding time: video length=10.81:1). For Part II, it took an average of 23.16 minutes to transcribe a single video clip (range: 2–150, SD 21.56), which was 5.12 hours per one-hour video.

Inter-rater Reliability

The whole refined CUED had adequate inter-rater reliability. For Part I, percent agreement ranged from 95.93% to 99.17% (all p<.001, ± 1s tolerance) and Cohen’s Kappa ranged from 0.95–0.99 (all p<.001, 95% CI = 0.91–0.99, ±1s tolerance). For Part II, percent agreement ranged from 94.51% to 97.60% (all p<.001, ± 1s tolerance) and Cohen’s Kappa ranged from 0.94–0.97 (all p<.001, 95% CI = 0.93–0.98, ±1s tolerance). For Part III, percent agreement ranged from 93.63–96.70% (all p<.001, ± 1s tolerance) and Cohen’s Kappa ranged from 0.93–0.96 (all p<.001, 95% CI = 0.92–0.97, ±1s tolerance).

DISCUSSION

This study psychometrically evaluated a refined computer-assisted behavioral coding scheme to assess characteristics of the dynamic intake process and dyadic verbal and nonverbal mealtime interactions. All the codes as well as their definitions and coding examples were identified from established tools and relevant literature related to dementia mealtime care, enhancing the content validity of the refined CUED. Findings provided preliminary evidence on the ease of use, feasibility and inter-rater reliability of the refined CUED using videotaped observations.

The refined CUED showed a higher ratio of coding time to video length (10.8:1), compared with the original CUED (6:1). This may be due to three reasons: 1) the refined CUED include an addition of 24 staff nonverbal behaviors and 14 resident nonverbal behaviors; 2) in the testing of the original CUED, verbal and nonverbal behaviors were coded in a way that the types of behavior were coded without the actual time point each behavior occurred; however, in this study both the type of each behavior and the actual time point the behavior occurred were coded; and 3) compared with the videos of whole meals used in testing the original CUED, video clips used in this study were mostly part of meals showing breakdown scenarios of mealtime and thus may require more coders’ time to understand the scenarios and transcribe and code the behaviors.

We identified some coding challenges in using the refined CUED, including identification of key characteristics of an intake episode and differentiation between similar verbal (nonverbal) behaviors. The targeted solutions that we developed to address these coding challenges will facilitate the understanding and application of the refined CUED in future research. The refined CUED showed good inter-rater reliability indicating that coders were able to operationalize the refined CUED coding scheme in a consistent way following the standard coding manual. The process of summarizing coding challenges and developing targeted solutions among all coders helped establish and maintain adequate inter-rater reliability throughout the coding process.

Implication for Research and Clinical Practice

This study is the first that refined the CUED and provided preliminary psychometric evidence for using the refined CUED as a feasible, ease to use and reliable computer-assisted behavioral coding scheme to assess the complex dynamic mealtime care scenarios using videotaped observations among a large and diverse sample of NH staff and residents with advanced dementia. While the refined CUED is an innovative coding scheme to assess multiple aspects of the intake process and dyadic mealtime interactions, it is considered time intensive and may not be applicable to address all research inquires related to dementia mealtime care. The use of the refined CUED requires decision-making on the appropriateness of research inquiries as well as adequateness of personnel and time. The use of the refined CUED and videotaped observations is very useful in understanding the complexity and dynamics of dementia mealtime care (i.e., intake process, dyadic mealtime interactions) and will provide much more in-depth data (e.g., type and time sequence of behaviors) than commonly used observational tools in real-time on-site observations. Data obtained will help address certain important questions, such as clustering and patterns of dyadic behaviors, temporal relationships between dyadic interactions and intake, which may not be addressed using existing observational tools developed for real-time on-site observations or behavioral coding tools that captured limited aspects of mealtime care interactions.

When applicable [e.g., a team thinks the refined CUED is an appropriate tool for their research/clinical question(s) and have adequate personnel/time], findings of this study including coding challenges and targeted solutions, time needed for the amount data to be coded using each part of CUED and the process of establishing inter-rater reliability will serve as a guide and inform the operations for future research. In addition, while the refined CUED was developed as a whole coding scheme, future dementia mealtime care research may use each part of the tool separately depending on the research purpose. In addition, while Part 1 of the CUED (intake process) was developed from research that involve persons with dementia, it may be applied to research that involve persons with other chronic conditions or diagnoses with declined functional ability in eating to examine characteristics of intake process and factors associated with intake in different populations.

Future research directions

In future research, the refined CUED may be used to inform intervention development by addressing four research aims: 1) characterize the distributions of dyadic verbal and nonverbal behaviors as well as the intake process, 2) explore the clustering of verbal and nonverbal behaviors for staff and residents; 3) examine the role of dyadic verbal and nonverbal behaviors on resident intake and 4) examine the temporal relationships between staff behaviors and resident behaviors and between dyadic behaviors and resident intake. For example, staff use of negative verbal prompts with an intent to maintain or promote eating independence in NH residents with dementia during mealtime were identified in case studies (Palese et al., 2018), yet, the characteristics and impact of such staff negative verbal behaviors on resident behaviors and intake haven’t been examined. In addition, the nonverbal behaviors for staff and residents in the refined CUED were grouped into multiple categories based on conceptual basis. Future work needs to explore the clustering of nonverbal behaviors (e.g., which behaviors cluster together in assessing resident resistive behaviors) and compare with current categories in the refined CUED using data from larger diverse samples of mealtime care scenarios.

Another example, current mealtime care interventions primarily focus on the use of direct feeding skills rather than positive dyadic interactions, have low to insufficient evidence to decrease resident mealtime difficulties and increase intake and fail to address staff’s needs for knowledge and skills to provide person-centered mealtime care (Batchelor-Murphy et al., 2017; Chang & Lin, 2005; Chen et al., 2016; Liu, Cheon, et al., 2014; Liu et al., 2015). Future research on the temporal relationship between staff verbal and nonverbal behaviors and resident mealtime difficulties and intake will help identify specific staff behaviors that may reduce (precede) resident mealtime difficulties and/or improve (decrease) intake. Such information will guide the use of innovative mealtime care strategies to improve care quality, minimize mealtime difficulties and optimize intake in residents with dementia. In addition, the refined CUED can be used to evaluate the effects of innovative mealtime care interventions on staff verbal and nonverbal behaviors, resident verbal and nonverbal behaviors and resident intake.

Limitations

This study is limited due to the use of video clips with varied durations instead of the whole mealtime. Selection of videos is limited to one-on-one interactions that involves one primary staff and one resident to minimize the complexity of dyadic interactions. The generalizability of the findings may be limited as the study focused on dyadic mealtime interactions between nursing home staff and residents with moderately severe to severe dementia and staff-reported resistiveness to daily care in the United States.

CONCLUSION

The study provided preliminary evidence on the ease of use, feasibility and inter-rater reliability of the refined CUED. In consideration of the balance among time intensity in using the tool, the richness of data obtained from using the tool and the implications for future research and clinical care, the tool may be useful and appropriate to address certain research inquires (e.g., temporal relationship between behaviors and between behaviors and intake). Such information will inform the development and evaluation of effective mealtime care interventions to decrease mealtime difficulties and maximize intake in residents. Future research is needed to test the refined CUED in diverse populations with dementia or other chronic conditions in different care settings (e.g., assisted living, hospitals, community/home-care) to accumulate evidence to support the application of the tool in mealtime care research.