Development and Psychometric Testing of the Mealtime Engagement Scale in Direct Care Providers of Nursing Home Residents With Dementia

Abstract

Background and Objectives

Mealtime engagement is defined as verbal and nonverbal assistance provided by caregivers to guide and motivate care recipients in eating. Quality mealtime engagement is critical to improve mealtime difficulties and intake among older adults with dementia requiring eating assistance. Few tools are feasible and valid to measure mealtime engagement. This study developed and tested the Mealtime Engagement Scale (MES).

Research Design and Methods

Items were developed based on literature review and expert review and finalized based on content validity and corrected item-total correlation. A secondary analysis of 87 videotaped observations capturing 18 nursing home staff providing mealtime care to residents with dementia was conducted. Internal consistency, interrater reliability, and intrarater reliability were assessed. Concurrent and convergent validity were examined through correlation (r_s) with the Relational Behavior Scale (RBS) and the Mealtime Relational Care Checklist (M-RCC), respectively.

Results

The 18-item MES was developed with adequate content validity (Scale-content validity index [CVI] = 1.00; Scale-CVI/Average = 0.962–0.987). Each item is scored from 0 (never) to 3 (always). The total scale score ranges from 0 to 54. Higher scores indicate greater mealtime engagement. The MES had very good internal consistency (Cronbach’s α = 0.837), outstanding interrater reliability (interclass correlation = 0.920), outstanding intrarater reliability (interclass correlation = 0.956), adequate concurrent validity based on strong correlation with the RBS (r_s = 0.821, p < .001), and fair convergent validity based on weak correlation with the M-RCC (r_s = 0.219, p = .042).

Discussion and Implications

Findings provide preliminary psychometric evidence of MES to measure mealtime engagement. Future testing is needed among more and diverse samples in different care settings to accumulate psychometric evidence.

Persons with advanced dementia in nursing homes (NHs; residents) commonly experience functional, cognitive, and behavioral symptoms interfering with the process of getting food into the mouth and swallowing it (mealtime difficulties). Mealtime difficulties are significant and complex issues and often result in low intake (Keller et al., 2017). Residents with dementia are at high risk of dehydration and malnutrition due to cognitive impairment, functional decline, and swallowing difficulty, leading to increased infection, weight loss, morbidity, and mortality (Bell et al., 2015).

Multilevel factors (i.e., intrapersonal, interpersonal, environmental, and institutional) influence mealtime difficulties and intake (Liu et al., 2016, 2017, 2018; Liu, Jao et al., 2019; Liu, Williams et al., 2019). Among these factors, lack of quality assistance from direct care providers (caregivers) to engage residents in eating is a significant modifiable factor that can be intervened to improve mealtime difficulties and intake (Liu et al., 2017; Liu, Perkhounkova et al., 2020b). Mealtime engagement is conceptually defined as verbal and nonverbal assistance provided by caregivers to guide and motivate residents in eating (Cohen-Mansfield et al., 2009; Liu et al., 2018). Caregivers’ verbal and nonverbal behaviors toward residents are the most direct stimulation that residents receive during mealtime and are associated with improved eating independence and intake (Liu et al., 2017). While prior work shows residents with or without dementia requiring limited to total eating assistance based on Minimum Data Set (MDS) often receive inadequate assistance from staff (Simmons et al., 2002), recent work on residents with advanced dementia requiring extensive to total assistance shows staff tend to provide full assistance for most meals, even to those residents with the potential functional ability to initiate eating by themselves (Liu, Williams et al., 2019). Providing full assistance may disengage residents from eating and is associated with decreased intake, whereas providing continuous facilitation using verbal and nonverbal strategies engages residents and increases intake (Liu, Williams et al., 2019).

Mealtime care that involves caregiver–resident (dyadic) interaction is interactive, dynamic, and complex in nature (Liu, Perkhounkova et al., 2020a). In order to understand dyadic mealtime interaction and develop effective mealtime assistance interventions, it is important to characterize mealtime engagement among caregivers. Few valid measurements are available to assess caregiver mealtime engagement (Liu, Kim et al., 2020). Current assessments are mostly developed to assess the overall quality of routine daily care (not mealtime-specific) among caregivers of people with or without dementia, and later used to assess mealtime care quality among caregivers of people with dementia, and have not been widely tested and have low to insufficient psychometric evidence (Liu, Kim et al., 2020). Two tools, the Feeding Skills Checklist (Aselage, 2011) and the Formal Caregivers’ Behaviors toward Feeding Dementia Patients Observation Checklist (Chang & Lin, 2005), were developed to assess caregivers’ behaviors in providing mealtime care to people with dementia in intervention studies. These tools have limited use in mealtime care research, are established with low psychometric evidence, and warrant further testing (Liu, Kim et al., 2020).

A psychometrically sound measure is needed to measure and characterize mealtime engagement, understand its role on resident outcomes, and evaluate innovative interventions to improve mealtime difficulties and intake. This study aimed to develop the Mealtime Engagement Scale (MES) and test its reliability (i.e., internal consistency, interrater reliability, and intrarater reliability) and validity (i.e., content validity, concurrent validity, and convergent validity).

Method

The MES was developed and tested based on Classical Test Theory following a systematic approach through four steps (Figure 1): (a) a literature review to generate study records and theoretical basis, (b) expert reviews to generate and refine item pool and establish face and content validity, (c) pretesting to develop response options and improve the feasibility of the scale using videotaped observations of part of meals obtained from a dementia communication trial (parent study A), and (d) psychometric testing using videotaped observations of entire meals obtained from a handfeeding trial (parent study B). Videotaped observations were used in this study because they allow for secondary observations and recording of multiple factors to obtain a precise measurement of mealtime engagement not achievable with direct onsite observations. Ethical approvals were obtained from the universities where parent studies and this study were conducted.

Figure 1.

Development and psychometric testing of the Mealtime Engagement Scale.

Literature Review

Data sources and search strategy

A literature review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Liberati et al., 2009). Five electronic databases (i.e., PubMed, CINAHL, Ageline, PsychINFO, and Cochrane Library) were searched for scholarly records published in English between January 1, 1980 and December 31, 2016. Other sources included a bibliography of eligible records for complete retrieval of relevant literature. Keywords included dementia, Alzheimer*, meal*, eat*, feed*, and matched subjects or MeSH terms. The search strategy used in PubMed was shown as an example (Supplementary Table 1). During the item development process, an updated literature search was performed to identify records published between January 1, 2017 and June 31, 2019.

Eligibility criteria of records

Inclusion criteria for full-text records were that (a) the study sample included caregivers of persons with dementia and (b) there were descriptions or measures of caregiver behaviors during mealtime care. Two reviewers independently screened titles, abstracts, and full texts. Reviewers’ results were compared and discrepancies were discussed to arrive at a consensus for eligible full texts.

Search results and study selection

In total, 9,438 records were identified. After removing 1,794 duplicates and excluding 7,171 ineligible records based on title/abstract screening, 454 full texts were reviewed to identify eligible records that described and/or measured caregiver mealtime behaviors for persons with dementia. Finally, 28 records were identified as eligible sources to develop an item pool.

Generation and Refinement of Item Pool

Theoretical framework

The Social Ecological Model has been used to examine factors at the resident, caregiver, and environmental levels that influence eating performance and intake in persons with dementia (Liu et al., 2016, 2017, 2018; Liu, Jao et al., 2019). In this study, the Social Ecological Model was used as a comprehensive framework to guide the development of potential items that assess caregiver mealtime engagement from three domains: (a) modification for resident cognitive/functional abilities, (b) modification for caregiving approaches, and (c) modification for dining environment elements (e.g., food and physical environment).

Item development and internal review

By reviewing the 28 eligible full texts, 160 items were extracted, among which 87 items were eliminated as duplicates or combined with other items due to similarity, resulting in 73 items. The 73 items were reviewed on face validity by the authors based on four criteria: (a) relevance (i.e., whether the item was relevant and applicable to mealtime care), (b) specificity (whether the item was specific and concrete to mealtime care such as providing verbal cues, rather than generic and abstract such as encouraging independence), (c) clarity/readability (whether the item was clear and easy to understand), and (d) feasibility (whether the item was observable and feasible to rate). Following the four criteria, 11 items were eliminated due to lack of relevance, specificity, or feasibility, resulting in a 62-item pool (Supplementary Table 2). We developed descriptions for all items based on a review of relevant literature.

Expert panel review

We conducted two rounds of review to evaluate items using the same expert panel and the same criteria. The panel included six research and/or clinical experts (i.e., one nursing gerontologist, two nursing researchers, one nutritionist, one statistician, and one geropsychiatric nurse) in the areas of dementia mealtime care and communication, feeding skills training, nutrition, measurement, and psychometrics. The panel provided both quantitative ratings and qualitative feedback following four criteria (relevance, specificity, clarity/readability, and feasibility) used in the internal review. Experts rated each item on each criterion using a 1–4 rating system (1 = not at all, 2 = somewhat, 3 = quite, and 4 = highly). If one item was scored 1 or 2 on any criterion, experts were encouraged to suggest refinements through qualitative feedback.

First-round expert review and item refinement

In total, 62 items were reviewed. Experts’ qualitative feedback and quantitative ratings were primarily used to guide the revision of items. Major qualitative feedback was a lack of relevance or feasibility, duplication, and an indication of multiple behaviors in one item. Based on experts’ feedback, we applied four approaches in refining the items: (a) items with ratings of 3 or 4 on all criteria were kept as original; (b) items with ratings of 1 or 2 on any criterion were reworded if appropriate, or combined or deleted due to duplication; (c) items that indicated multiple behaviors were divided into separate items, with each item indicating one behavior; and (d) new items were developed. In the refinement, 13 items were kept as original, nine items were reworded, 37 items were deleted, two original items were broken into four separate items, one original item was broken into three separate items, and two new items were developed. The number of items decreased from 62 to 31. The descriptions of the 31 items were revised as appropriate.

Second-round expert review and item refinement

In total, 31 items were reviewed. The focus of this round was to further refine the items and, more importantly, aggregate quantitative ratings to calculate content validity index (CVI). Following reviewers’ feedback, five approaches were used in refinement: (a) 22 items with ratings of 3 or 4 on all criteria were kept; (b) three items with ratings of 1 or 2 on more than one criteria were deleted (i.e., assess emotional comfort, take over the individual’s eating attempt, and interrupt individual); (c) two items with ratings of 1 or 2 on one of the four criteria were reworded and kept; (d) two items (i.e., offer different food and offer a different drink) were combined as one (i.e., offer a different item) to allow for recording care situations of offering a food (drink) after refusal of a drink (food); and (e) two items (i.e., ignore individual and reject individual) were combined as one (i.e., ignore individual), because declining an individual’s request (reject individual) was considered not responding to the individual’s request directly and was within the scope of ignoring the individual. After the refinement, the number of items decreased from 31 to 26. The descriptions of the 26 items were refined as appropriate.

Content validity index

For each criterion, we calculated item-CVI (I-CVI, the proportion of experts giving each item a rating of 3 or 4), Scale-CVI/Ave (S-CVI/Ave, the average of I-CVIs for all the items), and Scale-CVI (S-CVI, the proportion of items with I-CVI ≥0.8) for the 26-item set (Polit & Beck, 2006) using ratings from the second-round expert review. A scale that has items with I-CVIs of at least 0.78 for 6–10 experts and has an S-CVI/Ave of at least 0.90 is considered as having excellent content validity (Polit & Beck, 2006). For the 26-item set, the range of I-CVIs was 0.67–1.00 (feasibility) and 0.83–1.00 (relevance, specificity, and clarity/readability). The S-CVI/Ave was 0.987 (relevance), 0.962 (specificity), 0.987 (clarity/readability), and 0.968 (feasibility). The S-CVI was 1.00 for all criteria. Findings indicated that the 26-item set had good content validity.

Pretesting

The pretesting aimed to (a) validate and refine the description of each item; (b) describe the frequency of the caregiver behavior represented by each item; (c) determine whether a binary (0 = absent/not observed, 1 = present/observed) or polytomous (0 = never, 1 = sometimes, 2 = often, 3 = always) response option was appropriate; and (d) if the polytomous response option was appropriate, define sometimes, often, and always based on the observed frequency of all behaviors.

We used a purposive sampling of 14 videotaped mealtime observations obtained from a dementia communication trial (parent study A; Williams et al., 2016). In parent study A, 110 videos that recorded resident mealtime activities under usual care, captured one-on-one interaction between one staff and one resident, and lasted for at least 1 min were collected in Midwestern United States (Liu, Perkhounkova et al., 2020b). In this study, videos were included if they (a) lasted at least 5 min, (b) had a good quality to observe caregiver verbal and nonverbal behaviors, and (c) captured the adequate amount of dyadic interaction at different phases of mealtime (e.g., beginning, middle, and end of mealtime). Among the 110 videos, 14 eligible videos with a mean duration of 7.73 minutes (SD = 1.86, range = 5.0–10.3) were identified. The 14 videos involved 14 unique dyads including 10 staff (mean age = 36 years, 80% women, 50% white, and 50% African American, and 100% non-Hispanic) and nine residents with advanced dementia (mean age = 84 years, 56% women, 100% white and non-Hispanic) from five NHs.

Two raters (Tobokane Manthai and Elizabeth K. Theisen) who were trained by the first author watched each video and recorded data on (a) whether the description of each item was clear and inclusive, and if not, suggestions to improve clarity and inclusiveness, and (b) how many times the caregiver behavior was represented by each item observed. Findings showed that 22 items were observed (range = 1–7 times), and four items (i.e., describe meal items, re-approach resident to continue meal, ignore individual, and control individual) were not observed in any video. We kept the four items at this phase because these items were conceptually sound and could occur in real-life mealtime care (e.g., staff may “describe meal items” when the resident does not recognize the food being served). Based on the range of the frequency (i.e., 0–7), we defined and used a four-point polytomous response option (0 = never, 0 time; 1 = sometimes, 1–2 times; 2 = often, 3–5 times; 3 = always, 6 times or more). Item descriptions were refined based on raters’ suggestions to improve clarity.

Psychometric Testing

We conducted a secondary analysis of videotaped mealtime observations obtained from a clinical trial (parent study B), which tested a feeding skills training program that taught NH staff evidence-based interventions to assist residents with dementia during mealtime (Batchelor-Murphy & Crowgey, 2016).

Sample and setting

In the parent study B, residents were eligible if they were aged 60 years or more, residing in the participating NH for the previous 6 weeks, had a legally authorized representative to provide informed consent, had a diagnosis of dementia and required limited to total assistance for eating based on MDS 3.0, and had a Brief Interview for Mental Status score ranging from 0 to 12 (lower score indicates greater cognitive impairment; Saliba et al., 2012). Residents were excluded if they had a diagnosis of Parkinson’s disease, HIV infection, traumatic brain injury, swallowing disorder, parental/IV feedings, presence of a feeding tube, or any significant auditory or visual impairment that would prevent the resident from hearing or seeing verbal or visual cues. Staff were eligible if they were employed in the participating NH for the previous 30 days and able to speak/read English. Participation in training was not mandatory for staff. In total, 144 videotaped interactions of entire meals were collected among 25 staff and eight residents with advanced dementia in two NHs in the southeastern United States in the parent study B. Eighty-seven videos that captured one-on-one usual mealtime interaction between one untrained staff and one resident with good video/audio quality were selected for this study.

Measures

The Relational Behavior Scale (RBS) was used to evaluate the concurrent validity of MES. The RBS is developed to assess relational behaviors that caregivers demonstrate with long-term care residents (McGilton et al., 2012). The RBS has three items assessing the frequency of three pairs of relational-focused/task-focused behaviors: (not) staying with the resident during the care episode, (not) altering the pace of care by recognizing the person’s rhythm and adapting to it, and focusing care beyond the task (or task-focused). Each item is scored from 1 (consistently negative) to 7 (consistently positive). The total score ranges from 3 to 21. Higher scores indicate better relational behaviors and less task-focused care from caregivers. In this study, RBS had very good internal consistency (Cronbach’s α = 0.860, 95% confidence interval [CI] = 0.799–0.904), outstanding interrater reliability (interclass correlation [ICC] = 0.839, 95% CI = 0.590–0.936), and outstanding intrarater reliability (ICC = 0.874, 95% CI = 0.693–0.949).

The Mealtime Relational Care Checklist (M-RCC) was used to evaluate the convergent validity of MES. The M-RCC was originally developed to assess the overall relational care received by all residents across the dining room (rather than individual residents; Iuglio et al., 2019). The 17 M-RCC items were scored on the frequency of task-focused versus relational care practices using a five-point Likert format: 0 (no incidents of relational practice observed), 1 (more task-focused than relational), 2 (equal amount of task-focused and relational), 3 (more relational than task-focused), and 4 (relational care practices consistently observed). The total score ranges from 0 to 68. When used at the dining room level, M-RCC demonstrated good interrater reliability (ICC = 0.78, 95% CI = 0.62–0.87; Keller et al., 2019). When used at the individual resident level, M-RCC demonstrated preliminary evidence of construct validity based on weak associations with the functionality scale on the Dining Environment Audit Protocol (r_s = 0.23, p = .04), relational care received by all residents across the dining room (r_s = 0.25, p = .02), the dining room Person-Centered Summary Scale (r_s = 0.28, p = .01), the physical scale from the Meal Time Scan (r_s = 0.37, p = .0007), and resident malnutrition risk (r_s = 0.23, p < .0001; Iuglio et al., 2019). In this study, M-RCC was used to assess the relational care received by each individual resident and showed minimally acceptable internal consistency (Cronbach’s α = 0.656, 95% CI = 0.373–0.724) and substantial interrater reliability (ICC = 0.646, 95% CI = 0.174–0.878).

Data analysis

A corrected item-total correlation was performed to examine the correlation of each item score with the total score excluding the item score. The strength of the correlation was interpreted based on the size of the correlation coefficient: 0.20 was considered very weak or no relationship, 0.20–0.40 weak, 0.40–0.60 moderate, 0.60–0.80 strong, and 0.80–1.0 very strong (Salkind, 2012). Items with very weak or no relationship or very strong correlation were eliminated. Items with weak, moderate, or strong corrected item-total correlations (0.20–0.80) were kept.

Internal consistency was estimated using Cronbach’s α. The size of Cronbach’s α was interpreted as follows: below 0.60 was considered unacceptable, between 0.6 and 0.65 undesirable, between 0.65 and 0.70 minimally acceptable, between 0.70 and 0.80 respectable, between 0.80 and 0.90 very good, and above 0.90 leading to the consideration of item redundancy and potential needs of shortening the scale (DeVellis, 2016).

Interrater reliability was examined through ICC of absolute agreement on the total MES score of 20 videos across two independent trained raters (Tobokane Manthai and Elizabeth K. Theisen) during the same time period. Intrarater reliability was examined using ICC of absolute agreement on the total MES score of 20 videos from one trained rater (Elizabeth K. Theisen) at two different time periods (2–4 weeks apart). The trained raters watched each video independently and recorded data using the MES scale for each video. The size of ICC was interpreted as follows: between 0.40 and 0.59 is considered acceptable, between 0.60 and 0.79 substantial, and at least 0.80 outstanding (DeVellis, 2016).

Concurrent validity was examined through Spearman’s ρ correlation (r_s) between MES total score and RBS total score. Convergent validity was examined through r_s between MES total score and M-RCC total score of individual residents. Spearman’s ρ correlation was used because the total scores of the three measures were not normally distributed. The size of r_s was interpreted using the same cutoffs as those for corrected item-total correlation (Salkind, 2012). IBM SPSS Statistics 26.0 software (SPSS Inc., Chicago, IL) was used for all analyses. The level of significance was .05.

Results

Participant Characteristics

The 87 videos involved seven residents with dementia and 18 different staff in two NHs (Table 1). Residents had a mean age of 88 years. The majority were women (85.7%) and white (85.7%). Four residents needed extensive functional assistance and three needed limited assistance. Three residents had a minimal or moderate hearing impairment and four had intact hearing. Two residents had moderate or high visual impairment and five had normal vision.

Table 1.

Characteristics of Residents and Staff

Continuous variables	Mean ± SD	Range
Resident age (years)	88.12 ± 9.45	72–100
Staff age (years)	38.94 ± 13.46	22–63
Staff length of nursing home caregiving experiences (years)	14.12 ± 12.78	0.33–40
Categorical variables	N (%)
Resident gender
Female	6 (85.7)
Male	1 (14.3)
Resident race
White	6 (85.7)
Black or African American	1 (14.3)
Resident functional status
Limited assistance needed	3 (42.9)
Extensive assistance needed	4 (57.1)
Resident hearing impairment
None	4 (57.1)
Minimal impairment	2 (28.6)
Moderate impairment	1 (14.3)
Resident visual impairment
None	5 (71.4)
Moderate impairment	1 (14.3)
High impairment	1 (14.3)
Staff gender
Female	15 (83.3)
Male	3 (16.7)
Staff race
Black/African American	13 (72.2)
White/Caucasian	5 (27.8)
Staff education level
Up to high school	6 (33.3)
Some college education	9 (50.0)
Bachelor degree to graduate school	3 (16.7)
Staff role
CNA	15 (83.3)
RN, LPN, or other	3 (16.7)
Whether staff have worked with people with dementia?
Yes	17 (94.4)
Missing	1 (5.6)
Whether staff have received special training to care for people with dementia?
Yes	10 (55.6)
No	8 (44.4)
Whether staff have ever personally experienced a mealtime issue when providing care to a person with dementia?a
Yes	12 (66.7)
No	6 (33.3)

Notes: CNA = certified nursing assistant; LPN = licensed practical nurse; RN = registered nurse. The sample included seven residents and 18 staff participants.

^aMealtime issue indicates any functional, cognitive, and behavioral symptoms that interfere with the process of getting food into the mouth and swallowing it (mealtime difficulties) shown by the care recipient.

Staff had a mean age of 39 years and a mean caregiving length of 14 years. Most staff were female (83.3%), African American (72.2%), and certified nursing assistants (83.3%). The majority received high school or some college education (83.3%) and had experiences working with people with dementia (94.4%). More than half received special training to care for people with dementia (55.6%) and had personally experienced a mealtime issue (i.e., any mealtime difficulties shown by care recipients such as not opening mouth) when providing care to a person with dementia (66.7%).

Video Characteristics

The 87 videos had a mean duration of 32.94 min (range = 6.12–87.58) and captured a balanced number of breakfasts (25.3%), lunches (37.9%), and dinners (36.8%). Videos captured moderate levels of relational behaviors provided by caregivers as measured by RBS (mean = 15.83, SD = 3.83, range = 3–21), moderate levels of relational care received by residents as measured by M-RCC (mean = 32.74, SD = 10.28, range = 12–52), and low to moderate levels of mealtime engagement provided by caregivers as measured by MES (mean = 22.47, SD = 8.75, range = 0–38). Staff mealtime engagement skewed toward lower levels as indicated by the distribution of response options in each item (Table 2).

Table 2.

Mean Score and Item-Total Correlation of Mealtime Engagement Scale

		Never (0)	Sometimes (1)	Often (2)	Always (3)
#	Items	n (%)				Mean	SD	Skewness	Kurtosis	Corrected item-total correlation (rs)
1	Position individual upright	60 (69)	19 (21.8)	7 (8.0)	1 (1.1)	0.41	0.691	1.614	1.971	0.284
2	Address individual by preferred name	36 (41.4)	34 (39.1)	17 (19.5)	0	0.78	0.754	0.387	−1.138	0.178
3	Adjust proximity to individual	31 (35.6)	42 (48.3)	14 (16.1)	0	0.80	0.696	0.285	−0.897	0.225
4	Provide one-on-one assistance	2 (2.3)	7 (8.0)	20 (23.0)	58 (66.7)	2.54	0.744	−1.615	2.058	0.580
5	Describe meal items	36 (41.4)	27 (31.0)	10 (11.5)	14 (16.1)	1.02	1.089	0.728	−0.780	0.506
6	Assess food preferences	32 (36.8)	23 (26.4)	21 (24.1)	11 (12.6)	1.13	1.054	0.412	−1.097	0.561
7	Assist with one item at a time	13 (14.9)	9 (10.3)	17 (19.5)	48 (55.2)	2.15	1.116	−0.971	−0.546	0.492
8	Give bites of appropriate size	19 (21.8)	2 (2.3)	26 (29.9)	40 (46.0)	2.00	1.171	−0.844	−0.816	0.518
9	Offer drinks between bites	13 (14.9)	15 (17.2)	21 (24.1)	38 (43.7)	1.97	1.104	−0.620	−1.008	0.705
10	Assess readiness for next bite	23 (26.4)	14 (16.1)	13 (14.9)	37 (42.5)	1.74	1.262	−0.301	−1.595	0.714
11	Provide physical guidance	27 (31.0)	16 (18.4)	30 (34.5)	14 (16.1)	1.36	1.089	0.019	−1.342	0.188
12	Provide verbal guidance	10 (11.5)	17 (19.5)	25 (28.7)	35 (40.2)	1.98	1.034	−0.599	−0.855	0.406
13	Offer a different item	9 (10.3)	24 (27.6)	21 (24.1)	33 (37.9)	1.90	1.035	−0.369	−1.140	0.457
14	Remove clutter or uneaten items	42 (48.3)	22 (25.3)	14 (16.1)	9 (10.3)	0.89	1.028	0.827	−0.566	0.395
15	Wipe away oral spillage or drool	31 (35.6)	18 (20.7)	24 (27.6)	14 (16.1)	1.24	1.110	0.236	−1.336	0.527
16	Nonverbal engagement	71 (81.6)	11 (12.6)	4 (4.6)	1 (1.1)	0.25	0.595	2.572	6.642	0.337
17	Assess physical comfort	73 (83.9)	12 (13.8)	2 (2.3)	0	0.18	0.445	2.432	5.501	0.376
18	Reapproach individual to continue meal	75 (86.2)	12 (13.8)	0	0	0.14	0.347	2.137	2.627	0.185

Corrected Item-Total Correlation

In the 26-item set, eight items (i.e., provide assistance within 5 min from food being served, mix food up, arrange items for easy access, offer condiments, offer finger food, verbal engagement, control individual, and ignore individual) had very weak or no corrected item-total correlations (r_s range = −0.11 to 0.15, all p > .05) and were eliminated. The remaining 18 items had significantly weak to strong corrected item-total correlations (r_s range = 0.20–0.69, all p < .01) and were kept (Table 2). The item-to-item correlations of the 18 items were weak to strong (r_s range = 0.11–0.74). The 18-item MES was developed (Table 3) along with item descriptions (Supplementary Table 3).

Table 3.

Mealtime Engagement Scale

#	Items	Never (0 time; unobserved)	Sometimes (1–2 times)	Often (3–5 times)	Always (6+ times)
1	Position individual upright	0	1	2	3
2	Address individual by preferred name	0	1	2	3
3	Adjust proximity to individual	0	1	2	3
4	Provide one-on-one assistance	0	1	2	3
5	Describe meal items	0	1	2	3
6	Assess food preferences	0	1	2	3
7	Assist with one item at a time	0	1	2	3
8	Give bites of appropriate size	0	1	2	3
9	Offer drinks between bites	0	1	2	3
10	Assess readiness for next bite	0	1	2	3
11	Provide physical guidance	0	1	2	3
12	Provide verbal guidance	0	1	2	3
13	Offer a different item	0	1	2	3
14	Remove clutter or uneaten items	0	1	2	3
15	Wipe away oral spillage or drool	0	1	2	3
16	Nonverbal engagement	0	1	2	3
17	Assess physical comfort	0	1	2	3
18	Reapproach individual to continue meal	0	1	2	3
			Total score = ________

Notes: Data Collection Instructions: Observe an individual (e.g., patient, resident, or a family member) for a whole meal and record how often the caregiver (e.g., direct care provider and family caregiver) performs each of the following behaviors to the individual. Scoring Instruction: All item scores are added up for a total score, indicating the level of positive behavioral engagement from caregivers during mealtime care of individuals. Total score ranges from 0 to 54; higher score = higher level of positive engagement.

Reliability and Validity

When the eight items that had very weak to no corrected item-total correlation were not deleted, internal consistency of the 26-item set was respectable (Cronbach’s α = 0.790, 95% CI = 0.720–0.848). After deleting the eight items, the 18-item MES had very good internal consistency (Cronbach’s α = 0.837, 95% CI = 0.782–0.883). Cronbach’s α increased by little (0.002–0.005), remained the same, or decreased if removing any of the 18 items. The 18-item MES had outstanding interrater reliability (ICC = 0.920, 95% CI = 0.570–0.975), outstanding intrarater reliability (ICC = 0.956, 95% CI = 0.892–0.982), adequate concurrent validity based on strong correlation with the RBS (r_s = 0.821, p < .001), and fair convergent validity based on weak correlation with the M-RCC (r_s = 0.219, p = .042).

Discussion

In response to the need for a reliable and valid measure to assess mealtime engagement of caregivers of individuals with dementia, we developed the 18-item MES using a systematic approach and generated preliminary evidence on its reliability and validity using videotaped mealtime observations among NH staff and residents with dementia. Specifically, the Social Ecological Model was used as a conceptual framework to develop items that address modifications for resident cognitive and functional abilities, caregiving approaches, and dining environment elements. An inclusive pool of items characterizing caregiver mealtime engagement was identified and evaluated through two rounds of expert review to ensure face and content validity. Findings of pretesting supported the use of a four-point polytomous response option as well as the feasibility of using videotaped observations to test MES.

From the 26-item set with adequate content validity, 18 items with weak to strong corrected item-total correlation were identified and composed of a scale with very good internal consistency, indicating the 18 items were clustering together in measuring caregiver mealtime engagement. The 18-item MES had excellent interrater reliability, intrarater reliability, and concurrent validity. The 18-item MES showed fair convergent validity through weak correlation with the M-RCC, probably because M-RCC was originally developed to assess the overall relational care received by all residents across the dining room but was used to assess relational care received by each individual resident in this study.

The 18-item MES is a useful tool that can be used to address different inquiries related to caregiver mealtime engagement. These include (a) describing differences on mealtime engagement by caregiver and resident characteristics, (b) examining factors that influence mealtime engagement, (c) examining relationships between caregiver mealtime engagement and resident mealtime difficulties and food intake, and (d) evaluating the process and fidelity of novel staff mealtime assistance training interventions that aim to improve resident outcomes through enhancing mealtime engagement. For example, while prior research supports the association between positive caregiver engagement and resident eating performance and intake (Liu et al., 2017; Liu, Jao et al., 2019), such evidence was established using single-item measures that assess mealtime assistance and environmental stimulation. Future research is needed to confirm the associative relationships using reliable and valid multiple-item measures (e.g., MES).

Another example for using MES is the development and evaluation of innovative mealtime care interventions. Caregivers report a lack of knowledge and skills in providing optimal mealtime care (Liu et al., 2018). Current mealtime care interventions primarily focus on the use of feeding skills rather than positive engagement, have low to insufficient evidence to improve mealtime difficulties and intake, and fail to address the gap in optimal mealtime care (Liu et al., 2014, 2015). Person-centered mealtime care that features optimal care through providing choices, acknowledging preferences, supporting independence, and promoting respectful social interactions is highly recommended (Liu et al., 2018; Liu, Williams et al., 2019; Reimer & Keller, 2009). With an urgent need to develop innovative person-centered mealtime care interventions, a reliable and valid observational instrument to measure caregiver mealtime engagement is essential to evaluate the process and fidelity (e.g., enactment) of interventions.

Limitations

Videotaped observations of dyadic mealtime interactions from prior clinical trials were used. The selection of response option of “never (0 time, not observed)” for each item in a mealtime care scenario did not convey whether the unobserved behavior was applicable in the scenario. Concurrent validity was tested using RBS developed to assess generic (rather than mealtime-specific) relational care. While some videotaped observations were clustered at the staff and resident levels, the clustering effects were not adjusted for due to small sample sizes, which may influence psychometric estimates.

Directions for Future Research

This was the initial study that psychometrically tested the MES as a newly developed measure to assess caregiver mealtime engagement. Future research is needed to test its reliability and validity in a larger, more diverse sample of caregivers in different care settings. First, future testing may (a) use observations of varying mealtime care scenarios beyond 1:1 staff–resident interactions (e.g., 1:2, 2:1), (b) examine the feasibility of “not applicable” as a response option for each item, and (c) examine the feasibility of using MES based on direct onsite observations of one entire meal.

Second, future testing of MES may focus on generating estimates of criterion-related validity (i.e., concurrent and predictive validity) and construct validity (i.e., convergent and divergent validity) using mealtime-specific measurements for staff behaviors (e.g., duration of mealtime assistance provided) and resident outcomes (e.g., mealtime difficulties and intake). Future testing also needs to establish structural validity using factor analyses to identify the factor structure and dimensionalities using larger samples. The MES items were developed to address variations in resident cognitive and functional abilities, caregiving approaches, and dining environments based on the Social Ecological Model. Findings of factor analyses will inform the fit of the factor structure to the conceptual basis.

Third, future testing may examine item functioning and appropriateness of response options using Item Response Theory (IRT) approaches. The IRT approaches will identify potentially redundant items as well as items with inadequate discriminations to measure caregiver mealtime engagement along the continuum. The IRT approaches also provide insight into the appropriateness of the binary versus polytomous response options. Findings of the IRT approaches may inform the revision of items and response options for the MES.

Conclusions

Dementia places residents at high risk for mealtime difficulties and low intake, demanding optimal engagement from caregivers. We developed the 18-item MES, an observational instrument to assess caregiver mealtime engagement, and provided preliminary psychometric evidence. Further testing is needed among larger, more diverse caregiver samples in different care settings to accumulate evidence and expand its use.

Funding

The study was supported by the University of Iowa Center on Aging and Aging Mind and Brain Initiative Cluster Pilot Grant (PI: W. Liu).

Conflicts of Interest

None declared.

Submission Declaration and Verification

The work described has not been published previously, is not under consideration for publication elsewhere, its publication is approved by all authors and tacitly or explicitly by the responsible authorities where the work was carried out, and, if accepted, it will not be published elsewhere in the same form, in English or in any other language, including electronically without the written consent of the copyright holder.

Author Contributions

W. Liu contributed to study design, literature review, video screening and coding, data collection, data synthesis and analysis, interpretation of findings, and preparation and revision of the manuscript. M. Batchelor provided access to her P30 video data, was a member of the expert panel to review the items, and contributed to manuscript revision. K. Williams provided access to her R01 video data, was a member of the expert panel to review the items, and contributed to manuscript revision. All authors meet the criteria for authorship and have approved the final article. All those entitled to authorship are listed as authors.