Dyadic interactions and physical and social environment in dementia mealtime care: a systematic review of instruments

Abstract

Using valid instruments to measure dyadic interactions and physical and social environment during mealtime care of persons with dementia is critical to evaluate the process, fidelity, and impact of mealtime interventions. However, the characteristics and quality of existing instruments remain unexplored. This systematic review described the characteristics and synthesized psychometric quality of instruments originally developed or later modified to measure mealtime dyadic interactions and physical and/or social dining environment for people with dementia, based on published reports between January 1, 1980, and December 31, 2020. We identified 26 instruments: 17 assessed dyadic interactions, one assessed physical environment, and eight assessed physical and social environment. All instruments were used in research and none in clinical practice. All instruments were observational tools and scored as having low psychometric quality, except for the refined Cue Utilization and Engagement in Dementia (CUED) mealtime video‐coding scheme rated as having moderate quality. Reasons of low quality are the use of small samples compared with the number of items, limited psychometric testing, and inadequate estimates. All existing tools warrant further testing in larger diverse samples in varied settings and validation for use in clinical practice. The refined CUED is a potential tool for use and requires testing in direct on-site observations.

Graphical Abstract

This systematic review described the characteristics and synthesized psychometric quality of instruments originally developed or later modified to measure mealtime dyadic interactions and physical and/or social dining environment for people with dementia, based on published reports between January 1, 1980, and December 31, 2020. We identified 26 instruments: 17 assessed dyadic interactions, one assessed physical environment, and eight assessed physical and social environment. All instruments were observational tools and scored as having low psychometric quality, except for the refined Cue Utilization and Engagement in Dementia mealtime video‐coding scheme rated as having moderate quality.

Introduction

Mealtime is one of the most basic activities of daily living (ADLs) and plays an important role in maintaining social interactions as well as fundamental health needs, such as food intake, hydration, nutrition, and function.¹ Persons with dementia commonly experience mealtime difficulties, which are the functional, cognitive, and behavioral symptoms that interfere with eating.^{2, 3} Mealtime difficulties can result in inadequate food intake⁴ and subsequent malnutrition and dehydration,⁵ which can further lead to increased risks for infection, weight loss, decreased quality of life, and increased morbidity and mortality.⁶ Despite the increased risks and consequences of mealtime difficulties and inadequate food intake, optimal mealtime care is often not provided to people with dementia.⁷

Person-centered care is “a philosophy of care built around the needs of the individual and contingent upon knowing the person through an interpersonal relationship,”⁸ and is highly recommended as the fundamental principle for optimal dementia care practice.⁹ Support of mealtime and other ADLs following person-centered care has been identified as one of the nine goals of quality dementia care in the Alzheimer’s Association’s Dementia Care Practice Recommendations⁹. Particularly, person-centered mealtime care practice should attend to “individualized abilities, likes, and dislikes” as well as “dignity, respect and choice; the dining process; the dining environment; health and biological considerations; adaptations and functioning; and food, beverage and appetite.” Following the recommendations, person-centered dementia mealtime care that features individualized, multifaceted, and person-oriented care can be achieved through the RECIPE principles: (1) showing Respect; (2) creating Environment; (3) offering Choices; (4) supporting Independence; (5) acknowledging Preferences; and (6) Maintaining Engagement.^{7, 10–12} These principles are important foundations for the development and evaluation of innovative person-centered mealtime care interventions.

In this review, dyadic (staff-resident) interactions are defined as the features and quality of verbal and nonverbal communications between people with dementia and their formal/informal caregivers^{12, 13}. Physical and social environment are defined as the features and quality of the surroundings, atmosphere, stimuli, as well as food and meal-related items in the dining locations where people with dementia eat and/or receive mealtime care.^{10, 14} Dyadic interactions and physical and social environment during mealtime care are key elements of and fundamental pathways to person-centered mealtime care, which further result in improved individual outcomes. For example, caregivers communicate with people with dementia in a respectful way to check preferences, offer choices, and engage them in eating and social conversations. Recent work shows that positive dyadic interactions and high-quality social/physical environment stimuli are associated with reduced mealtime challenging behaviors and increased food intake in people with dementia.^{10–13, 15–18} Both dyadic interactions and physical and social environment are important, modifiable factors that influence resident mealtime behaviors and food intake, and therefore have been promising targets of person-centered mealtime care interventions.^{19, 20}

While the long-term goal of a person-centered mealtime care intervention is commonly optimization of clinically relevant outcomes at the resident, caregiver, facility, and/or societal levels, dyadic interactions and physical and social environment as the target of the intervention are important process outcomes to evaluate the fidelity of interventions. However, dyadic interactions and physical and social environment during mealtime care involve multiple factors at the individual, caregiver, and environmental levels, and can be complex, dynamic, interactive, and evolving nonlinearly or sequentially.^{7, 12, 13} The use of psychometrically sound instruments to measure dyadic interactions and physical and social environment is critical to evaluate the process, fidelity, as well as impacts of mealtime interventions targeting dyadic and environmental factors. However, the characteristics and psychometric quality of existing instruments have not been synthesized or compared.

Objectives

The purpose of this systematic review was to synthesize the characteristics and psychometric quality of existing instruments that were developed and/or used to measure dyadic interactions and physical and/or social environment during mealtime care for persons with dementia. Findings of the study will identify evidence as well as gaps in the development, testing, and use of the instruments, which further guide the use of appropriate and valid instruments to assess outcomes of interest in dementia mealtime care research as well as inform future research in the development, refinement, and validation of relevant instruments.

Methods

Study design, data sources, and search strategy

This systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Guideline: the PRISMA Statement.²¹ We searched five electronic databases (i.e., PubMed, CINAHL, AgeLine, PsychInfo, and Cochrane Library) and bibliographies of eligible records for peer-reviewed scholarly records published in English between January 1, 1980, and June 30, 2019. Keywords and matched subjects included dementia, Alzheimer*, feed*, eat*, meal*, and intake. An example of the search strategy in CINAHL was provided (Table 1). Follow-up searches were conducted using the same five databases and search strategies between June 30, 2019, and December 31, 2020. After eligible instruments were identified from these searches, we confirmed the full names and abbreviated names (if available) of all the eligible instruments based on the search results. Then we conducted additional searches through two additional electronic resources (i.e., Health and Psychosocial Instruments and Google Scholar) using the full names and abbreviated names (if available) of all the eligible instruments with the last search dated on December 31, 2020. We also searched for records that cited the original studies in which eligible instruments were developed and/or used. These additional searches aimed to retrieve an inclusive list of records that modified, used, and/or tested eligible instruments, from which we could extract all published data possible, including grey literature if relevant, as evidence to evaluate the psychometric quality of eligible instruments. The review protocol was predetermined and was not registered.

Table 1.

Search strategy and filter criteria using CINHAL

Concept	Keywords/subject terms
Dementia	(MH “Dementia+”) OR “dementia” OR (MH “Frontotemporal Dementia+”) OR (MH “Dementia, Vascular+”) OR (MH “Dementia, Multi-Infarct”) OR (MH “AIDS Dementia Complex”) OR (MH “Lewy Body Disease”) OR (MH “Dementia, Senile+”) OR (MH “Dementia, Presenile+”) OR (MH “Alzheimer’s Disease”) OR (MH “Lewy Body Disease”) OR “alzheimer” OR (MH “Alzheimer’s Disease”) OR “alzheimer’s disease” OR (MH “Lewy Body Disease”) OR (MH “Alzheimer’s Disease”) OR (MH “Lewy Body Disease”) OR “alzheimers”  AND “feeding”  OR (MH “Eating Behavior+”) OR “feeding behavior” OR (MH “Eating”) OR “eating” OR (MH “Eating Behavior”) OR (MH “Meals+”) OR “meals” OR (MH “Food Assistance”) OR (MH “Meals”) OR “mealtime” OR (MH “Nutrition”) OR “nutrition” OR (MH “Food Intake+”) OR “food intake” OR (MH “Nutritional Status: Food & Fluid Intake (Iowa NOC)”) OR (MH “Functional Food”) OR (MH “Food Assistance”) OR (MH “Nutritional Status: Nutrient Intake (Iowa NOC)”) OR (MH “Nutritional Status: Food & Fluid Intake (Iowa NOC)”) OR (MH “Fluid Intake-Output Measures”) OR (MH “Dietary Reference Intakes”) OR “nutritional intake” OR (MH “Drinking Behavior”) OR “drinking”  OR (MH “Drinking Behavior”) OR “drinking” OR feeding OR eating OR feed OR eat OR hand feeding OR oral feeding OR mealtimes OR mealtime OR nutrition OR nutritional OR oral intake OR food intake OR food OR meals OR meal
Mealtime, eating, feeding, and food intake

Filter criteria: Publication date between January 1, 1980, and December 31, 2020; humans; english; middle aged + aged (45+ years).

Inclusion and exclusion criteria

Records were eligible if they reported any instrument that was developed, tested, and/or used to measure any of the three concepts of interest (i.e., dyadic interactions and physical and social environment) during mealtime care of persons with dementia in primary research or original studies. All types of care settings (e.g., home care, long-term care, and hospitals) were included. Records were excluded if they focused on nonmealtime activities, people without dementia, or other types of publications than primary research (e.g., reviews, commentaries, and editorials). After we identified eligible records, instruments described in the records were identified and extracted. Instruments were eligible if they were originally developed (1) to measure any of the three concepts of interest or (2) in nonmealtime activities or people without dementia and later modified and/or tested to measure the concepts of interest in dementia mealtime care.

Study selection and data extraction

Following the inclusion and exclusion criteria, two independent reviewers screened all retrieved records by title, abstract, and full text to identify eligible instruments. The two reviewers compared their results and discussed discrepancies to reach an agreement. When discrepancies were not solved by the two reviewers, the first author was involved in reviewing the records and/or instruments and make final decisions through discussion with the two reviewers. Eight characteristics of each eligible instrument were described using a data extraction worksheet: (1) development process; (2) the concept or construct the instrument operationalizes; (3) sample and setting the instrument was used or tested in; (4) administration method; (5) description of items; (6) scoring format and interpretation; (7) reliability; and (8) validity.

Assessment of psychometric quality

In this review, we used the Psychometric Assessment for Self-report and Observational Tools (PAT, Table 2) to evaluate the psychometric quality of eligible instruments. The PAT tool was developed following the classical test theory (CTT), a commonly used measurement framework that assumes the true score cannot be directly observed and can only be assessed indirectly through the observed score with random measurement error.²² Following this framework, a total of 12 CTT-based quality properties were included in the PAT tool: (1) ratio of the sample size to the number of items; (2) internal consistency; (3) test–retest reliability; (4) intra-rater reliability; (5) inter-rater reliability; (6) content validity; (7) concurrent validity; (8) predictive validity; (9) divergent/discriminant validity; (10) convergent validity; (11) known group difference; and (12) structural validity. The definitions and criteria of the quality properties were initially developed on the basis of published psychometric assessment criteria^23–26 and a review of the literature.^{22, 26–30} The criteria of quality properties were further refined to include different types of statistics that were commonly used across studies to demonstrate certain psychometric properties. For example, different types of reliability coefficients (e.g., intraclass correlation, Kappa, Pearson’s correlation coefficient, and Spearman’s rank correlation coefficient) were incorporated in the criteria for all the three reliability properties.

Table 2.

The psychometric assessment for self-report and observational tools (PAT)

#	Properties	Definition	Score	Criteria
1	Sample size: the number of items	The size of the sample per item being tested	2	≥10: 1
			1	5:1–10:1
			0	<5:1
2	Homogeneity: internal consistency	The extent to which items in a (sub)scale are intercorrelated, thus measuring the same construct/concept	2	If 0.70 ≤ Cronbach’s alpha ≤ 0.90
			1	If Cronbach’s alpha > 0.90 (indicates potential redundancy) or .60 ≤ Cronbach’s alpha < 0.70.
			0	If Cronbach’s alpha < 0.60 or no information is provided
			-	Not applicable (e.g., instruments with only 1 item)
3	Test–retest reliability	The level of agreement on item responses between two or more raters at the same time (applicable for instruments administrated through self- or proxy report)	2	If reliability coefficient (e.g., ICC, Kappa, r, and rs) ≥ 0.80.
			1	If 0.60 ≤ reliability coefficient < 0.80 or some coefficients are ≥ 0.80 but others are < 0.60
			0	If reliability coefficient < 0.60 or no information is provided
			-	Not applicable (e.g., instruments administrated through using raters)
4	Intra-rater reliability	The consistency of item responses between one rater’s two assessments over time (applicable for instruments administrated through using raters)	2	If reliability coefficient (e.g., ICC, Kappa, r, and rs) ≥ 0.80
			1	If 0.60 ≤ reliability coefficient < 0.80 or some coefficients are ≥0.80 but others are <0.60
			0	If reliability coefficient <0.60 or no information is provided
			-	Not applicable (e.g., instruments administrated through self- or proxy report)
5	Inter-rater reliability	The consistency of item responses over time (applicable for instruments administrated through using raters)	2	If reliability coefficient (e.g., ICC, Kappa, r, and rs) ≥0.80
			1	If 0.60 ≤ reliability coefficient < 0.80 or some coefficients are ≥0.80 but others are <0.60
			0	If reliability coefficient <0.60 or no information is provided
			-	Not applicable (e.g., instruments that are administrated through self- or proxy report)
6	Content validity	The degree to which elements of a measure are relevant to and representative of the targeted construct for a particular assessment purpose	2	All aspects, including the instrument aim, target population, measured constructs, AND the item selection process involved the review by target population or experts (e.g., the developer), AND were clearly described (in reviewer’s opinion) with evidence of excellent CVI scores (i.e., I-CVI ≥ 0.78, S-CVI/UA ≥ 0.80, and S-CVI/Ave ≥ 0.90)
			1	Most or all aspects, including the instrument aim, target population, measured constructs AND the item selection process involved the review of target population or experts (e.g., the developer), AND were described in moderate clarity (in reviewer’s opinion) with fair to excellent CVI scores (I-CVI = 0.67–0.78, 0.70 ≤ S-CVI/Ave < 0.90, 0.70 ≤ S-CVI/UA < 0.80) of CVI scores
			0	Some or all aspects, including the instrument aim, target population, measured constructs with the item selection process involved the review of target population or experts (e.g., the developer), were poorly described or not described at all (in reviewer’s opinion) with unacceptable CVI scores (I-CVI ≤ 0.67, S-CVI/Ave < 0.70, S-CVI/UA < 0.70) or without CVI score
7	Criterion validity: concurrent validity	The extent to which the construct measure under development/testing and a criterion measure collected simultaneously or concurrently are correlated	2	If correlation is acceptable to high [correlation coefficient (i.e., r and rs) ≥ 0.60, all P’s < 0.05 based on t-test, ANOVA, or chi-square test, or all 95% CIs are in the significant range], according to the “gold standard” or acceptable according to a “silver standard” and sensitivity/specificity is determined to be acceptable
			1	If correlation is moderate to acceptable [0.40 ≤ correlation coefficient (i.e., r and rs) < 0.60, all P values are ranged from 0.05 to 0.10, or some P values/95% CIs are significant, and others are not significant] according to the “gold standard” or acceptable according to a “silver standard”
			0	If correlation is low [correlation coefficient (i.e., r and rs) < 0.40, all P > 0.10, all 95% CIs are not in the significant range, or no information is provided]
			-	Not applicable (e.g., no comparator is identified by authors from literature as criterion for the instrument being tested)
8	Criterion validity: predictive validity	The ability of a measure to effectively predict some subsequent and temporally ordered criterion	2	If correlation is acceptable to high (|correlation coefficient| ≥ 0.60, all P’s < 0.05 based on t-test, ANOVA, or chi-square test, or all 95% CIs are in the significant range), according to the “gold standard” or acceptable according to a “silver standard” and sensitivity/specificity is determined to be acceptable
			1	If correlation is moderate to acceptable (0.40 ≤ |correlation coefficient| < 0.60, all p values are ranged from 0.05 to 0.10, or some P values/95% CIs are significant, but others are not significant) according to the “gold standard’ or acceptable according to a “silver standard”
			0	If correlation is low (|correlation coefficient| < 0.40), all P’s > 0.10 or all 95% CIs are not in the significant range, or no information is provided
			-	Not applicable (e.g., no comparator is identified by authors from literature as criterion for the instrument being tested)
9	Construct validity: convergent validity	The extent to which independent measures of theoretically related constructs converge or are highly correlated	2	If correlation is acceptable to high (|correlation coefficient| ≥ 0.60, all P’s < 0.05 based on t-test, ANOVA, or chi-square test, or all 95% CIs are in the significant range)
			1	If correlation is moderate to acceptable (0.40 ≤ |correlation coefficient| < 0.60, all p values are ranged from 0.05–0.10, or some P values/95% CIs are significant, but others are not significant)
			0	If correlation is low (|correlation coefficient| < 0.40), all P’s > 0.10 or all 95% CIs are not in the significant range, or no information is provided
			-	Not applicable (e.g., no comparator is identified by authors from literature for this type of validity for the instrument being tested)
10	Construct validity: divergent validity	The extent to which independent measures of theoretically unrelated or distinct constructs diverge or are not correlated	2	If correlation is low (|correlation coefficient| < 0.40), all P’s > 0.10 or all 95% CIs are not in the significant range
			1	If correlation is moderate to acceptable (0.40 ≤ |correlation coefficient| < 0.60, all P values are ranged from 0.05 to 0.10, or some P values/95% CIs are significant, but others are not significant)
			0	If correlation is acceptable to high (|correlation coefficient| ≥ 0.60, all P’s < 0.05 based on t-test, ANOVA, or chi-square/Fisher’s exact test, or all 95% CIs are in the significant range), or no information is provided
			-	Not applicable (e.g., no comparator is identified by authors from literature for this type of validity for the instrument being tested)
11	Construct validity: known different groups	The extent to which a measure differs as predicted/hypothesized between groups with different levels of the trait being measured	2	If the scale differentiates very well (all P’s < 0.05 based on t-test, ANOVA, or chi-square/Fisher’s exact test, or all 95% CIs are in significant range) between different groups on the level of measured construct
			1	If the scale differentiates moderately well (all p values are ranged from 0.05–0.10, or some P values/95% CIs are significant, but others are not significant) between different groups on the level of measured construct
			0	If the scale does not differentiate (all P’s > 0.10 or all 95% CIs are not in the significant range), or no information of P values is provided
			-	Not applicable (e.g., no group difference is identified by authors from literature for the instrument being tested)
12	Construct validity: structural validity	Whether a measure assesses a unidimensional construct or multiple domains/factors of a construct	2	Both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were done, providing confirmed factor structure of the instrument with acceptable model fit
			1	EFA was done resulting in explored factor structure; CFA was not done to confirm the explored factor structure in the population of interest
			0	Both EFA and CFA were not performed (principal component analysis is not considered equivalent as factor analysis)
			-	Not applicable (e.g., instruments with less than three items that may not allow for factor analysis)

Note: CVI, content validity index; I-CVI, item-content validity index; ICC, intraclass correlation; r, Pearson correlation coefficient; r_s, Spearman’s rank correlation coefficient; S-CVI/UA, scale-content validity index/universal agreement; S-CVI/Ave, scale-content validity index/average.

Reproduced from Liu, Kim, et al., 2021 with automatic permission following the copyright and permission guidelines of Elsevier (publisher of the International Journal of Nursing Studies where Liu, Kim, et al., 2021 was published). Both Elsevier and John Wiley & Sons Inc. (publisher of Annals of the New York Academy of Sciences) are on the updated list of STM publishers (International Association of Scientific, Technical and Medical Publishers), who have opted out of notifications for permission requests within the specified limits (use no more than three figures or tables from a journal article published by STM publishers).

Among the 12 quality properties in the PAT tool, test-retest reliability is only applicable for self-report instruments. Intra- and inter-rater reliability are only applicable for observational instruments that involve raters. Each of the 12 quality properties is scored from 0 to 2. The total quality assessment score for an instrument is computed by adding scores of all the quality properties. The total score for self-report instruments ranges from 0 to 20 (low quality: 0–6; moderate quality: 7–13; and high quality: 14–20), and that for observational instruments ranges from 0 to 22 (low quality: 0–7; moderate quality: 8–15; and high quality: 16–22). In this review, the second author evaluated the psychometric properties of eligible instruments using the PAT tool. Results were reviewed by the first author, and disagreements were discussed among the two authors to reach a consensus.

We applied a conservative rating system when a specific quality property of any instrument received two or more different scores based on results from one or more records, we used the lower or lowest score for this quality property of the instrument. While the use of the lower or lowest score to represent a specific quality property may underestimate an instrument’s specific quality based on existing evidence, this conservative approach will provide a baseline psychometric estimate that can be most easily replicated and maintained in the future use of this instrument in the dementia population. The use of the higher or highest score as an alternative option was considered but not selected because the higher or highest score obtained for an individual research study may not be representative of all the existing/published evidence, and it would require more efforts to replicate and maintain such high-quality estimates in future research studies.

The PAT, as a newly developed psychometric quality assessment tool, has four advantages over previously published quality assessment criteria.^23–26 These advantages are (1) inclusion of the ratio of the sample size to the number of items as one quality property; (2) inclusion of different types of reliability and validity as separate quality properties; (3) ability to assess psychometric properties of both observational and self-report instruments; and (4) use of a simple, easy to use numerical scoring system to obtain a total score for each instrument to facilitate comparison of overall psychometric quality across instruments. The PAT was used in our recent systematic review of instruments that assess mealtime caregiving knowledge, attitudes, skills, and behaviors for people with dementia and showed adequate feasibility and usability.³¹

Results

In total, 9728 records were retrieved, among which 80 full texts were included in the review (Fig. 1). From the 80 scholarly records, 26 eligible instruments were identified, of which 17 assessed dyadic interactions, one assessed physical environment, and eight assessed both physical and social environment during mealtime care (Table S1, online only). Fifteen of the 26 instruments were reported in two or more full-text records. Four of the 80 full-text records reported two instruments (mealtime scan (MTS)/MTS+ and Dining Environment Assessment Protocol (DEAP)). Among the 26 instruments, 22 were originally developed among the dyads of direct care providers and persons with dementia during mealtime care, and four were originally developed in other populations (i.e., mother–infant and long-term care staff–resident dyads and romantic/married couples) during feeding, mealtime care, or casual conversations, and later used in persons with dementia and their caregivers during mealtime.

Figure 1.

PRISMA flowchart for instruments developed or used to assess dyadic interactions, physical environment, and social environment during mealtime care for persons living with dementia.

Characteristics of instruments

Dyadic interactions.

Target population.

Among the 17 instruments identified to assess mealtime dyadic interactions, 16 were used in people with a specified dementia diagnosis, and one was developed and tested in long-term care residents with and without dementia (Mealtime Social Interaction Measure for Long-Term Care (MSILTC)³²). Among the 16 instruments that were used in dementia population, 13 were originally developed in persons with dementia and their caregivers or partners, and three were not developed for but later used in persons with dementia and their caregivers (modified Nursing Child Assessment Scale (NCAS),^{33, 34} Feeding Assistance Observational Protocol (FAOP),^35–39 and Marital Interaction Coding System-IV (MICS-IV)^{40, 41}). Among the 13 instruments that were originally developed in dementia population and their caregivers or partners, eight were originally developed for mealtime care activities (Feeding Traceline Technique (FTT),^42–44 Trouble Source Repair (TSR),^45–47 Priefer and Robbins’ observation tool,⁴⁸ Altus et al.’s observation checklist,⁴⁹ Feeding Cycle Recording (FCR),⁵⁰ Levy-Storms et al.’s observation tool,⁵¹ the refined CUED mealtime video-coding scheme,^{11–13, 16–18, 52} and Gilmore-Bykovskyi’s coding scheme^{53, 54}). Four of the 13 instruments were originally developed in ADLs, including but not limited to mealtime (Armstrong-Esther and Browne’s observation tool,^55–57 Hallberg et al.’s audio analysis,^58–61 Dementia Care Mapping (DCM),^62–64 and Small et al.’s observation tool⁶⁵). One of the 13 instruments (Gentry et al.’s coding scheme^{66, 67}) was developed to assess dyadic interactions between persons with dementia and researchers and later used in persons with dementia and their family caregivers during mealtime.

Development process.

Of the 17 instruments, 12 were developed based on extensive literature reviews and/or specific conceptual frameworks, or adapted from existing instruments to dementia population (FTT, TSR, Gentry et al.’s and Gilmore-Bykovskyi’s coding schemes, FAOP, FCR, DCM, Small et al.’s observation tool, the refined CUED mealtime video-coding scheme, Modified NCAS, MISC-IV, and MSILTC). The use of literature reviews and/or frameworks was not reported in the development process of five instruments (Armstrong-Esther and Browne’s observation tool, Hallberg et al.’s audio analysis, Altus et al.’s observation checklist, Priefer and Robbins’ observation tool, and Levy-Storms et al.’s observation tool).

Sample/setting and administration method.

Among the 17 instruments, 14 were developed and/or used in residents and staff in long-term care settings, and three in individuals and their family caregivers in home settings (TSR, MISC-IV, and Gentry et al.’s coding scheme). The sample size (e.g., the number of participants or mealtime observations) ranged from three (residents) to 2713 (observations). Seven instruments were administered through observing videos by raters (FTT, TSR, Gilmore-Bykovskyi’s coding scheme, Priefer and Robbins’, Small et al.’s, and Levy-Storms et al.’s observation tools, and the refined CUED mealtime video-coding scheme). Seven instruments were administrated through direct on-site observation by raters (Armstrong-Esther and Browne’s observation tool, modified NCAS, Altus et al.’s observation checklist, DCM, FAOP, FCR, and MSILTC). One instrument was administrated through nonparticipant and audio observation (Hallberg et al.’s audio analysis), and two instruments through video and audio observation (MISC-IV and Gentry et al.’s coding scheme).

Items and scoring.

The number of items in each of the 17 instruments varied from 1 to 76. The frequency of behaviors in the 17 instruments was scored using different formats: (1) the raw number of behaviors was counted in 14 instruments (Armstrong-Esther and Browne’s observation tool, Hallberg et al.’s audio analysis, FTT, TSR, Gilmore-Bykovskyi’s coding scheme, Gentry et al.’s coding scheme, MISC-IV, observation checklist, FAOP, FCR, Priefer and Robbins’, Small et al.’s, and Levy-Storms et al.’s observation tools, and the refined CUED mealtime video-coding scheme), (2) binary scoring was used in four instruments (Hallberg et al.’s audio analysis, modified NCAS, observation checklist, and FAOP), and (3) Likert scoring was used in two instruments (DCM and MSILTC). In addition, three of the 17 instruments were scored used more than one scoring formats: (1) Altus et al.’s observation checklist and FAOP used both the raw number of behaviors and binary scoring, and (2) Hallberg et al.’s audio analysis was originally scored using the raw number of behaviors, and later was scored using the ratio of the frequency of behaviors in people with dementia to the frequency of behaviors in caregivers (range of ratio: 1:1 to 5:1).

Psychometric properties.

Among the 17 instruments, one was not tested for any psychometric properties based on the reports included in this review (Small et al.’s observation tool), and the other 16 instruments were tested for one to five types of psychometric properties. Particularly, nine instruments were tested for three to five types of psychometric properties (modified NCAS (four types), the refined CUED mealtime video-coding scheme (five types), TSR (four types), MISC-IV (three types), DCM (three types), FAOP (three types), Hallberg et al.’s audio analysis (three types), FTT (three types), and Gilmore-Bykovskyi’s coding scheme (three types)). Three instruments were tested for one to two types of validity and were not tested for any reliability (Armstrong-Esther and Browne’s observation tool (two types), FCR (two types), Priefer and Robbins’ (1997) observation tool (one type)). Levy-Storms et al.’s observation tool was tested for inter-rater reliability and known group difference validity. Three instruments (Altus et al.’s observation checklist, Gentry et al.’s coding scheme, and MSILTC) were only tested for inter-rater reliability.

Physical environment.

The DEAP was developed to assess the physical environment during dementia mealtime care based on specific frameworks, existing instruments, and pilot testing.^{4, 68–73} The use of this instrument was reported in seven records, of which only three types of psychometric properties (inter-rater reliability and predictive and convergent validity) were reported. The unit of observation varied from the resident with dementia, the mealtime observation, to the dining area. The sample size ranged from 10 (dining areas) to 639 (residents) in long-term care settings. Different items in the DEAP were scored using different formats, including a binary scoring (0–1), and 3- (0–2) and 4-category (0–3) Likert scoring.

Physical and social environment.

Target population.

All eight instruments were originally developed for caregivers and persons with dementia during mealtime care, except for the Person–Environment Apathy Rating (PEAR)-Environment subscale, which was originally developed to assess the quality of environmental stimulation provided to persons with dementia in ADLs (including but not limited to mealtime) and was later used in mealtime.^{10, 74–78}

Development process.

Among the eight instruments, six were developed based on extensive literature review, expert review, specific frameworks, and/or field and pilot testing. Five of the eight instruments that were reported in the original development studies were not reported for use in any other scholarly records (Eating Behavior Observation Scale (EBOS),⁷⁹ Kayser-Jones and Schell’s observation tool,⁸⁰ Roder-Allen et al.’s observation tool,⁸¹ meal observation tool,⁸² and Hung and Chaudhury’s observation tool⁸³).

Sample/setting and administration method.

Of the eight instruments, two were developed in relatively large samples (N > 300), including structured meal observation (SMO)^84–87 and MTS/MTS+.^{68–70, 72, 88–91}. Six instruments were tested in smaller samples ranging from 9 to 131. All the eight instruments were developed and tested in long-term care settings. All instruments were administered using on-site observations except for the PEAR–Environment subscale, which was administered using both videotaped and on-site observations.

Items and scoring.

The number of items ranges from 6 to 40. A Likert scoring format was used in ratings of four instruments (PEAR–Environment subscale, EBOS, SMO, and MTS/MTS+). For the other four instruments, a qualitative approach was used to assess the physical and/or social environment (Kayser-Jones and Schell’s observation tool, Roder-Allen et al.’s observation tool, Meal observation tool, Hung and Chaudhury’s observation tool).

Psychometric properties.

Among the eight instruments, only four were tested for reliability and/or validity. The PEAR-Environment subscale was tested most extensively for six types of psychometric properties, followed by the MTS/MTS+ tested for five types, the SMO for three types, and the EBOS for two types.

Psychometric quality assessment

Table S2 (online only) shows the psychometric quality of all 26 instruments. Twenty-five instruments had low psychometric quality (total scores range: 0–6), and one instrument had moderate quality (total score: 8). Among all the 17 instruments that assessed mealtime dyadic interaction, the refined Cue Utilization and Engagement in Dementia (CUED) mealtime video-coding scheme was scored 8, and TSR was scored 6, followed by three instruments that were scored 5 (i.e., FTT, FAOP, FCR). While all the five instruments showed moderate-to-good psychometric evidence in two to five types of psychometric properties (intra- and inter-rater reliability, predictive, convergent, and divergent validity, and/or known group difference), evidence was obtained from small samples for three of the instruments (i.e., TSR, the refined CUED mealtime video-coding scheme, and FTT). While the other two instruments (i.e., FAOP and FCR) were tested in large samples, the limited number of psychometric properties were evaluated. Among all the nine instruments that assessed physical and/or social dining environment, two instruments were scored 4 (i.e., PEAR-Environment subscale and SMO). Each of the two instruments showed moderate-to-good evidence in two to three types of psychometric properties (i.e., internal consistency, intra- and inter-rater reliability, and predictive validity) using varied sample sizes.

Table 3 summarized scores of all psychometric quality properties for the 26 instruments. The reasons for low-to-moderate psychometric quality was the use of a small sample size, limited psychometric testing or lack of testing of fundamental properties, and inadequate psychometric estimates. For example, the ratio of the sample size to the number of items was insufficient for 16 instruments. Reliability was barely established among all the instruments except inter-rater reliability, for which 14 instruments had moderate-to-good evidence (quality property score = 1 or 2). The estimates for internal consistency were inadequate or not reported for 25 instruments. All 26 instruments were administrated through observations; however, 23 were not tested for or showed low evidence of intra-rater reliability. Some types of validity were tested less intensively than the other types. All the instruments were neither tested for nor showed low evidence of content, concurrent, or validity. Only one instrument was tested for and showed good evidence of divergent validity. Only three instruments were tested for and showed moderate-to-good evidence of predictive validity. Eight instruments were tested for and showed moderate-to-good evidence in convergent validity. Twelve instruments were tested for and showed moderate-to-good evidence for known group differences.

Table 3.

Summary of quality assessment criteria scores for instruments developed and/or used in persons with dementia and their caregivers during mealtime care (N = 26 instruments)

Criteria score	Sample size: no. of items	Reliability, n (%)			Validity, n (%)
		Internal consistency	Intra-rater reliability	Inter-rater reliability	Content validity	Concurrent validity	Predictive validity	Convergent validity	Divergent validity	Known group difference	Structural validity
0	16 (61.5)	25 (96.2)	23 (88.5)	12 (46.2)	26 (100)	26 (100.0)	23 (88.5)	18 (69.2)	25 (96.2)	14 (54.0)	26 (100)
1	8 (30.8)	1 (3.8)	1 (3.8)	10 (38.5)	0	0	2 (7.7)	5 (19.2)	0	4 (15.3)	0
2	2 (7.7)	0	2 (7.7)	4 (15.3)	0	0	1 (3.8)	3 (11.6)	1 (3.8)	8 (30.7)	0

Discussion

This study is the first that described the characteristics and synthesized the psychometric quality of instruments developed and/or used to assess dyadic interactions and physical and/or social environment during mealtime care for persons with dementia. A total of 26 instruments were identified from 80 studies published between 1985 and 2020. All instruments were identified from reports based on research studies and had been primarily used by researchers. None of the identified instruments have been reported for use in clinical practice. Of all 26 instruments, two thirds (n = 17) assessed dyadic mealtime interactions and one third (n = 8) assessed physical and/or social dining environment; more than two thirds (n = 19) were developed based on extensive literature and/or theoretical frameworks; the majority (n = 21) were originally developed in dementia populations, and the others were later adapted for use in dementia population; most (n = 23) were developed and/or used in long-term care settings, and the others (n = 3) were used in home settings.

Noticeably, 11 of the 26 instruments were administered through video and/or audio observations of mealtime care scenarios. While direct on-site observations are commonly used to administer observational tools, the use of video/audio observations has been increasing as an emerging, innovative methodology in dementia mealtime care research to address questions that may not be achievable through direct on-site observations.^{52, 92} While this methodology can be time- and labor-intensive, it is an advanced approach in that it allows for (1) repetitive viewing and coding of multiple factors and in-depth analyses to better understand the complex and dynamic interactions and dining environment^{11–13, 16–18}, and (2) coding of time when an event (e.g., behavior, intake, environment stimulation) occurs to address specific questions such as change of event patterns over time and temporal relationships between events (e.g., how caregiver-individual mealtime behaviors are temporally related to individual food intake).^{15, 53}

Psychometric quality

In this review, the PAT tool that was developed based on published criteria and extensive review of the literature reported between 1986 and 2018 was used to appraise the psychometric quality of all the eligible instruments published between 1985 and 2020, indicating a good fit of the tool in evaluating these instruments. In our prior review on instruments of dementia mealtime caregiving knowledge, attitude, and skills, 19 instruments were identified, with only one having moderate psychometric quality.³¹ Comparatively, a larger number of instruments of dyadic interactions and physical and social environment (n = 26) were identified, with only one having moderate psychometric quality. Reasons for low-to-moderate quality are the use of small sample size compared with the number of items in the instrument, limited psychometric testing or lack of testing of fundamental properties, and inadequate psychometric estimates for tested properties, which are similar to our prior review of instruments on mealtime caregiving knowledge, attitude, and skills.³¹

The ratio of the sample size to the number of items

The ratio of the sample size to the number of items was included in the PAT tool because the adequate sample size is critical to generate valid psychometric estimates in consideration of the number of items in a specific instrument.^{22, 27}. In this review, the unit of sample for instruments assessing dyadic interactions were mostly the individual, dyad, and/or mealtime observation, compared with the unit of sample for instruments assessing physical and/or social environment that varied from the individual, mealtime observation, to the dining area. However, the use of inadequate samples is common across most included studies (i.e., three residents to 2713 mealtime observations instruments assessing dyadic interactions, 10 dining areas to 639 residents for instruments assessing physical and/or social environment), compared with the number of items the included instruments were consisted of (i.e., 1–76 items for instruments assessing dyadic interactions, 6–40 items for instruments assessing physical and/or social environment).

In this review, the sample sizes were considered acceptable (the ratio of the sample size to the number of items ≥10:1), small (ratio = 5:1–10:1), and insufficient (ratio < 5:1) for 1, 5, and 11 instruments that assess dyadic interactions respectively, compared with one, three, and five instruments that assess physical and/or social environment, respectively. While there is no standardized rule on the minimum ratio, it is recommended that a ratio of 10:1 be acceptable and the higher the better.^{22, 27} While the acceptable sample size was used in the testing and use of only two eligible instruments (i.e., one for dyadic interactions and one for physical and/or social environment), future testing is urgently warranted using larger samples.

Reliability

While all included instruments are observational tools requiring raters’ effort, reliability, which refers to the consistency of a measure, was barely established. Almost all the instruments were not tested for or had inadequate estimates of internal consistency. As a fundamental estimate for homogeneity, internal consistency is recommended for all instruments that has more than one items.^{22, 27} For multitrait instruments that assess multiple domains of a concept/construct or have multiple subscales, internal consistency should be established for each domain or subscale.^{22, 27} Furthermore, intra- and inter-rater reliability were not tested for or showed low evidence in most instruments, raising questions on the stability of the instruments over time and across raters. It is recommended that some or all the types of reliability be tested based on the needs of specific research in targeted populations. For example, if two or more raters are involved in administering one instrument, inter-rater reliability should be established before data collection as well as periodically during data collection.^{22, 27} If one instrument is administered repeatedly by one rater, intra-rater reliability should be established.^{22, 27}

In addition, intra- and inter-rater reliability are two different types of reliability and establishing evidence in one type does not indicate evidence in the other type. Mealtime care scenarios, including dyadic interactions and physical and/or social environment, may change within dyads or residents across meals, increasing difficulty of establishing intra-rater reliability of observational instruments. while the use of videotaped observations is ideal for establishing intra-rater reliability, the use of direct on-site observations is commonly used and acceptable when factors that may influence the concept or construct of interest (e.g., dyadic interactions) are maintained with high similarities or consistency across meals (e.g., same meal type, same caregiver–individual dyad, and same dining location/area).

Validity

Validity refers to the accuracy of an instrument and indicates whether an instrument measures the concept or construct it is supposed to measure.^{22, 27} Validity was also insufficiently established for the included instruments. Particularly, some types of validity were tested more scarcely compared with other types. For example, three types of validity (i.e., content, concurrent, and structural) were not tested for or had inadequate estimates in all the instruments, and the other types (divergent, predictive, and convergent validity, and known group differences,) were tested in one or more instruments. Content validity which is established during the process of instrument development is a basic type of validity that ensures the items are relevant to and representative of the concept/construct being measured.^{22, 27} Surprisingly, the Content Validity Index, which is recommended as a standardized approach to establish content validity,^{29, 30} was not reported for any of the included instruments.

Concurrent validity is a type of criterion validity evaluating to what extent the instrument being tested is correlated with a gold standard measure collected simultaneously that assess the same construct.^{22, 27} One of the reasons that concurrent validity was inadequately tested among eligible instruments is the lack of an appropriate criterion measure for mealtime dyadic interactions and physical/social environment, indicating the need to further validate instruments scored relatively high in this review to accumulate psychometric evidence and establish gold standard measures.

Structural validity, the most straightforward and direct way to evaluate construct validity, is usually examined through exploration and confirmation of the factor structure of an instrument.^{22, 27} This approach usually requires much larger samples to generate valid factor structures, especially for multitrait or -domain instruments or instruments that have multiple subscales which require the use of multifactor models.^{22, 27} Most of the included instruments, whether single-trait or multitrait, were tested in small or insufficient samples, not allowing for the testing of structural validity.

The ability of an instrument to detect clinically relevant changes over time or due to intervention effects (responsiveness) is a critical criterion for selection of instruments, especially in longitudinal and intervention studies. In this review, among the 12 existing instruments that showed moderate-to-good evidence on known group differences, only two were identified being used in intervention studies (i.e., the FAOP for treatment effect of the feeding assistance intervention, the MTS+ for treatment effect of relational-centered mealtime care intervention). Overall, content, concurrent, and structural validity, and responsiveness are fundamental indicators of validity and existing instruments warrant attention in future testing in cross-sectional as well as longitudinal and intervention studies.

Implications

Measuring mealtime dyadic interactions and physical and/or social dining environment are important to evaluate the process and fidelity of behavioral interventions that focus on caregiver training, mealtime assistance, and/or dining environment, as well as to examine the impact of these interventions on health-related outcomes of people with dementia. In this review, among the 26 instruments that were identified in 80 published records over the past 35 years, only one was rated with moderate psychometric quality and the rest were rated with low quality, and all instruments warrant further testing in larger, diverse samples in varied settings. This information confirmed that the development and validation of instruments is a time-intensive and continuing process and that these instruments that assess dyadic interactions and physical and/or social environment are still in the testing stage. In addition, none of the identified instruments have been reported for use in clinical practice, indicating the gaps in as well as the need to validate simple, easy-for-use instruments for use in clinical practice. In this review, several instruments that were rated the best among existing instruments may become potential targets for further efforts of validation and/or refinement to accumulate evidence on psychometric quality and/or practicality, which will expand their use in future research and clinical practice.

Among all the identified instruments, the refined CUED mealtime video-coding scheme was the only instrument that showed moderate quality. The refined CUED assessed both verbal and nonverbal behaviors from staff and residents from positive and negative perspectives during the intake process and shows potential for use in the field of dementia mealtime care. For example, the use of the refined CUED and video data can illuminate the complexity and intercorrelation of staff-resident mealtime interactions and their impact on resident intake, as well as facilitate the understanding of the temporal relationships among staff-resident mealtime behaviors and resident intake.^{11–13, 15–18, 52}

In addition to the refined CUED, TSR that assess dyadic mealtime interaction was scored highest among all the rest instruments that showed low quality. While the two instruments showed moderate-to-good evidence in both reliability and validity, testing was limited to certain types of properties (i.e., intra- and/or inter-rater reliability, convergent validity, and known group difference). There is potential for improvement following future testing of additional properties in larger diverse samples (i.e., internal consistency, and content, concurrent, and structural validity for both instruments; intra-rater reliability for the refined CUED; and predictive and divergent validity for TSR).

Comparatively, the instruments that assess physical and/or social environment were scored lower on psychometric quality and could benefit from additional testing in larger diverse samples. Particularly, the PEAR-Environment subscale that was most extensively tested with mixed and inconsistent evidence and SMO with good predictive validity were the two instruments that scored highest among all the existing instruments that assess physical and/or social environment. While all existing instruments that assess physical and/or social environment need further testing in dementia populations, we suggest the use of these two instruments be carefully considered as appropriate and continuous efforts to validate these two instruments to accumulate their psychometric evidence.

Directions for future research

This review provided seven important directions for future dementia mealtime care research in the assessment of dyadic interactions and physical and/or social environment. First, future testing is needed to accumulate evidence on reliability and validity, especially internal consistency, inter- and intra-rater reliability, and content validity, which are fundamental psychometric properties of observational tools. Second, it is critical that psychometric testing of all properties, including but not limited to structural validity as well as responsiveness over time or to intervention effects, be conducted using larger samples in consideration of the number of items in a specific instrument to generate valid estimates.

Third, most of the identified instruments were developed and/or tested for use in long-term care settings, and the use of instruments may be extended through testing in people with dementia and their family caregivers in home settings. Fourth, when no instrument is identified as appropriate and best to assess specific aspects of dyadic interactions and physical and/or social environment and a need for a new tool raises, rather than developing a completely new tool that involves intensive time and efforts, an alternative may be refining and/or modifying existing instruments and validating fundamental psychometric properties in larger diverse samples.

Fifth, none of the instruments have been reported for use in clinical care to evaluate current mealtime care practice or the impact of evidence-based quality improvement projects. Future research is needed to translate and validate existing instruments into feasible and easy-to-use tools for clinical practice. Sixth, none of the eligible instruments identified in this review were tested using the Item Response Theory (IRT), indicating the need of validating and/or refining existing instruments using IRT-based techniques.

Lastly, the PAT tool used in this review was developed following existing criteria and the CTT framework to evaluate the psychometric quality of eligible, existing instruments. While the tool was theoretically grounded and showed good feasibility and usability in this and prior review,³¹ its own psychometric properties have not been established and may warrant future testing. Noticeably, the sample for testing reliability and validity of the PAT tool will be published reports of measurement development and validation, such as the eligible full texts identified in this review, rather than persons with dementia, caregivers, dyads, or dining rooms which are usually the sample for validating instruments that assess constructs or concepts of interest in behavioral, psychological, and social research. In addition, the PAT tool addresses all the CTT-based psychometric parameters involving reliability and validity instead of any IRT-based Item-level parameters. Therefore, for criterion validity testing, the domains of the existing psychometric quality criteria^23–26 that focus on CTT-based psychometric properties rather than IRT-based parameters may be considered as criterion measures.

Limitations

This review included instruments that were reported in peer-reviewed records published in English only, which may lead to potential selection bias. It focused on psychometric evidence of eligible instruments in people with dementia during mealtime care. Therefore, psychometric evidence of these instruments in other populations and/or nonmealtime activities (if any) except for the evidence from the original development studies were not considered in the evaluation of their psychometric quality in this review.

Conclusions

To date, a number of instruments have been developed and/or used to assess dyadic interactions and physical and/or social environment during mealtime care for people with dementia. All the instruments had low psychometric quality, except for the refined CUED that showed moderate quality. All instruments warrant future testing of fundamental psychometric properties in larger samples and diverse care settings. The refined CUED is a potential tool for use and requires testing in direct on-site observations.

Supplementary Material

Table S2

Psychometric quality assessment of identified instruments

Table S1

Characteristics and psychometric properties of identified instruments