Assessing Mechanisms of Benefit in Adult Day Programs: The Adult Day Services Process and Use Measures

Abstract

Objectives.

A limitation of adult day service (ADS) research is that there remains little understanding of how these community-based long-term care programs operate to benefit clients or family caregivers (i.e., the process of ADS use). The purpose of this study was to validate the “ADS Process and Use Measures” (APUM) which were developed to assess such mechanisms.

Method.

Participant observation and semi-structured interviews in two ADS settings resulted in qualitative data to inform a conceptual model, subscales, and Likert-scale items. Three experts in ADS research reviewed the initial 129-item version of the APUM to establish content validity, and 27 family caregivers of current or prior ADS clients provided feedback on face validity of a subsequent 58-item version.

Results.

Principal components and confirmatory factor analyses on a sample of 269 family members of ADS clients recruited from 90 programs throughout the U.S. established a measure featuring 5 domains, 12 reliable subscales, and 49 items. Analysis of discriminant and convergent validity found that various subscales from four of the domains (Why ADS is Used, Events Prior to Use, Why ADS Does Not Work, and Pathways to Benefits) were significantly associated (p < .05) with family caregiver distress and ADS client quality of life variables.

Conclusion.

The ADS Process and Use Measures effectively assess mechanisms of program benefit and could help to enhance the overall quality of these critical community-based long-term care options for older persons and their families.

Introduction

Adult day service (ADS) programs in the United States offer out-of-home, supervised, community-based social and health services. A 2014 national survey of ADS programs found that there are 4,800 operational programs that serve more than a quarter of a million clients in the U.S. (Rome, Harris-Kojetin, & Park-Lee, 2015). Several systematic reviews suggest that ADS has mixed or no effects on client function, client nursing home admission, family caregiver well-being, and costs (Fields, Anderson, & Dabelko-Schoeny, 2014; Forster, Young, Lambley, & Langhorne, 2008; Gaugler & Zarit, 2001; Mason, Weatherly, Spilsbury, Arksey, et al., 2007; Mason, Weatherly, Spilsbury, Golder, et al., 2007). Recent reviews of qualitative and mixed methods research (Tretteteig, Vatne, & Rokstad, 2016) as well as within-person evaluations of ADS for family caregivers suggest positive effects of these programs on several caregiving outcomes, however (Zarit, Kim, Femia, Almeida, & Klein, 2014; Zarit et al., 2011; Zarit, Whetzel, et al., 2014). A limitation of the current ADS literature is that there remains little understanding of how these community-based long-term care programs operate to benefit clients or family caregivers (i.e., the process of ADS use) or how to measure such mechanisms. It is possible that the outcomes measured in existing empirical studies of ADS do not align with the types and processes of activities and services provided (Bull & McShane, 2008; Dabelko & Zimmerman, 2008; Gaugler, 2014b; Zarit, 2017). The objective of this study was to develop new measures of the process and mechanisms of ADS use for family caregivers and clients (the “ADS Process and Use Measures,” or APUM). These measures could offer a new set of tools for providers and researchers to evaluate how families and their elderly relatives use ADS and serve as indicators of program quality.

Current state of ADS measurement: Process and outcomes

There is little research identifying what aspects or processes of ADS (activity components, therapeutic services, program environment) could potentially influence the outcomes that researchers and policymakers have assumed ADS can modify (e.g., client institutionalization, family caregiver stress) (Cohen-Mansfield, Lipson, Brenneman, & Pawlson, 2001; Gaugler, 2014a; Gaugler et al., 2003; McCann et al., 2005). Several recent efforts have attempted to develop more uniform measurement of ADS outcomes. In 2015, the National Adult Day Services Association (NADSA) and Easterseals organized an unsponsored summit meeting on outcome measures in ADS. Results from the summit meeting indicated that there was a high level of agreement on the general domains of outcome measures in ADS – participant well-being, family caregiver well-being, and healthcare utilization. A set of specific measures for each these domains were proposed (Anderson et al., 2018). In addition, a Delphi review (Jarrott & Ogletree, 2016) of six ADS practitioners and six ADS researchers was convened to identify meaningful outcome indicators of older adults attending ADS and to also explore logistic considerations of integrating an assessment system into daily ADS operations. Jarrott and Ogletree (2016) noted that cost and staffing requirements along with appropriateness for specific populations are potential barriers to implementation.

Conceptualization of the process of ADS use

Gaugler (2014b) conducted a qualitative study of two ADS programs (one rural, one suburban) to understand the process of ADS use and pathways to potential positive or negative outcomes. The investigation of multiple sources of information yielded several key conceptual dimensions/constructs: the policy/environmental context of ADS; reasons why ADS was used; how ADS is used; and why ADS does or does not work. These categories are organized across a temporal dimension in the conceptual model of Figure 1, and within each category/construct emerged a number of specific themes based on activities and services, client behavior, family and staff perceptions, and interaction/communication between clients, staff, and/or family members in ADS. In the context of the current analysis, this conceptual model provided the framework of the potential constructs, and eventual measurement scales of the APUM that were refined and empirically validated with classic measurement testing approaches.

Figure 1.

Conceptual model: Qualitative phase (see Gaugler, 2014b)

Research focus of the current study

Adult day services operate as a “black box” of community-based long-term care, although some research on ADS has sought to determine whether these programs were efficacious within randomized controlled or quasi-experimental designs (Gaugler & Zarit, 2001; Fields et al., 2014). In light of less than consistent positive results on family caregiver or client outcomes, an initial conclusion was that ADS simply did not benefit clients and families consistently enough to warrant support (Callahan, 1989; Flint, 1995), although more recent methodological treatises challenge this assumption (Zarit, Bangerter, Liu, & Rovine, 2017). Before the goal of enhancing practice in ADS can be achieved, however, improved assessment of how clients and families utilize ADS is likely required. By relying on the rich information available from foundational qualitative research (Gaugler, 2014b), this study aimed to develop more sensitive, ADS-specific measures of the process of use (APUM). The grounding of ADS items in quotes and observational notes aligned the operationalization of ADS use with theoretically rich qualitative data to create measures that are more closely linked to the expectations, preferences, and achievable benefits of ADS for families, clients, and staff.

Methods

This study received human subject approval from the University of Minnesota Institutional Review Board (#0807S39521). Figure 2 provides a diagram of the multi-phase, sequential exploratory design (Creswell & Plano Clark, 2010) procedures.

Figure 2.

Sequential-explanatory design.

Qualitative Phase

As noted above, Gaugler (2014b) previously conducted participant observation and semi-structured interviews with 14 family members and 12 staff members to better understand the process of ADS use and pathways to potential positive or negative outcomes. More specific details of the methodology and results of the qualitative phase are available in Gaugler (2014b).

Quantitative Phase

Procedure and sample

The quantitative phase was the principal focus of the current paper. In the quantitative phase (QUAN; capitalization in mixed methods notation reflects the study strand that has “priority” in the study; see Morse, 1991), the first author and research staff members reached out to all ADS programs in an upper Midwestern state and then, to increase the sample size, randomly selected ADS programs listed in the National Adult Day Association’s online directory (approximately 400 eligible and active programs). Ninety programs responded and assisted the research team identify family members. Directors or another senior staff person in the program shared study information with family members and if family members agreed, shared the name and contact information with the research team. Upon receiving family contact information, the research team contacted each family member, obtained informed consent, and administered a survey that included APUM. To enhance response rates, the survey could be completed online, over the telephone, or via mail survey. Two follow-up email or mail queries (via ADS programs) were sent to ensure completion. Inclusion criteria were that family members had to have primary responsibility in the care of clients 60 years of age or older and cared for the client in a home setting. This recruitment approach resulted in a sample of 269 family members of ADS clients.

Data collection

Information on caregiver and care recipient sociodemographic characteristics as well as other variables related to the caregiving context are included in Table 1. Additional measures were administered in the quantitative component in order to establish validity of APUM:

Table 1.

Background Characteristics of Family Members and Adult Day Service Clients (N = 269)

Variable	M (SD) n (%)
Caregiver
Age	65.54 (12.20)
Female	212 (78.8%)
Caucasian	247 (91.8%)
Married	219 (81.4%)
Living children	2.58 (1.79)
Bachelor’ degree or higher	119 (44.2%)
Annual income $60,000 or higher	113 (42.0%)
Working full-time	65 (24.2%)
Spouse or partner of client	134 (49.8%)
Minutes to travel to ADS	17.75 (16.38)
Lives at home with client	216 (80.3%)
Provide help to the client  because of memory problems	226 (84.0%)
Client is diagnosed with dementia	199 (74.0%)
Hours spent taking care of client  at home during typical week	78.39 (65.78)
Client
Age	79.87 (10.22)
Female	124 (46.1%)
Caucasian	245 (91.1%)
Married	145 (53.9%)
Living children	3.15 (1.96)
Bachelor’s degree or higher	80 (29.7%)
Annual income $60,000 or higher	51 (18.9%)
Average activity of daily living  dependency (Katz, Ford, Moskowitz, Jackson, & Jaffe, 1963)a	.78 (.57)
Average instrumental activity of  daily living dependency (Lawton &  Brody, 1969)a	1.29 (.66)

NOTE: M = mean

SD = standard deviation

ADS = Adult day service

a0 = no help, 1 = some help, 2 = a lot of help

Caregiver distress.

A 3-item scale assessing involuntary aspects of the caregiving role (role captivity; α = .86) and a 3-item scale measuring family caregivers’ feelings of emotional and physical fatigue were utilized (role overload; α = .85) (Pearlin, Mullan, Semple, & Skaff, 1990). A 7-item version of the Zarit Burden Interview (ZBI) was also included (Newcomer, Yordi, DuNah, Fox, & Wilkinson, 1999; Zarit, Reever, & Bach-Peterson, 1980) (α = .92).

Client well-being.

The 13-item Quality of Life-Alzheimer’s Disease (QOL-AD) instrument measured family caregivers’ perceptions of clients’ mood, physical condition, interpersonal relationships, ability to participate in meaningful activities, and financial situation to create an overall assessment of global well-being (Logsdon, Gibbons, McCurry, & Teri, 1999) (α = .78).

Data analysis

Phase 1: Scale development.

The 1^st author analyzed qualitative data from Gaugler (2014b) to develop content for either: 1) new constructs; or 2) new items that added depth to existing constructs. Specifically, to create the items for APUM, the 1^st author conducted a line-by-line review of all interview transcripts and field notes from the qualitative phase to create Likert items that reflected the various themes identified. Items were reviewed and synthesized to reduce redundancy initially by the 1^st author. Using: a) the previous categories identified in the conceptual model in Figure 1 as the constructs of interest to measure; b) the themes identified in this earlier qualitative research as measurement scales; and c) specific quotes or observation notes as the basis for instrument items, the first author (JEG) created the initial version of APUM.

Phase 2: Content validity.

Following the initial construction of the quantitative measures, the content validity of APUM was analyzed (Li, 1995; Nunnally, 1978; Nunnally & Bernstein, 1994). The raters of the newly generated APUM items included three experts in field of ADS research: Dr. Keith Anderson, Dr. Holly Dabelko-Schoeny, and Dr. Shannon Jarrott. The raters examined the appropriateness of particular measures and items, their plan of administration, utilization of scores, and feasibility of inclusion in the quantitative measurement protocol. Recommendations forwarded by these experts were incorporated in APUM to enhance the content validity of the measure and items. This procedure was also utilized to reduce the length of APUM by eliminating conceptually redundant items.

Phase 3: Face validity.

To test the face validity of APUM, a sample of 27 family members from the University of Minnesota Caregiver Registry were provided with an online version of the measure. The University of Minnesota Registry includes family caregivers (N = 742 as of April 2018) who have volunteered to learn more about the 1^st author’s research projects following a variety of community outreach efforts. For the Phase 3 face validity testing of this study, the 1^st author extended an email invitation to family members on the Registry who were currently or had used ADS for relatives to complete APUM; 27 individuals responded. In addition, an open-ended item was provided to solicit feedback on the wording and clarity of the items (“Please describe in the text box below if: 1) any of the questions above did not make sense, and if so, which ones; 2) any other issues that made the questions difficult to answer or understand; 3) any questions are redundant, and if so, which ones; or 4) it took too long for you to complete the questions above.”). A final open-ended item asked respondents how long it took to complete the survey. Family members of the Registry completed the survey anonymously.

Phase 4: Construct validity.

Principal components analyses were then conducted to identify the measurement structure of each domain (Tabachnick & Fidell, 2007) on the sample of 269 family caregivers recruited for the current study (see above and Table 1) IBM SPSS version 24 was used to conduct the principal components analysis (IBM Corporation, 2016). Reliability analyses were also performed to examine each subscale’s structural integrity using Cronbach coefficients. To further examine and establish the measurement structure of APUM, a confirmatory factor analysis was conducted. AMOS 24.0.0 was used to conduct the confirmatory factor analysis (Arbuckle, 2016). An analysis of discriminant/convergent validity was also performed (Nunnally, 1978; Nunnally & Bernstein, 1994). Specifically, bivariate correlations between various subscales and items of APUM were performed with established measures of caregiver distress and client well-being to determine if they were significantly (p < .05) associated.

Results

Phase 1: Scale development

The 1^st author based the development of APUM items and measures from a review of 136 pages of transcripts from field notes and semi-structured interviews conducted in a previously published qualitative study (Gaugler, 2014b). Initially, a set of 129 items measuring the following constructs were developed (Reasons for Adult Day Service Use, 29 items; Process of ADS Use, 70 items; Pathways to Outcomes, 30 items). Where possible, the items were worded according to the voices of family and staff participants.

Phase 2: Content validity

After the initial draft of items, the three national experts in ADS research reviewed the items to determine if they were appropriately worded, clear, and captured the extent of the construct domains. In addition, these experts provided feedback to reduce unnecessary redundancy in items and the refine scale domains. Following the assessment of content validity, a 58-item measure (Reasons for Adult Day Service Use, 10 items; Process of ADS Use, 32 items; Pathways to Outcomes, 16 items) was created for further psychometric testing.

Phase 3: Face validity

It took family caregivers from 5 to 15 minutes to complete APUM. Open-ended feedback from family caregivers largely focused on the clarity of item wording. Items were rephrased accordingly for subsequent versions of the measure; no items were deemed as consistently redundant. Participants answered all items. Domains/scales were further refined to capture the existing subscales and their corresponding items. Following the incorporation of the open-ended feedback of participants, the final 58-item version of APUM was prepared for further psychometric testing.

Phase 4: Construct validity

Principal components analysis (PCA) was initially conducted with varimax rotation on the sample of 269 family members of ADS clients. Separate principal components analyses were conducted for the following re-classified scales of APUM: Why ADS is Used (4 items), Events Prior to ADS Use (6 items), Process of ADS Use: Why ADS Does Not Work (18 items), Process of ADS Use: Why ADS Does Work (16 items), and Pathways to Benefit (14 items). Cut-off points of .45 were used when including items on a given factor (Tabachnick & Fidell, 2007). Table 2 presents the item factor loadings; loadings under .45 are not reported to enhance interpretation (Tabachnick & Fidell, 2007).

Table 2.

Principal components analysis, rotated factor loadings^a

	Why ADS is Used		Events Prior to ADS Use	Process of ADS Use: Why ADS Does Not Work
Please tell us how much you agree or disagree with the following statements:b	Social reasons for ADS use (2 items)*	Logistical Reasons for ADS use (2 items)*	Events prior to ADS use (5 items)*	Dissatisfaction with ADS use (7 items)*	Problems with ADS use (4 items)*	Temporal Issues with ADS use (2 items)*
1. ADS allows my relative to talk to other  people/socialize.*	.88
2. Using ADS allows my relative to spend time  doing things she/he would not do at home,  which is good for my relative.*	.90
3. My relative uses ADS because I or other  family members need time away from my  relative to work, spend time with family,  run errands, or emotional relief.*		.84
4. My relative uses ADS because it’s more  affordable than other service alternatives  (e.g., nursing home, home health care).*		.85
5. My relative was experiencing serious  health problems (e.g., falls), and using  ADS was necessary.*			.72
6. A healthcare professional (doctor, social  worker, nurse) recommended that I use ADS  for my relative.*			.61
7. My relative was suffering from problem  behaviors (e.g., crying, arguing,  agitation, anxiety, etc.), so I/we thought  that using ADS would help her/him.*			.74
8. My relative was feeling down and even a  little depressed, and I/we thought that  using ADS would help her/him.*			.71
9. The other services my relative was using  (e.g., nursing home, home health care) did  not meet her/his needs or my needs, which  led us to use this ADS program.*			.46
10. The staff do not have the expertise   needed to effectively care for my   relative.*				.61
11. I am not too sure what my relative does   at the ADS.*				.74
12. Staff don’t listen to what my relative   has to say.*				.76
13. I’ve got a feeling there’s more down time   than the ADS is telling me about.*				.75
14. The physical layout of the ADS makes it   difficult for my relative to get around.*				.61
15. There is no flexibility in ADS hours.*				.59
16. A lot of times the ADS is too flexible,   and my relative needs a little more   structure and consistency to benefit.*				.53
17. I don’t have the time to use the services   the ADS offers me (e.g., support group).*					.65
18. My relative thinks that the activities   she/he is asked to participate in are   repetitive and/or childish.*					.72
19. My relative does not like having to get   ready to go to the ADS.*					.73
20. My relative does not like ADS.*					.56
21. The whole process of trying to make my   relative go to ADS takes more time than   it is worth.*						.85
22. I could benefit more from ADS if my   relative used it more.*						.90
	Process of ADS Use:   Why ADS Does Work		Pathways to Benefits
Please tell us how much you agree or disagree with the following statements:b	Perceived quality of ADS (6 items)*	Engagement process in ADS (7 items)	Health benefit (7 items)*	Memory and functional benefit of ADS use (3 items)*	Lack of interaction problems (2 items)*	ADS is essential (2 items)*
23. My relative realizes that there is some   value in going to ADS.*	.66
24. I wish I had used ADS earlier for my   relative.*	.77
25. The routine at ADS is too unpredictable   for my relative.*,c	.86
26. The ADS staff does a good job of getting   my relative ready to leave at the end   of the day.*	.85
27. Because of my relative’s resistance to   ADS activities, she/he has not benefitted   3 as much as she/he could from ADS.*,c	.82
28. The relationships my relative has   3developed at ADS has benefitted her/him.*	.61
29. The ADS staff is very good at   3communicating any changes in my   3relative’s behavior or health.		.47
30. Some of the activities at the ADS have   3really helped my relative learn new   3things.		.71
31. The ADS gives my relative choices.		.75
32. The ADS gives my relative a sense of   purpose.		.68
33. The staff at the ADS is compassionate.		.68
34. The ADS tries to customize their care to   meet individual’s needs and interests.		.77
35. The activities provided by the ADS are   for the benefit of participants, not   because they are easy for the staff to   manage.		.50
36. The activities of the ADS tire my   relative out and by the time my relative   gets home she/he sleeps better than on   days she/he is not at ADS.*			.69
37. Using ADS allows me to manage my health   problems a little better.*			.53
38. When my relative comes home, she is worse   in terms of her/his behavior.*,c			.53
39. There is no real change in my relative   since using ADS.*,c			.61
40. The ADS has improved my relative’s   physical health.*			.47
41. Using ADS helped reduce my relative’s   feelings of depression.*			.84
42. My relative uses her/his mind more   because of the ADS activities.*			.83
43. Since enrolling at ADS, my relative’s   decline in memory seems to have slowed.*				.77
44. Even on days that my relative is not   using ADS, she/he can do some of the   exercises or activities at home, and that   helps.*				.74
45. After using ADS, my relative has been   more steady on his/her feet and moves   around better.*				.76
46. My relative’s behavior bothers other   clients at the ADS.*,c					.78
47. I don’t think my relative is that engaged   when she/he is at the ADS.*.,c					.77
48. When my relative does not go to ADS,   she/he doesn’t know what to do with   herself/himself.*						.82
49. If it wasn’t for the ADS, our family   would have tried using a nursing home or   similar facility for my relative.*						.66

NOTE: ADS = adult day service

^*Item/subscale of final Adult Day Services Process and Use Measure

^aFactor loadings.45 above shown

^b1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree

^cItem scoring reversed

Additional single items: “I could benefit more from ADS if my relative used it more;” “My relative does not really like some of the food that the ADS provides; “ and “Turnover in staff and leadership at the ADS has been disruptive for my relative.”

Two factors were extracted on the Why ADS is Used subscale. The first factor yielded an eigenvalue of 1.87 and explained 47% of the variance while the second factor had an eigenvalue of 1.17 and explained 29.3% of the variance. Two items loaded on the first factor (social reasons for ADS use) that were significantly and strongly correlated (r = .60, p < .001) and two items loaded on the second factor (logistic reasons for ADS use) that were also significantly correlated (r = .44, p < .001).

One factor was extracted from the Events Prior to ADS Use scale. The first factor yielded an eigenvalue of 2.18 and explained 36.37% of the variance. Five items loaded onto the first factor (events prior to ADS use). Reliability analyses using Cronbach’s alpha indicated that the events prior to ADS use subscale achieved moderate reliability (α = .67).

Three factors were extracted from the Process of ADS Use: Why ADS Does Not Work scale. The first factor had an eigenvalue of 4.37 and explained 24.25% of the variance. The second factor had an eigenvalue of 2.53 and explained 14.03% of the variance. The third factor had an eigenvalue of 1.27 and accounted for 10.24% of the variance. Seven items loaded onto the first factor (dissatisfaction with ADS), four items loaded onto the 2^nd factor (problems with ADS use), and two items loaded onto the 3^rd factor (temporal issues with ADS). The dissatisfaction with ADS subscale demonstrated good internal reliability (α = .80), while the problems with ADS use showed moderate reliability (α = .66). The two items of the temporal issues with ADS use were highly correlated (r = .63; p < .001). Two additional factors were extracted from the Process of ADS Use: Why ADS Works scale. The first factor included an eigenvalue of 4.85 and accounted for 30.29% of the variance, while the second factor had an eigenvalue of 3.34 and accounted for 20.85% of the variance. Six items (perceived quality of ADS) loaded onto the first factor and seven items (engagement process of ADS) loaded onto the 2^nd factor. Both subscales (α = .87, α = .77, respectively) demonstrated excellent to good internal reliability.

Four additional factors were extracted from the Pathways to Benefits scale. The first factor yielded an eigenvalue of 3.77 and explained 26.94% of the variance; the second factor had an eigenvalue of 2.46 and explained 17.58% of the variance. Eigenvalues of 1.51 and 1.00 and percent variances of 10.76% and 7.16% for factors 3 and 4 were found, respectively. Seven items loaded onto the first factor (health benefit of ADS; α = .79), three items loaded onto the second factor (memory and functional benefit of ADS use; α = .73), two items loaded onto the 3^rd factor (lack of interaction problems in ADS; r = .31; p < .001), and two items loaded onto the 4^th factor (ADS is essential; r = .24; p < .001).

A confirmatory factor analysis (CFA) was conducted on the subscales identified in the principal components analysis. The results of the standardized solutions are presented in Figures 3 and 4. Maximum likelihood estimation was used to estimate factor loadings, inter-factor correlations, and correlations between item errors. Due in part to the sheer number of parameters and available sample size, two CFA models were conducted: one that included the Reasons for Use, Events Prior to Use and Why ADS Does Not Work subscales (Figure 3), and another that included the Why ADS Works and Pathways to Benefits subscales (Figure 4). The first CFA model demonstrated a good fit to the observed data (GFI = .92, CFI = .93, RMSEA = .045) with high standardized factor loadings, suggesting and confirming the earlier factor analysis findings. The 2^nd model showed inadequate fit when the engagement process subscale was included; when excluded moderate fit to the observed data was achieved as illustrated in Figure 4 (GFI = .91, CFI = .92, RMSEA = .07).

Figure 3.

Confirmatory factor analysis: Why ADS is Used, Events Prior to Use, Process of Use: Why ADS Does Not Work

NOTE: Correlates among errors within domain were also estimated. Numbers in boxes represent the numbered items in Table 2.

Figure 4.

Confirmatory factor analysis: Process of Use: Why ADS Works and Pathways to Benefits

NOTE: Correlates among errors within domain were also estimated. Numbers in boxes represent the numbered items in Table 2.

To further demonstrate construct validity, a series of bivariate correlations were performed to determine the statistical associations between the subscales of APUM and established measures of caregiver distress (role overload, role captivity, burden) and family caregivers’’ perceptions of their relatives’ quality of life. Family caregivers who were more likely to utilize ADS for logistical reasons also indicated greater role overload (r = .32), role captivity (r = .32), and burden (r = .33) (p < .001) as well as lower relative quality of life (r = −.16; p < .05). Similarly, family caregivers who indicated higher scores on the events prior to ADS use subscale reported greater role overload, role captivity, and burden (r = .21, .18, .19, respectively; p < .01) and lower relative quality of life (r = −.27, p < .001).

Family caregivers who expressed greater dissatisfaction with ADS use, problems with ADS use, and temporal issues with ADS also indicated greater role captivity (r = .18, p < .01; r = .15, p < .05; r = .22, p < .001, respectively) and burden (r = .15, p < .05; r = .13, p < .05; r = .20, p < .01, respectively). Additionally, family caregivers who reported greater temporal issues with ADS also experienced greater role overload (r = .19, p < .01), whereas respondents who indicated greater problems with ADS use as well as temporal issues with ADS perceived their relatives as having lower quality of life (r = −.30, p < .001; r = −.18, p < .01, respectively).

Among the Pathways to Benefits subscales, one (ADS is essential) was significantly and positively correlated with family caregiver role overload, role captivity, and burden (r = .25, .25, .25, p < .001, respectively) and negatively associated with family caregivers’ perceptions of relatives’ quality of life (r = −.21). In addition, family caregivers who expressed greater memory and functional benefits of ADS (r = .14, p < .05) and lack of interaction problems in ADS (r = .27, p < .001) also indicated greater quality of life for their relatives who used ADS.

Because of the factor analytic approaches described above, multiple single items from the 58-item APUM were not included in the various subscales. However, some of these items may still hold worth as measures of important mechanisms related to ADS quality. Subsequent bivariate correlations with family caregiver distress variables and family caregivers’ perceptions of their relatives’ quality of life implied their construct validity. Family caregivers who agreed more strongly with the items “I could benefit more from ADS if my relative used it more” and “My relative does not really like some of the food that the ADS provides” were more likely to report greater role overload (r = .18, p < . 01; r = .15, p < .05, respectively), role captivity (r = .24, p < . 001; r = .20, p < .01, respectively), and family caregiver burden (r = .20, p < . 01; r = .21, p < .01, respectively) as well as impaired quality of life for their relatives in ADS (r = −.17, p < . 01; r = −.15, p < .05, respectively). Family caregivers who were less likely to indicate staff turnover as a problem in ADS also reported less role captivity and burden (r = −.19, p < .01; r = −.20, p < .01, respectively) and greater quality of life for their relatives in ADS (r = .16, p < .05). Table 2 indicates those items that comprise the final, 45-item APUM as well as the number of items in each subscale.

Discussion

The results of this study complement several recent efforts to identify consistent outcome measures for ADS (Anderson et al., 2018; Jarrott & Ogletree, 2016) by creating reliable and valid measures to determine how and why such outcomes are achieved. Similar to interventions developed in other areas of gerontology (e.g., dementia caregiving; see Burgio & Gaugler, 2016; Gitlin & Hodgson, 2015; Gitlin et al., 2015; Wethington & Burgio, 2015), the lack of research attention on the processes and mechanisms of benefit in ADS have hindered efforts to enhance ADS program delivery. Avoiding key questions of process has potentially obscured whether and how ADS programs are effective in improving client quality of life and function, reducing family caregiver distress, and facilitating older adults’ desire to age in place. The current study addresses an important research and practice gap, as the use of mixed methods allowed for the construction of APUM with items and domains that heretofore have been unavailable to providers and researchers.

Many of APUMs’ subscales demonstrated construct validity in their empirical associations with important caregiver and perceived client outcomes. For example, higher scores on the logistical reasons for use (family caregivers were more likely to indicate that ADS was used because it offered time away from the relative to complete other activities or because it is affordable), the events prior to ADS use (family caregivers were more likely to agree that ADS was used because the relative had health problems, because of a professional recommendation, because the relative was feeling depressed) and the ADS is essential (e.g., “the relative does not know what to do when not attending ADS”) subscales were linked to family caregivers’ distress and lower client quality of life. Family members who were more likely to agree as to why ADS was used and viewed these programs as essential may have been those in the most need of the respite, relief, and therapeutic activities ADS offer. It is important to note that because these findings are cross-sectional, it remains unclear whether ADS programs reduce such unmet needs over time.

In addition, many of the Process of ADS Use: Why ADS Does not Work subscales were associated with greater subjective stress and burden on the part of family caregivers as well as lower perceived quality of life of ADS clients. Family caregivers who indicated greater dissatisfaction with ADS, problems with ADS use, and temporal issues with ADS all appeared more likely to suffer from emotional exhaustion, feelings of entrapment in care responsibilities, and burden in addition to perceiving their relatives’ quality of life as impaired. The results of the validity analysis strongly suggests that the Process of ADS Use: Why ADS Does not Work subscales are potentially powerful indicators of ADS quality. The information these subscales yield could offer ADS providers with important insights as to areas they could attend to when engaging in quality improvement efforts or similar initiatives. In contrast, the subscales from the Process of ADS Use: Why ADS Works scale were not empirically associated with any caregiver or client outcomes.

Family caregivers who indicated greater memory and functional benefits of ADS use as well as a lack of interaction problems were more likely to indicate positive quality of life on behalf of their relatives. Specifically, improved socialization and function (memory or otherwise) on the part of relatives who used ADS programs were associated with less caregiver distress and greater well-being on the part of relatives. Unlike the various other APUM subscales, where higher scores seemed to indicate that ADS use mirrored current unmet needs on the part of family caregivers or relatives, these two Pathways to Benefits subscales were more indicative of how ADS was of benefit.

Single items representing family caregivers’ desire to use ADS more and relatives’ dislike of food were significantly associated with greater caregiver distress as well as less positive quality of life on the part of relatives. Alternatively, family caregivers who reported that turnover was not a problem/issue in ADS not only experienced less subjective stress but also felt that their relatives had greater well-being. Such findings imply the utility of these three items in serving as additional key markers of ADS processes and mechanisms of benefit. These single items, deployed with the subscales above, would allow providers or other key stakeholders to ascertain how well ADS is working or not for older clients and their family members.

Although APUM demonstrated appropriate validity and reliability, there are several important limitations to note. The psychometric testing was cross-sectional; how the various subscales are related to caregiver distress and care recipient outcomes over time necessitates longitudinal analysis. The study sample is not generalizable to all family members of ADS users, as participants were volunteers from a small subset of programs in Minnesota and nationally. Although several efforts are in development to identify appropriate outcomes measures for ADS clients and their family caregivers (Anderson et al., 2018; Jarrott & Ogletree, 2016), the ongoing lack of clarity in criterion/”gold-standard” outcome measures for ADS further complicates the development of quality measures in these and other community-based long-term care programs. Family members completed APUM; to obtain a fuller perspective of program quality and mechanisms of benefit, staff and perhaps even client measures are required. To partly address this limitation, a staff version of the APUM is currently under development and is also built upon the qualitative strand of this mixed methods study.

The need to measure how and why ADS programs work is pressing. Adult day programs have continued to grow in the U.S. over the past several decades, but because of randomized controlled evaluations dating from the late 1970s and early 1980s (Fields et al., 2014; Forster et al., 2008; Gaugler & Zarit, 2001; Mason, Weatherly, Spilsbury, Arksey, et al., 2007; Mason, Weatherly, Spilsbury, Golder, et al., 2007), ADS has difficulty laying claim to the mantle of “evidence-based.” However, at least some ADS programs have evolved considerably to provide a range of therapeutic services, person-centered care activities, and rehabilitative therapy. In this regard, earlier classifications of “social” and “medical” ADS models likely do not capture the diverse approaches that ADS programs adopt to serve their clients and families (Conrad, Hughes, Hanrahan, & Wang, 1993; Gaugler et al., 2011). This has resulted in a practice and scientific gap where it is unknown whether modern iterations of ADS programs are offering quality care to clients and family members. The ADS Process and Use Measures could help providers: 1) begin to effectively identify areas of strength and opportunities for improvement in their particular programs; and b) utilize findings from APUM to document areas in their particular programs that are high quality. Adult day providers as well as state and national advocacy organizations have often bemoaned the lack of recent data demonstrating how “ADS works,” and APUM may offer programs with specific, actionable data on how they do so (or not).

The ADS Process and Use Measures are freely available (to obtain, please contact the authors). Although providers may choose to administer these surveys differently according to their own needs, recent studies in ADS outcome measure development recommend bi-annual assessment intervals (as 3-month assessment intervals proved too burdensome to ADS staff) (Jarrott & Ogletree, 2016). Based on our experience, both online and mail survey formats would be appropriate. Future work is required to develop clinical scoring guidelines. However, we can recommend a general guideline that we have adopted when measuring the acceptability and utility of various interventions for family caregivers of persons with dementia (Garlinghouse et al., 2017; Gaugler, Reese, & Mittelman, 2017; Mitchell et al., 2018). If an item-response average of 3 or less occurs on positively worded subscales/items and 3 or higher occurs on negatively worded subscales/items, then quality improvement attention is warranted.

Other long-term care providers are mandated to regularly report quality indicators and outcomes via standardized assessments. The costs associated with these efforts (e.g., hiring of specialized data collection staff, reporting length) make such efforts prohibitive for ADS providers at present (Jarrott & Ogletree, 2016). Although a centralized data source of ADS data holds appeal from a scientific standpoint, the effects of these onerous data collection efforts on care quality raise concerns. For example, uniform outcome or quality indicator measurement may serve to “standardize” ADS programs away from models that emphasize psychosocial well-being towards those approaches that are more tightly focused on medical/rehabilitative outcomes (Anderson et al., 2018; Mukamel, Haeder, & Weimer, 2014). One could argue such trends have already taken place in nursing homes over the past three decades, necessitating culture change (Gaugler, 2016; Grabowski et al., 2014). Finding a balance that results in provider and/or stakeholder-initiated approaches that result in more localized, relevant quality improvement initiatives would be an optimal goal (e.g., the Performance Improvement Pilot Projects have demonstrated considerable success in Minnesota residential long-term care settings; see Arling et al., 2013). In such an approach, providers would identify those areas where quality improvement is needed based on APUM, such as efforts to reduce staff turnover. In doing so, the value of quality improvement assessment to the day-to-day practice of ADS becomes more apparent, versus the hierarchical, regulatory, and some would argue punitive approach utilized to ensure quality in other long-term care contexts (Gaugler, 2016).