Abstract
Background.
The National Council on Aging’s Aging Mastery Program (AMP) aims to help older adults implement health behavior and lifestyle changes to promote healthy aging and social engagement. The purpose of the present community-partnered evaluation was to test the effectiveness of AMP implementation in Los Angeles County to improve participants’ quality of life, global physical and mental health, and patient activation.
Method.
A modified randomized wait-list controlled trial design was used to examine experimental, quasi-experimental, and dose-response evidence in five senior centers. Participants completed questionnaires at baseline and after the 10-week intervention, self-reporting their overall quality of life, physical health, mental health, and patient activation.
Results.
Experimental, intention-to-treat analyses found AMP assignment did not affect any measured outcomes (n = 71). Quasi-experimental, “as treated” analyses (n = 106) controlling for study site and sociodemographic characteristics indicated that participants who attended AMP reported more positive changes in global mental health than the control group. Attending AMP was not associated with changes in quality of life, physical health, or patient activation. Dose-response analyses among AMP participants who attended at least one class (n = 75) found that attending more classes was not significantly associated with greater improvements in mental health.
Conclusions.
Experimental, intention-to-treat analyses did not support effectiveness of AMP on quality of life, physical or mental health, or patient activation; quasi-experimental analyses found attending AMP was associated with improvements in mental health. Recruitment challenges and participants’ nonadherence with condition assignment decreased our ability to detect effects. https://clinicaltrials.gov/ct2/show/NCT03342729?term=Aging+Mastery+Program&rank=1.
Keywords: community health, dissemination and implementation, evaluation, healthy aging, lifestyle modification
There is large-scale interest in developing, and documenting impacts of, community-based programs to promote healthy aging and social engagement for diverse older adults (Belza, Altpeter, Smith, & Ory, 2017; Davitt, Greenfield, Lehning, & Scharlach, 2017; Pruchno, 2015). Community-based prevention programs focused on exercise or common chronic medical conditions among low-income older adults are associated with reduction in key risk factors and improved health outcomes (Robare et al., 2011; Smith, Ory, Ahn, Bazzarre, & Resnick, 2011; Zgibor et al., 2017). Frameworks for evaluating successful aging emphasize the value of supporting emotional health, autonomy, engagement, and physical activity (i.e., extending beyond optimizing biological processes; Depp, Vahia, & Jeste, 2010; Martin et al., 2015; Zgibor et al., 2017). Evidence-based programs can improve emotional health, but this is often included as a secondary outcome (Acumen, LLC, 2017).
Given many health limitations are chronic and incurable, it is crucial to attend to older adults’ health and overall well-being, as well as how “activated”—how willing and able—they are to take actions to manage their health conditions. Quality of life reflects personal experience of health, depression, and loneliness (Henning-Smith, 2016); and patient activation may contribute to improved quality of life among persons with chronic conditions (Greene & Hibbard, 2012). Disease self-management helps improve health-related quality of life in middle-age participants, but more information is needed to know how such interventions affect older adults (Ory et al., 2014).
The National Council on Aging (NCOA) aims to improve older adults’ lives by generating solutions addressing challenges of aging. Recognizing the scarcity of comprehensive (vs. disease- or skill-specific) programs for older adults, NCOA conceived the Aging Mastery Program (AMP): an evidence-informed, engaging education and behavior change program for aging well. Best practices are embedded within the curriculum spanning diverse themes of (1) navigating longer lives, (2) exercise, (3) sleep, (4) healthy eating and hydration, (5) financial fitness, (6) advance care planning, (7) healthy relationships, (8) medication management, (9) community engagement, and (10) falls prevention.
AMP posits that modest lifestyle changes can produce large results and aims for participants to learn, implement, and “master” lifestyle changes by incorporating them into existing routines. The in-person, 10-part program focuses on empowering older adults to cultivate health and well-being. It incorporates learning objectives from published research on each topic, in-class activities promoting skill building, opportunities for expert community speakers, and incentives for participants to practice the skills and tools learned in class (action steps). The incentives aim to motivate participation by allowing participants to accumulate points by completing program activities. Participants can redeem points for rewards. Social activity, active participant engagement, and/or support within a group format are hallmarks of effective interventions (Dickens, Richards, Greaves, & Campbell, 2011). Accordingly, in-class activities are designed to facilitate group discussion and peer support. Table 1, an example outline of one session curriculum, illustrates how these elements were incorporated.
Table 1.
Example Curriculum Outline for Sleep Class.
| Activity | Time (min) |
|---|---|
| Welcome and warm up discussion | 20 |
| • Are you happy with your sleep pattern? | |
| • What are your biggest obstacles to getting enough sleep? | |
| Sleep lecture provided by staff or expert speaker | 20 |
| Talk about it: The importance of sleep | 10 |
| • Share thoughts and feelings about why sleep is important | |
| • Discuss the importance of finding sleep strategies that work | |
| In-class activity: Identifying sleep problems | 10 |
| • Think about what makes it harder to sleep | |
| • Write down pitfalls and underlying causes, and share with class | |
| Make a plan: My sleep plan | 20 |
| • In small groups or all together, participants read through the sleep tips and advice, then share their own tips for sleeping well | |
| • Participants pick at least two healthy habits to include in their sleep plan | |
| Action steps | 10 |
| • Review action steps and address questions | |
| ○ Fill in sleep diary for one week (20 points) | |
| ○ Commit to at least two healthy habits in their sleep plan (10 points) | |
| ○ Follow sleep plan for seven days. At the end of week, make a note of what worked or not, and adjust sleep plan accordingly (10 points) | |
| • Distribute 20 points for completion of the day’s class; update points from last session’s completed action steps |
Note. Action steps are part of the incentive reward system; accruing more points allow participants to receive incentives at the end of the Aging Mastery Program course. After class, facilitators are encouraged to review community resources relating to sleep and share other opportunities for learning about sleep at the center.
The AMP core curriculum was developed for broad dissemination and is available online at www.ncoa.org/AMPMaterials for organizations that hold program licenses. A standard AMP core participant kit (which participants received) includes supplementary materials and handouts including exercise DVDs, a Five Wishes Packet (advance directives form), and a weekly check-in notebook for goal setting and tracking progress. Facilitators can access online resources such as in-class presentations, lesson plans, implementation and fidelity guides, and evaluation and marketing materials.
To date, AMP has reached 10,559 older adults across 263 sites and 32 states. Research into AMP includes pretest–posttest studies between 2013 and 2015 and a quasi-experimental study in 2016 in 10 New York senior centers. The unpublished pretest–posttest evaluations demonstrated positive changes in social connectedness, physical activity levels, healthy eating habits, use of advance planning, medication management, participation in evidence-based programs, adoption of other healthy behaviors and high rates of satisfaction and retention (Herrera-Venson, 2018). In a published quasi-experimental study, sites were matched by key demographics and randomized to AMP or regular senior programming in a nested, partial crossover control group design, and assessed at baseline, 10 weeks, and 20 weeks (Ferretti et al., 2018). Examination of the means found that the intervention group showed greater improvements in physical activity and in advance care planning activity than the control group, but there were no differences between groups in changes in patient activation or single-item measures of general health and quality of life. This research suggests that AMP may be effective in improving secondary drivers of healthy, successful aging.
As a next step, NCOA wanted to examine AMP’s association with more detailed measures of quality of life and emotional and physical health, and to see whether AMP could improve participants’ skills and confidence to better manage their health within the health care setting (patient activation) in a different urban population.
Current Study
For the past 10 years, University of California, Los Angeles’ (UCLA) Los Angeles Community Academic Partnership for Research in Aging (L.A. CAPRA) and the City of Los Angeles Department of Aging (DoA) have partnered to implement and test practical interventions to improve quality of life of older adults in the greater Los Angeles area. This partnership was founded on the principles of community-based participatory research, in which academic and community partners are equal members of the study team and participate together in shared decision making on the design, implementation, evaluation, and dissemination of research findings. In collaboration with NCOA, the General City Manager at DoA invited L.A. CAPRA to participate in an evaluation of the effectiveness of AMP at community senior centers. A testament to the bidirectionality of this community–academic partnered research, this project presented a distinctive collaborative opportunity in which most of the program and research protocol was implemented by staff at the community sites, with L.A. CAPRA staff analyzing the data and summarizing the results.
This community-initiated project evaluated AMP’s effectiveness increasing the quality of life, physical health, mental health, and patient activation of older adults in the greater Los Angeles area using a modified randomized, wait-list control trial. Five Los Angeles city and county senior service sites implemented the AMP intervention and recruited participants; university researchers helped collect and analyzed survey data before and after the intervention.
Method
Procedure and Participants
Two City of Los Angeles senior center sites and three Los Angeles County senior center sites were selected by the DoA to implement the intervention and evaluation. Site leaders and facilitators completed a 2-hour webinar training with NCOA staff on implementation of AMP and a 1-hour training from the L.A. CAPRA team explaining the planned evaluation process to test AMP effectiveness. After enrolling and completing a baseline questionnaire, participants were to be randomly assigned to either the intervention group (i.e., attend AMP classes for 1 hour, once a week, for 10 weeks at the community site) or the wait-list control group (i.e., wait to attend the class 2.5 months after the intervention group). All participants were to be recontacted for a second “postintervention” questionnaire after the intervention group completed the 10-week AMP course. All participants received $15 for completing the baseline and $20 gift cards for completing the postintervention survey. Site leaders and the evaluation team worked together to select dates to conduct assessments. Site leaders were responsible for recruitment and retention of 40 participants at each site.
Recruitment methods included flyers and newsletters at sites, invitations to seniors already enrolled in other site programs, and word of mouth from participants and/or from staff/site leaders. Eligibility criteria included being able to speak and read English, being able to provide informed consent and complete the surveys, and planning to live in the region during the next 6 months. L.A. CAPRA staff confirmed eligibility of interested participants and obtained oral informed consent following a protocol approved by the UCLA Institutional Review Board.
Two sites modified the planned randomized controlled trial (RCT) study design because they had trouble meeting the recruitment goal of 40 participants within the necessary time frame. Site D used a quasi-experimental design, assigning the first cohort of participants to AMP and then recruiting a second cohort as a nonequivalent wait-list control group. Site E did not use a control group; all participants recruited at that site were invited to attend AMP immediately.
Outcome Measures
We used the CASP-19 (Hyde, Wiggins, Higgs, & Blane, 2003) to measure overall quality of life, which covers four domains: control, autonomy, pleasure, and self-realization. The overall 19-item scale has a possible range from 0 to 57; higher scores indicate better quality of life. Summary scores were generated when participants skipped two or fewer items, using mean imputation for missing values (Cronbach α = .87).1
Participants completed physical and mental health subscales from the Patient-Reported Outcomes Measurement Information System (PROMIS) 10-item Global Health measure (Hays, Bjorner, Revicki, Spritzer, & Cella, 2009), which was scored using the HealthMeasures online scoring service (https://www.assessmentcenter.net/ac_scoringservice). The service generated t scores for physical health from rating scales assessing overall physical health, ability to carry out every day physical activities, fatigue, and pain; and t scores for mental health from four items rating overall quality of life, mental health (e.g., mood and ability to think), satisfaction with social activities and relationships, and frequency of emotional problems (e.g., feeling anxious, depressed, or irritable). The service used response pattern scoring, calibrated to the PROMIS Wave 1 sample, to generate the t scores; higher scores indicate better mental and physical health.
We measured patient activation using the Insignia’s Patient Activation Measure (PAM-10) scale to assess patient’s knowledge, skill, and confidence in managing their health (Hibbard, Stockard, Mahoney, & Tusler, 2004). The measure is scored on a scale from 0 to 100. Four levels of activation have been identified, which reflect a developmental progression from being passive with regard to one’s health to being proactive (Hibbard et al., 2004). Higher patient activation scores have been linked to greater likelihood of beneficial self-management behaviors, better quality of life, and better functional status among adults with chronic conditions (Mosen et al., 2007).
Background Measures
Participants self-reported their age, gender, ethnicity (i.e., Hispanic, Latino, or Spanish origin) and race.2 Participants also reported their highest year of school completed and average monthly income in the previous year before taxes and other deductions. Finally, participants provided health information such as whether their health care provider had ever told them they had any of 14 different chronic conditions.
Data Analysis Plan
Our original a priori study design (submitted to clinicaltrials. gov) was to conduct an intention-to-treat (ITT) analysis and treat all participants assigned to AMP as the experimental group regardless of whether or not they received the intervention. We selected this study design because it avoids the bias associated with nonrandom loss of participants and it is accordingly the recommended protocol in the Consolidated Standards of Reporting Trials (CONSORT) guidelines on the reporting of randomized controlled trials (Gupta, 2011; Moher, Schulz, Altman, & CONSORT Group, 2001). As shown in Figure 1, due to poor adherence to the recruitment and randomization protocol at the community sites, however, many participants either were not randomized at all (Site D), were at a site with no control group (Site E), or did not comply with their randomization assignment (Sites A, B, and C). To capture as much information as possible from this study, we amended our analysis plan so that we conducted three sets of analyses: (1) ITT analyses with the participants from sites using the RCT design; (2) as-treated, quasi-experimental analyses with the participants from sites with both intervention and control groups; and (3) dose-response analyses with the participants from all sites who attended any AMP classes.
Figure 1.
Flow diagram for each set of analyses in modified randomized trial.
Note. We conducted three sets of analyses: intention-to-treat (ITT) analyses with the participants from sites using the randomized controlled trial (RCT) design (dark-gray boxes); as-treated (AT), quasi-experimental analyses with the participants from sites with both intervention and control groups (light-gray boxes); and dose-response analyses with the participants from all sites who attended any Aging Mastery Program (AMP) classes (white boxes). For the ITT analyses, we included Sites A, B, and C because they used random assignment, and we excluded 13 participants from these sites who joined the study after randomization. For the AT analyses, we included participants from Sites A, B, C, and D. Site E was excluded from these analyses because there was no control group at Site E. For the dose–response analyses, we included all participants from any site who attended any AMP class.
The ITT analyses used regression models with condition assignment predicting pre–post change in each outcome of interest adjusting for site (the randomization stratification factor). The as-treated (AT) analyses used regression models with attendance (i.e., whether attended at least one class) predicting pre–post change in each outcome of interest adjusting for site. Additional AT regression models were used to test whether observed changes were consistent after adjusting for age, gender, high school education, income, and number of chronic conditions.3 The dose–response analyses tested whether people who attended more classes had larger changes in the outcomes of interest over time. Specifically, in regression models for participants who attended at least one AMP class, number of classes attended was used to predict magnitude of change in each outcome of interest.
Results
Participants
We recruited 180 participants across all five sites. The sample ranged from 52 to 93 years old and was predominantly female. Less than half the sample was non-Latino White and less than one third attended college. Participants managed multiple health conditions such as arthritis (46% of sample), high blood pressure (42%), high cholesterol (41%), depression or anxiety (22%), and chronic pain (22%) among others. Table 2 presents additional sample descriptive information.
Table 2.
Baseline Recruited Sample Characteristics.
| Overall |
Site |
|||||
|---|---|---|---|---|---|---|
| Characteristics | Total (N = 180) | A (N = 38) | B (N = 47) | C (N = 37) | D (N = 40) | E (N = 18) |
| Age, years, M (SD) | 73.8 (10.0) | 78.6 (9.4) | 70.2 (7.8) | 71.8 (9.6) | 77.2 (11.4) | 69.8 (8.5) |
| Female, n (%) | 121 (67.2) | 26 (68.4) | 30 (63.8) | 18 (48.7) | 34 (85.0) | 13 (72.2) |
| Race/ethnicity, n (%) | ||||||
| White | 81 (45.0) | 31 (81.6) | 9 (19.2) | 6 (16.2) | 34 (85.0) | 1 (5.6) |
| Hispanic, Latino, or Spanish | 49 (27.2) | 4 (10.5) | 14 (29.8) | 22 (59.5) | 2 (5.0) | 7 (38.9) |
| Black or African American | 24 (13.3) | 2 (5.3) | 17 (36.2) | 2 (5.4) | 3 (7.5) | 0 |
| Asian | 14 (7.8) | 0 | 4 (8.5) | 2 (5.4) | 0 | 8 (44.4) |
| Other or mixed race/ethnicity | 10 (5.6) | 1 (2.6) | 3 (6.4) | 3 (8.11) | 1 (2.5) | 2 (11.1) |
| American Indian or Alaska Native | 2 (1.1) | 0 | 0 | 2 (5.4) | 0 | 0 |
| Income level, n (%) | ||||||
| <$999 | 51 (28.3) | 3 (7.9) | 16 (34.0) | 20 (54.1) | 8 (20.0) | 4 (22.2) |
| $1000–$1999 | 52 (28.9) | 11 (28.9) | 18 (38.3) | 6 (16.2) | 13 (32.5) | 4 (22.2) |
| $2000–$2999 | 29 (16.1) | 6 (15.8) | 8 (17.0) | 4 (10.8) | 6 (15.0) | 5 (27.8) |
| $3000–3999 | 16 (8.9) | 2 (5.3) | 2 (4.3) | 2 (5.4) | 8 (20.0) | 2 (11.1) |
| ≥ $4000 | 19 (10.8) | 13 (34.2) | 0 | 0 | 4 (10.0) | 2 (11.1) |
| Refused | 13 (7.2) | 3 (7.9) | 3 (6.4) | 5 (13.5) | 1 (2.5) | 1 (5.6) |
| Education level, n (%) | ||||||
| Completed high school | 151 (84.4) | 36 (94.7) | 41 (89.1) | 20 (54.1) | 36 (90.0) | 18 (100) |
| Completed 4 years of college | 57 (31.8) | 16 (42.1) | 8 (17.4) | 7 (18.9) | 20 (50.0) | 6 (33.3) |
| No. of comorbid conditions, M (SD) | 3.1 (2.2) | 3.2 (2.6) | 3.1 (2.1) | 2.3 (1.8) | 3.6 (2.2) | 3.3 (2.4) |
| Baseline level of outcomes of interest, M (SD) | ||||||
| Quality of life (CASP-19) | 41.4 (8.8) | 43.0 (7.2) | 40.8 (10.6) | 41.7 (6.9) | 40.3 (9.5) | 41.0 (9.6) |
| Physical health (PROMIS Global Health) | 43.5 (9.0) | 42.7 (8.0) | 43.2 (10.4) | 44.4 (8.1) | 43.7 (9.2) | 43.6 (9.1) |
| Mental health (PROMIS Global Health) | 48.1 (8.5) | 47.2 (7.7) | 49.3 (9.4) | 50.5 (9.2) | 45.8 (7.9) | 47.1 (6.6) |
| Patient activation (PAM score) | 68.6 (16.8) | 68.9 (12.9) | 67.0 (17.8) | 72.0 (17.9) | 68.6 (17.4) | 65.1 (18.8) |
| Graduation rate if attended AMP, % | 82% | 89% | 85% | 67% | 75% | 88% |
Note. CASP-19 = 19-item control, autonomy, pleasure, and self-realization assessment; PROMIS = Patient-Reported Outcomes Measurement Information System; PAM = Patient Activation Measure; AMP = Aging Mastery Program. Not all recruited participants had complete data on all baseline items. Thus, some characteristics have n < 180. For example, two participants reported their age as “65+” and were not included in the mean age calculations. Participants graduated when they attended at least 7 out of the 10 total class sessions.
As described above, community site adherence to the recruitment and randomization protocol was poor, so we amended our analysis plan to conduct not only the pre-planned ITT analysis, but also an analysis limited to participants who were randomized, and a third analysis simply including anyone who was exposed to the intervention regardless of randomization assignment or whether the site even had a control group. Information illustrating which participants were included in each set of analyses (e.g., how many participants were randomized to AMP or the wait-list control) is presented in Figure 1. Among the participants who enrolled at baseline, 125 (69%) also completed a follow-up survey. Due to study attrition and missing data, 71 participants were included in ITT analyses, 106 participants were included in AT analyses, and up to 75 participants were included in dose-response analyses (depending on the outcome of interest).
Participant nonadherence to condition assignment (i.e., attending the AMP class when randomized to the wait-list control or not coming to at least one AMP class when randomized to the class) differed across sites, with Site C having the highest nonadherence. The only other correlate of nonadherence, other than site, was race/ethnicity; Latino participants were more adherent than White participants. There was no differential attrition (i.e., “drop out”) between the AMP and waitlist groups. However, older participants were less likely to complete the postassessment, and White and Latino participants were less likely than Black participants to complete the postassessment.
Intention-to-Treat Analyses
At baseline, the two study arms were equivalent; the participants assigned to AMP and the wait-list control at each site had comparable scores on each of the outcomes of interest. As illustrated in Table 3, the ITT analyses showed that participants assigned to AMP had no statistically significant improvement over time in overall quality of life, physical health, mental health, or patient activation (score or level) relative to the participants assigned to the wait-list control. Participants assigned to AMP had a mean increase in their self-reported mental health t scores over time of 2.1 points, but the increase was not statistically significantly larger than for the waitlist control group.
Table 3.
Results of Intention-to-Treat Analyses.
| Change from baseline |
AMP regression coefficient predicting change, adjusting for site |
|||||
|---|---|---|---|---|---|---|
| AMP (n = 39) |
Wait-list control (n = 32) |
|||||
| Outcome | n | Unadjusted mean [95% CI] | n | Unadjusted mean [95% CI] | b [95% CI] | p |
| Quality of life | 35 | 0.2 [−2.0, 2.4] | 28 | 0.8 [−2.6, 4.2] | −0.6 [−4.5, 3.3] | .77 |
| Physical health | 39 | 0.3 [−2.1, 2.7] | 32 | 0.4 [−3.2, 3.9] | −0.1 [−4.2, 4.0] | .97 |
| Mental health | 39 | 2.1 [0.02, 4.1] | 32 | −0.8 [−3.7, 2.1] | 2.8 [−0.6, 6.3] | .10 |
| PAM score | 37 | 2.5 [−2.4, 7.4] | 27 | 0.3 [−4.7, 5.4] | 2.4 [−4.7, 9.5] | .50 |
| PAM level | 37 | 0.2 [−0.1, 0.4] | 27 | −0.04 [−0.3, 0.2] | 0.2 [−0.1, 0.5] | .24 |
Note. AMP = Aging Mastery Program; PAM = Patient Activation Measure. Only includes the three sites that allowed random assignment (A-C). Analyses are based on assigned condition, not actual exposure (12 assigned to control attended AMP; 18 assigned to AMP attended no classes). There were 39 in the treatment condition who completed at least one outcome at postassessment (out of 56 who were randomized to the treatment arm) and 32 in the wait-list control who completed at least one outcome at post-assessment (out of 53 who were assigned to the waitlist arm). Boldfaced results have 95% confidence intervals that do not include zero, which indicates a statistically significant increase in the outcome from baseline.
As-Treated Analyses
Participants who attended at least one AMP class had a larger improvement in self-reported mental health over time than participants who did not attend AMP at all (p = .007; see Table 4); the intervention group t scores increased by 2.3, while the control group nonsignificantly decreased by 1.2. The relative increase of 3.6 points in mental health t scores is a small to medium effect size.
Table 4.
Results of “As-Treated” Analyses.
| Change from baseline |
AMP attendance regression coefficient predicting change, adjusting for site |
|||||
|---|---|---|---|---|---|---|
| Attended AMP (n = 60) |
No class (n = 46) |
|||||
| Outcome | n | Unadjusted mean [95% CI] | n | Unadjusted mean [95% CI] | b [95% CI] | p |
| Quality of life | 56 | 0.6 [−1.2, 2.5] | 42 | −0.5 [−3.0, 1.9] | 1.1 [−2.0, 4.1] | .49 |
| Physical health | 60 | 0.7 [−1.3, 2.7] | 46 | −0.8 [−3.2, 1.6] | 1.4 [−1.8, 4.5] | .39 |
| Mental health | 60 | 2.3 [0.8, 3.9] | 46 | −1.2 [−3.4, 1.0] | 3.6 [1.0, 6.2] | .007 |
| PAM score | 56 | 1.1 [−3.2, 5.4] | 42 | 1.0 [−3.3, 5.3] | 0.6 [−5.8, 7.0] | .86 |
| PAM level | 56 | 0.1 [−0.1, 0.3] | 42 | 0.1 [−0.2, 0.3] | 0.02 [−0.3, 0.3] | .92 |
Note. AMP = Aging Mastery Program; PAM = Patient Activation Measure. Only includes the four sites that had control groups (Sites A-D). Of the 70 who attended AMP, 60 completed at least one outcome at postassessment, while of the 79 who attended no AMP classes, 46 completed at least one outcome at postassessment. Boldfaced results have 95% confidence intervals that do not include zero, which indicates a statistically significant increase in the outcome from baseline.
On further analysis of this scale, we found the primary contributor to the observed association between attending an AMP class and improvement in the self-reported mental health was the single item, “In general how would you rate your satisfaction with your social activities and relationships.” Controlling for site, participants who attended AMP had a larger increase (on average 0.4 points more) on this item relative to participants who did not attend AMP, SE = 0.2, p = .033, 95% CI [0.03, 0.8]. This item was scored from 1 = poor to 5 = excellent.
The participants who attended AMP did not show any other statistically significant improvements (and did not have greater improvements relative to the participants who did not attend AMP) in any other measured outcomes. Attending AMP was not significantly associated with self-reported quality of life, physical health, or patient activation.
Dose–Response Analyses
Among participants who attended at least one AMP class, participants who attended more classes were less likely to drop-out of the study and more likely to complete the post-assessment than participants who attended fewer classes (within each site). Among the participants who attended at least one class, number of classes attended was not associated with the magnitude of change in any of the outcomes of interest (see Table 5).
Table 5.
Results of Dose–Response Analyses.
| Change in outcome | N = 75 | Association with attendance (b) | p |
|---|---|---|---|
| Quality of life | 71 | −0.4 | .36 |
| Physical health | 75 | −0.1 | .82 |
| Mental health | 75 | −0.3 | .46 |
| PAM score | 71 | 0.9 | .43 |
| PAM level | 71 | −0.01 | .83 |
Note. PAM = Patient Activation Measure. Pre–post analysis of all participants, who were exposed to the intervention, regardless of randomization assignment or whether there was a control group at that site. Of the 88 participants who attended AMP, 75 completed at least one outcome on the pre- and postassessment.
Participants who attended at least one class were less likely to drop out of the study and more likely to complete the post-assessment questionnaire than participants who did not attend any classes (controlling for site). There was also overall differential attrition by site. Compared with Site D, Sites B and C had significantly more attrition. Controlling for site, older participants were again less likely to complete the post-assessment.
Discussion
This study examined the impact of AMP on the health and well-being of older adults in Los Angeles County. ITT analyses—specified a priori because this is the recommended method for testing efficacy of interventions (Gupta, 2011; Moher, 2001)—found AMP assignment did not affect any of the measured outcomes. By definition this was a negative study.
Though the RCT design and ITT analyses were intended to provide the strongest causal inference, we overestimated the extent to which community sites would follow the recruitment and randomization protocol. The net result was that statistical power was dramatically decreased by the smaller than planned sample size (only three sites used random assignment). Random error from participant nonadherence to condition assignment also limited our ability to detect intervention effects. With all these limitations in the operationalization of the study protocol, we cannot conclude with certainty that the program cannot improve health outcomes in these settings; we only know that it did not do so in this study as it was executed.
It is not surprising that the AT analyses found a single outcome that was associated with the intervention (self-reported mental health and specifically the single item about satisfaction with social activities and relationships); this is a much less conservative study design than the ITT and does not prove causality. It is discouraging that dose–response analyses did not find evidence that attending more classes was associated with larger increases in mental health; however, it is encouraging that within each site, participants who attended more classes were less likely to drop out of the study and more likely to complete the postassessment. The participants who only attended a small number of classes but still completed the postassessment may have been especially satisfied with their experience. Indeed, a subset of participants who completed the postassessment also completed anonymous satisfaction surveys for NCOA and 96% of these participants said AMP was fun and that they would recommend AMP to a friend. Together, the results indicate that participants enjoyed AMP and that participating can contribute to improved satisfaction with social activities. Given the group-based nature of the intervention, this encouragingly shows that attending AMP was a satisfying social activity.
How might AMP be improved to have greater impact on broader measures of quality of life? The AMP curriculum is consistent with recommendations from some health behavior change models, but other models highlight possible areas for improvement. Consistent with the health belief model (Janz & Becker, 1984), the AMP curriculum aims to shift perceived susceptibility, severity, benefits, and barriers through content education and group discussions and provides cues to action through the incentivized action steps. The program also incorporated goal setting and self-monitoring—both key aspects of social cognitive theory (Bandura, 2001), recommending participants set personalized goals adopting learned skills into their daily habits even after program completion. The curriculum encourages participants to contemplate healthy behavior change, prepare for change, and take action, which is consistent with the transtheoretical/stages of change model (Prochaska & Velicer, 1997). The current project unfortunately was not able to measure these behavioral constructs; future work should directly test these potential mechanisms and whether AMP successfully shifts them in the intended direction.
Although participants often take action during the program, AMP might increase impact on quality of life and health outcomes by increasing its focus on activation and promoting behavior change maintenance. With the diverse topic curriculum, participants are only held directly accountable for any given behavior change over the course of 1 week. Bolstering the training components, which guide facilitators to check in more consistently with participants about incorporation of learned skills and goals into their daily lives, may promote sustained behavior change. More important, the curriculum was not specifically designed to increase patient activation (i.e., it did not explicitly address patients’ confidence related to following through on medical treatments, interactions with medical providers, and ability to resolve health problems). The curriculum might benefit from expansion to include modules from other successful programs like the Chronic Disease Self-Management Program (Lorig, Sobel, Ritter, Laurent, & Hobbs, 2001) if it intends to promote changes in patient activation.
While AMP offers guidance on fidelity testing and implementation checklists, program sites may not have always used these tools. Moreover, although NCOA provided facilitator training at each site, guest speakers were not required to attend this orientation. Sites used anywhere from two to nine guest speakers to deliver AMP, and variation in the program’s implementation introduced additional statistical noise. As mentioned above, only three of the sites followed the planned randomization protocol for this study. Future evaluations should strive to continue to strengthen community site understanding of, investment in, and adherence to the planned evaluation design. The L.A. CAPRA research team is planning to partner with the community sites to conduct a process evaluation to advance future program evaluation research efforts. Improvements in process collaboration, fidelity assessment, and program evaluation implementation could reduce random error and make it easier to detect intervention effects in future evaluations. Moreover, culturally tailoring the intervention to diverse aging populations might increase its impact; the L.A. CAPRA research team is currently conducting site interviews to identify how the program may need to be modified beyond translating materials.
In sum, we did not find evidence that AMP improved quality of life, physical health, or patient activation in this community-based study with many operational challenges. Nonetheless, this study provides preliminary evidence that attending AMP was associated with improvements in global mental health and satisfaction with social activities, in particular.
Acknowledgments
We would like to acknowledge our NCOA collaborators, Hayoung Kye and Angelica Herrera-Venson, for their assistance with implementing the Aging Mastery Program (AMP), for sharing their expertise on AMP and its previous evaluations, and for their feedback on this article. Our research team at UCLA received no payment from NCOA for this evaluation and NCOA had no involvement in the data analysis or interpretation of results. We would also like to acknowledge the critical contributions of Ilene Parker, site leader and class facilitator, Sherman Oaks East Valley Center; Mandi Carpenter, site leader and class facilitator, Freda Mohr, Jewish Family Services; Arthur Zarigan, class facilitator, Antelope Valley; Juan Carlos Martinez, class facilitator, San Pedro; Ross Lenihan, class facilitator, Potrero Heights; Angela Bagmanian, site leader, Antelope Valley; Gregory Robinson, site leader, San Pedro; Maria Cerdas, site leader, Potrero Heights; Erika Brown, lead for the City of Los Angeles; and Guillermo Medina, lead for Los Angeles County.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: NIH/NIA Mid-career Award in Patient-Oriented Research (1K24AGO47899), NIH/NIA UCLA Resource Center for Minority Aging Research/Center for Health Improvement of Minority Elders (RCMAR/CHIME; 2P30AG081684), and NIH National Center for Advancing Translational Science (NCATS) UCLA CTSI Grant Number (UL1TR001881).
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The presented Cronbach α values reflect reliability of the full available participant data in the current study.
Participants who reported Latino ethnicity were categorized as Latino even if they also endorsed multiple races; other participants who self-reported multiple races were grouped in the “other” race/ethnicity category.
Current presented results do not adjust for race/ethnicity (to avoid multicollinearity with site), but the conclusions are the same when also adjusting for race/ethnicity.
References
- Acumen, LLC (2017, November). Wellness prospective evaluation report on six-month follow-up survey outcomes and estimated operational costs (Executive Summary). Retrieved from https://downloads.cms.gov/files/cmmi/community-basedwellnessrrevention-sixthmnthoutcomes-operationalcostrpt.pdf
- Bandura A (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26. [DOI] [PubMed] [Google Scholar]
- Belza B, Altpeter M, Smith ML, & Ory MG (2017). The Healthy Aging Research Network: Modeling collaboration for community impact. American Journal of Preventive Medicine, 52(Suppl. 3), S228–S232. [DOI] [PubMed] [Google Scholar]
- Davitt JK, Greenfield E, Lehning A, & Scharlach A (2017). Challenges to engaging diverse participants in community-based aging in place initiatives. Journal of Community Practice, 25, 325–343. doi: 10.1080/10705422.2017.1354346 [DOI] [Google Scholar]
- Depp C, Vahia IV, & Jeste D (2010). Successful aging: Focus on cognitive and emotional Health. Annual Review of Clinical Psychology, 6, 527–550. Retrieved from https://www.annualreviews.org/doi/abs/10.1146/annurev.clinpsy.121208.131449. doi: 10.1146/annurev.clinpsy.121208.131449 [DOI] [PubMed] [Google Scholar]
- Dickens AP, Richards SH, Greaves CJ, & Campbell JL (2011). Interventions targeting social isolation in older people: a systematic review. BMC Public Health, 11, 647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ferretti L, McCallion P, McDonald E, Kye H, Hererra-Vensom A, & Firman J (2018). Assessing the effectiveness of the Aging Mastery Program. Healthcare (Basel, Switzerland), 6, 41. doi: 10.3390/healthcare6020041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greene J, & Hibbard JH (2012). Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. Journal of General Internal Medicine, 27, 520–526. doi: 10.1007/s11606-011-1931-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gupta SK (2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3), 109–112. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3159210/. doi: 10.4103/2229-3485.83221 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hays RD, Bjorner JB, Revicki DA, Spritzer KL, & Cella D (2009). Development of physical and mental health summary scores from the Patient-Reported Outcomes Measurement Information System (PROMIS) global items. Quality of Life Research, 18, 873–880. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Henning-Smith C (2016). Quality of life and psychological distress among older adults: The role of living arrangements. Journal of Applied Gerontology, 35, 39–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herrera-Venson A (2018). Compendium of research and impact on older adult’s quality of life. Unpublished manuscript. National Council on Aging, Arlington, VA. [Google Scholar]
- Hibbard JH, Stockard J, Mahoney ER, & Tusler M (2004). Development of the Patient Activation Measure (PAM): Conceptualizing and measuring activation in patients and consumers. Health Services Research, 39(4 Pt. 1), 1005–1026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hyde M, Wiggins RD, Higgs P, & Blane DB (2003). A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19). Aging & Mental Health, 7, 186–194. doi: 10.1080/1360786031000101157 [DOI] [PubMed] [Google Scholar]
- Janz NK, & Becker MH (1984). The health belief model: A decade later. Health Education Quarterly, 11, 1–47. doi: 10.1177/109019818401100101 [DOI] [PubMed] [Google Scholar]
- Lorig KR, Sobel DS, Ritter PL, Laurent D, & Hobbs M (2001). Effect of a self-management program on patients with chronic disease. Effective Clinical Practice, 4, 256–262. [PubMed] [Google Scholar]
- Martin P, Kelly N, Kahana B, Kahana E, Willcox BJ, Willcox DC, & Poon LW (2015). Defining successful aging: A tangible or elusive concept? The Gerontologist, 55, 14–25. doi: 10.1093/geront/gnu044 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher D, Schulz KF, & Altman DG, & CONSORT Group (Consolidated Standards of Reporting Trials). (2001). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Annals of Internal Medicine, 134, 657–662. [DOI] [PubMed] [Google Scholar]
- Mosen DM, Schmittdiel J, Hibbard J, Sobel D, Remmers C, & Bellows J (2007). Is patient activation associated with outcomes of care for adults with chronic conditions? Journal of Ambulatory Care Management, 30, 21–29. [DOI] [PubMed] [Google Scholar]
- Ory MG, Smith ML, Ahn S, Jiang L, Lorig K, & Whitelaw N (2014). National study of chronic disease self-management: Age comparison of outcome findings. Health Education & Behavior, 41(1 Suppl), 34S–42S. [DOI] [PubMed] [Google Scholar]
- Prochaska JO, & Velicer WF (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12, 38–48. [DOI] [PubMed] [Google Scholar]
- Pruchno R (2015). Successful aging: Contentious past, productive future. The Gerontologist, 55, 1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robare JF, Bayles CM, Newman AB, Williams K, Milas C, Boudreau R, … Kuller LH (2011). The “10 keys” to healthy aging: 24-month follow-up results from an innovative community-based prevention program. Health Education & Behavior, 38, 379–388. [DOI] [PubMed] [Google Scholar]
- Smith ML, Ory MG, Ahn S, Bazzarre TL, & Resnick B (2011). Older adults’ participation in a community-based falls prevention exercise program: Relationships between the EASY tool, program attendance, and health outcomes. The Gerontologist, 51, 809–821. [DOI] [PubMed] [Google Scholar]
- Zgibor JC, Ye L, Boudreau RM, Conroy MB, Vander Bilt J, Rodgers EA, … Albert SM. (2017). Community-based healthy aging interventions for older adults with arthritis and multimorbidity. Journal of Community Health, 42, 390–399. [DOI] [PMC free article] [PubMed] [Google Scholar]