Abstract
Objectives
To identify factors associated with weekly adherence to a 12-week tracker-based activity intervention in older adults.
Method
Using generalized linear mixed models in a secondary analysis (n = 35), we examined factors associated with weekly adherence (defined as wore the Jawbone Up24 tracker for ≥ 5 days and completed weekly calls). Factors examined included: individual characteristics, gave and received virtual support, achieved step goals, barrier and task self-efficacy, and self-comment.
Results
Participants’ (Mage = 61.7, SDage = 5.7) adherence changed from 86% (week 1) to 74% (week 12). Achieved the previous week’s goal (β = 1.13, p = 0.01) and received virtual support (β = 0.01, p = 0.02) significantly increased the odds of weekly adherence.
Discussion
Achieved step goals and received virtual support were associated with improved adherence to our tracker-based activity intervention, which has promising potential to be translated into the clinical setting to promote active lifestyles.
Keywords: physical activity, adherence, activity tracker, successful aging, health promotion
Introduction
Physical activity may help older adults maintain their independence and delay development of mobility disability (Crane, Macneil, & Tarnopolsky, 2013). However, only one in six older adults meet the recommended 150 minutes of activity per week or 7,000 steps per day for healthy older adults (Jefferis et al., 2014; Tudor-Locke et al., 2011). Physical activity interventions promote an increase in activity (Chase, 2015; Geraedts, Zijlstra, Bulstra, Stevens, & Zijlstra, 2013) and improve physical functioning (Campbell et al., 1997). Nevertheless, interventions can only be effective if participants adhere to the intervention (Martin & Sinden, 2001).
Factors associated with adherence to exercise interventions or activity recommendations among aging adults include: younger age (Aartolahti, Tolppanen, Lönnroos, Hartikainen, & Häkkinen, 2015; Jefferis et al., 2014), men (Cox et al., 2013; Abby C. King, 2001), White (Irwin et al., 2004), greater self-efficacy (Cox et al., 2013; Jefferis et al., 2014), normal weight (Abby C. King, 2001), and greater social support (Visser, Brychta, Chen, & Koster, 2014). Yet, adherence to activity interventions remains an unresolved issue among older adults (Mercer et al., 2016; Nyman & Victor, 2012). In fact, in one study adherence rates decreased from 80% to 50% over 12 months (Nyman & Victor, 2012). Technology-based activity interventions may improve adherence among older adults (Valenzuela, Okubo, Woodbury, Lord, & Delbaere, 2016). Wearable activity trackers (e.g., Fitbit, Jawbone) are a promising tool that can augment existing activity interventions to increase adherence (Lisa Cadmus-Bertram, Marcus, Patterson, Parker, & Morey, 2015; Mercer et al., 2016). However, factors associated with tracker-based activity interventions and adherence remained largely unstudied.
With the anticipated increase in the older adult population (Ortman, Velkoff, & Hogan, 2014) and an increase in the adoption of technological devices (Gell, Rosenberg, Demiris, LaCroix, & Patel, 2013; Mercer et al., 2016), it is critical to identify factors that are associated with adherence to an activity program using technology. Thus, we aimed to identify factors that are associated with weekly adherence to a 12-week tracker-based physical activity intervention.
Method
Participants
We conducted a secondary data analysis using data from a 12-week intervention using Jawbone™ Up24 and its associated mobile application (app). Here, we included data from 35 of the 40 participants (19 intervention and 16 control) because only 35 were able to interact with other study participants (peers) during the 12-week intervention. However, the number of peers fluctuated from zero to 10 because of continuous enrollment.
Procedure
Details of the primary study was published elsewhere (Lyons, Swartz, Lewis, Martinez, & Jennings, 2017). Briefly, participants were randomized into the intervention group (n = 20) or the wait-list control group (n = 20). Eligibility criteria included: 1) Aged 55 to 79, 2) body mass index (BMI), 25 to 35 kg/m2, 3) < 60 minutes of activity per week. The intervention provided an Up24 activity tracker, an iPad mini with an Up account using pseudonyms (e.g. Monopoly® pieces), negotiated weekly step goals and a step goal of 7,000 steps per day by the end of the intervention (Tudor-Locke et al., 2011), and 12 weekly counseling calls. Up accounts were connected to peers and the research lab account so usage could be monitored. During orientation, the interactive “likes/comments” features were demonstrated. Per protocol, research staff did not comment on the app. Staff trained in behavioral counseling conducted the weekly calls. Calls consisted of reviewing the previous week’s activity, counseling topics, and setting goals. The counseling topics were scripted based on the Social Cognitive Theory (Bandura, 2004). We offered wait-list control participants the intervention after a 12-week waiting period, but some did not participate (n=4). The intervention protocol was approved by the University of Texas Medical Branch Institutional Review Board (#13-071).
Measures (variables of interest)
The primary outcome of the current study was weekly intervention adherence (adhere or not adhere). The adherence variable was computed on a weekly basis. We combined the following variables: whether a participant wore the activity tracker for ≥ 5 days/week (> 0 steps on the app/day) and whether a participant completed the counseling call for each week. The independent factors investigated in the analytical model were as follows: individual characteristics (age, baseline BMI, gender, race/ethnicity), virtual support variables (total number of “likes/comments” a participant received and/or given over the 12-week period), achieved the previous week’s step goal, self-comment (total number of comments participants gave themselves on the app over the 12-week period), baseline barrier self-efficacy (confidence in being active when facing barriers), and baseline task self-efficacy (perceived ability to exercise as planned). Barrier and task self-efficacy were evaluated by using the validated Rogers et al. scale (Rogers et al., 2006). The scale consisted of 17 questions, 13 for barrier and four for task self-efficacy. Jawbone’s step count was validated (Evenson, Goto, & Furberg, 2015).
Statistical Approach
We used a generalized linear mixed model (GLMM) with purposeful selection of covariates and backwards selection to identify independent factors associated with weekly adherence over the 12 week period (Hosmer Jr, Lemeshow, & Sturdivant, 2013). GLMMs allowed us to evaluate weekly adherence over time. Independent factors with a p < .25 in a bivariate analysis were included in the multivariable model. Backwards selection was conducted iteratively; variables with the largest p-value were removed until all variables included had a p < .25. We examined the pseudo-AICC and pseudo-BIC to ensure each was reduced after removal of variables. The final model had the smallest pseudo-AICC and pseudo-BIC values. Factors with a p < .05 in the final model were reported as statistically significant. Analyses were conducted with SAS 9.4 (Cary, NC).
Results
Mean age was 61.7 years (SD = 5.9). The majority were female and more than half were Non-Hipsanic white (Table 1). Participants were also overweight (M = 30.2, SD = 3.34 kg/m2) (Table 1). On average, participants wore the tracker 81 out of 84 days. Ninety-five percent wore it for five or more days each week. The mean percentage of counseling sessions completed was 80%. The percentage of weekly adherence to the intervention fluctuated from 86% (week 1) to 74% (week 12) (Figure 1). On average, 75% of participants adhered (completion of weekly counseling calls combined with activity tracker usage) to the intervention over the 12-week period. Weekly step average ranged from 5453 (week 1) to 6875 (week 12) (Figure 2). Figure 3 depicts a type of virtual support from peers on the app. On average as a group over the 12-week period, participants received less virtual support (M = 123, SD = 141) than was given to peers (M = 141, SD = 278). On average, the total number of self-comments over the 12-week period was 22 (SD = 44).
Table 1.
Participant Characteristics (N = 35)
Mean | SD | Range | |
| |||
Age (years) | 61.67 | 5.86 | 55–79 |
BMI (kg/m2) | 30.24 | 3.34 | 25–35.28 |
Days monitored | 6.5 | 0.56 | 5–7 |
Number of “likes/comments” received | 123.39 | 141.02 | 0–579 |
Number of “likes/comments” given | 141.16 | 278.82 | 0–1326 |
Self-comment | 21.76 | 44.10 | 0 to 232 |
Barrier self-efficacy | 48.96 | 26.44 | 0–100 |
Task self-efficacy | 55.23 | 24.27 | 0–100 |
| |||
N | % | ||
| |||
Gender | |||
Male | 6 | 17 | |
Female | 29 | 83 | |
Race/Ethnicity | |||
Non-Hispanic White | 20 | 57 | |
Hispanics | 10 | 29 | |
Other | 5 | 14 |
Figure 1.
Weekly Adherence (%)
Figure 2.
Weekly average steps
Figure 3.
Example of “like/comment” on the Up app using research staff account.
The final model is presented in Table 2. Achieved the previous week’s step goal and received virtual support significantly increased the odds of weekly intervention adherence. The odds of weekly intervention adherence for those who met the previous week’s step goal is 3.10 times higher than the odds for those who did not meet the previous week’s step goal. Regarding virtual support received, an increase in total comments received by one “like/comment” showed a one percent increase in the odds of weekly intervention adherence.
Table 2.
Final Adherence Model
Independent Factors |
Estimate (SE) | Odds Ratio Estimates |
95% CI | Significance |
---|---|---|---|---|
Intercept | −1.62 (0.595) | <0.05 | ||
Number of “likes/comments” received | 0.01 (0.004) | 1.01 | 1.002 to 1.020 | <0.05 |
Achieve previous week’s step goal | 1.13 (0.407) | 3.10 | 1.391 to 6.899 | <0.01 |
Barrier self-efficacy | 0.02 (0.012) | 1.02 | 0.990 to 1.041 | 0.24 |
Task self-efficacy | 0.02 (0.011) | 1.02 | 0.993 to 1.041 | 0.16 |
Pseudo-AICC | 1947.84 | |||
Pseudo-BIC | 1949.29 |
Discussion
Our study took the first step to improve activity intervention adherence among older adults. We found high adherence to a 12-week tracker-based activity intervention in our sample of overweight, sedentary older adults. Our exploratory results suggest that setting attainable step goals (achieved the previous week’s step goal) and receiving virtual support improved activity intervention adherence in older adults.
There was a slight decrease in adherence between week 1 and 12, but this was expected (L. Cadmus-Bertram et al., 2014; Martin & Sinden, 2001). The percentage of participants who used the tracker for ≥ 5 days (95%) was comparable to a previous 16-week Fitbit-based activity intervention (Lisa Cadmus-Bertram et al., 2015). The high percentage of participants who wore the tracker in our study ≥ 5 days/week throughout the 12-week period indicated acceptance of the tracker, contrary to other research that reported older adults’ aversion towards using technology (Mitzner et al., 2010).
Previous research confirmed that achieving the previous week’s goals by setting attainable goals and receiving virtual support significantly improved adherence. Inclusion of telephone-based goal setting sessions that provided meaningful feedback on tracker data have helped participants achieve goals and sustain activity levels (Miyamoto, Henderson, Young, Pande, & Han, 2016; Young et al., 2014). Our intervention achieved a higher percentage of completed sessions (80%) within 12 weeks compared to other unsupervised-exercise interventions (54%) (Valenzuela et al., 2016). Furthermore, a social network feature that allows for social interaction is one of the key elements for motivating behavior change in mhealth interventions (Chang, Lu, Yang, & Luarn, 2016; A. C. King et al., 2016; Miyamoto et al., 2016). By using the tracker’s associated app, which required fewer resources, we were able to provide an environment of camaraderie often found in group-based interventions (Valenzuela et al., 2016). Our study not only found high adherence, but we also found that one more like/comment received was associated with a one percent increase in adherence, which suggests a potential mechanism for long-term adherence. Increased adherence could lead to increased number of steps and even a 10-minute bout of activity (~1000 steps) could improve health outcomes (i.e., improvements in cholesterol) (Loprinzi & Cardinal, 2013; Tudor-Locke et al., 2011). By incorporating virtual support and achieving the previous weeks goals, researchers and clinicians may be able to overcome decreased activity tracker usage over time (Ledger & McCaffrey, 2014).
The notable strength of our study was the longitudinal evaluation of weekly adherence data. We were able to examine adherence patterns and investigate factors that can potentially improve intervention adherence over time. However, our results must be interpreted with some caution. Our small sample size may have reduced the statistical power and we may have missed some potentially important associations, but we found significant factors that are reflected in the literature. Additionally, individualized step goals may not be generalizable. However, they are important for sustainability and achievement of goals. We were not able to obtain the wear time information because the Up app algorithm cannot distinguish between non-wear time and sedentary time. For future tracker-based studies, it is important to work with commercial tracker companies to obtain wear time information to assess adherence. Lastly, by including wait-list control individuals, we were not able to evaluate the adherence effect on physical activity. Future studies are needed with larger sample sizes to test long-term adherence and activity in older adults by setting smaller achievable goals and promoting virtual support on the activity tracker apps.
Conclusion
Our sample of older, overweight, and sedentary adults showed high adherence to wearing the activity tracker. We also found that achievable goals and virtual support may be important components in future intervention design for intervention adherence. Our activity-based intervention has a great potential to be translated into the clinical setting to enhance promotion of an active lifestyle. Using the social network feature included in the tracker’s app allows for anonymous interactions that may motivate individuals to become more active. The addition of telephone counseling sessions could promote long-term intervention adherence and maintenance of activity among patients.
Acknowledgments
This study was internally funded by the Claude D. Pepper Older Americans Independence Center (grant number P30AG024832) and Sealy Center on Aging. Additional salary support was provided by the Cancer Prevention Research Institute of Texas (grant number RP140020), the National Institute on Disability and Rehabilitation Research (grant number H133P110012), a Mentored Research Scholar Grant in Applied and Clinical Research (grant number MRSG-14-165-01-CPPB) from the American Cancer Society, and the American Heart Association (grant number 13BGIA17110021, 16PRE27090012).
Footnotes
Trial Registration: clinicaltrials.gov, NCT01869348
The University of Texas Medical Branch Institutional Review Board Protocol #: 13-071
Declaration of Conflict of Interests:
MCS's spouse has an equity interest in Apple Inc., a company that may potentially benefit from the research results. In addition, ZHL is employed by Beachbody, a company that may potentially benefit from the research results. ZHL’s employment began after data collection and analysis. UTMB’s Conflicts of Interest Committee has reviewed these conflicts and a management plan was implemented to prevent any appearance of a conflict of interests. Any inquiries regarding this management plan can be directed to UTMB's Office of Institutional Compliance, (409) 747-8701.
Author contributions to the manuscript:
MC Swartz planned the study, conducted the data analysis, and wrote the paper. MD Swartz supervised the data analysis and revised the manuscript. Z.H. Lewis, EM Martinez, and EJ Lyons assisted with study design and revised the manuscript.
Contributor Information
Maria C. Swartz, Email: mcswartz@utmb.edu.
Zakkoyya H. Lewis, Email: zalewisp@utmb.edu, zpowell@beachbody.com.
Michael D. Swartz, Email: Michael.D.Swartz@utmb.edu.
Eloisa Martinez, Email: esmartin@utmb.edu.
Elizabeth J. Lyons, Email: ellyons@utmb.edu.
References
- Aartolahti E, Tolppanen A-M, Lönnroos E, Hartikainen S, Häkkinen A. Health condition and physical function as predictors of adherence in long-term strength and balance training among community-dwelling older adults. Archives of Gerontology and Geriatrics. 2015;61(3):452–457. doi: 10.1016/j.archger.2015.06.016. [DOI] [PubMed] [Google Scholar]
- Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31(2):143–164. doi: 10.1177/1090198104263660. [DOI] [PubMed] [Google Scholar]
- Cadmus-Bertram L, Irwin M, Alfano C, Campbell K, Duggan C, Foster-Schubert K, McTiernan A. Predicting adherence of adults to a 12-month exercise intervention. J Phys Act Health. 2014;11(7):1304–1312. doi: 10.1123/jpah.2012-0258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cadmus-Bertram L, Marcus HB, Patterson ER, Parker AB, Morey LB. Use of the Fitbit to Measure Adherence to a Physical Activity Intervention Among Overweight or Obese, Postmenopausal Women: Self-Monitoring Trajectory During 16 Weeks. JMIR Mhealth Uhealth. 2015;3(4):e96. doi: 10.2196/mhealth.4229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell AJ, Robertson MC, Gardner MM, Norton RN, Tilyard MW, Buchner DM. Randomised controlled trial of a general practice programme of home based exercise to prevent falls in elderly women. BMJ. 1997;315(7115):1065–1069. doi: 10.1136/bmj.315.7115.1065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chang RC, Lu HP, Yang P, Luarn P. Reciprocal Reinforcement Between Wearable Activity Trackers and Social Network Services in Influencing Physical Activity Behaviors. JMIR Mhealth Uhealth. 2016;4(3):e84. doi: 10.2196/mhealth.5637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chase J-AD. Interventions to Increase Physical Activity Among Older Adults: A Meta-Analysis. The Gerontologist. 2015;55(4):706–718. doi: 10.1093/geront/gnu090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cox KL, Flicker L, Almeida OP, Xiao J, Greenop KR, Hendriks J, Lautenschlager NT. The FABS trial: a randomised control trial of the effects of a 6-month physical activity intervention on adherence and long-term physical activity and self-efficacy in older adults with memory complaints. Prev Med. 2013;57(6):824–830. doi: 10.1016/j.ypmed.2013.09.010. [DOI] [PubMed] [Google Scholar]
- Crane JD, Macneil LG, Tarnopolsky MA. Long-term aerobic exercise is associated with greater muscle strength throughout the life span. J Gerontol A Biol Sci Med Sci. 2013;68(6):631–638. doi: 10.1093/gerona/gls237. [DOI] [PubMed] [Google Scholar]
- Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act. 2015;12:159. doi: 10.1186/s12966-015-0314-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gell NM, Rosenberg DE, Demiris G, LaCroix AZ, Patel KV. Patterns of Technology Use Among Older Adults With and Without Disabilities. The Gerontologist. 2013 doi: 10.1093/geront/gnt166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Geraedts H, Zijlstra A, Bulstra SK, Stevens M, Zijlstra W. Effects of remote feedback in home-based physical activity interventions for older adults: a systematic review. Patient Educ Couns. 2013;91(1):14–24. doi: 10.1016/j.pec.2012.10.018. [DOI] [PubMed] [Google Scholar]
- Hosmer DW, Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. Vol. 398. John Wiley & Sons; 2013. [Google Scholar]
- Irwin ML, Tworoger SS, Yasui Y, Rajan B, McVarish L, LaCroix K, McTiernan A. Influence of demographic, physiologic, and psychosocial variables on adherence to a yearlong moderate-intensity exercise trial in postmenopausal women. Prev Med. 2004;39(6):1080–1086. doi: 10.1016/j.ypmed.2004.04.017. [DOI] [PubMed] [Google Scholar]
- Jefferis BJ, Sartini C, Lee IM, Choi M, Amuzu A, Gutierrez C, Whincup PH. Adherence to physical activity guidelines in older adults, using objectively measured physical activity in a population-based study. BMC Public Health. 2014;14:382. doi: 10.1186/1471-2458-14-382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- King AC. Interventions to Promote Physical Activity by Older Adults. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 2001;56(suppl 2):36–46. doi: 10.1093/gerona/56.suppl_2.36. [DOI] [PubMed] [Google Scholar]
- King AC, Hekler EB, Grieco LA, Winter SJ, Sheats JL, Buman MP, Cirimele J. Effects of Three Motivationally Targeted Mobile Device Applications on Initial Physical Activity and Sedentary Behavior Change in Midlife and Older Adults: A Randomized Trial. PLoS One. 2016;11(6):e0156370. doi: 10.1371/journal.pone.0156370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ledger D, McCaffrey D. Endeavour Partners Report: Inside Wearables: How the Science of Human Behavior Change Offers the Secret to Long-term Engagement. Endeavour website. 2014 (January 2014 ed.) [Google Scholar]
- Loprinzi PD, Cardinal BJ. Association between Biologic Outcomes and Objectively Measured Physical Activity Accumulated in ≥ 10-Minute Bouts and < 10-Minute Bouts. American Journal of Health Promotion. 2013;27(3):143–151. doi: 10.4278/ajhp.110916-QUAN-348. [DOI] [PubMed] [Google Scholar]
- Lyons EJ, Swartz MC, Lewis ZH, Martinez E, Jennings K. Feasibility and Acceptability of a Wearable Technology Physical Activity Intervention With Telephone Counseling for Mid-Aged and Older Adults: A Randomized Controlled Pilot Trial. JMIR Mhealth Uhealth. 2017;5(3):e28. doi: 10.2196/mhealth.6967. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martin KA, Sinden AR. Who Will Stay and Who Will Go? A Review of Older Adults’ Adherence to Randomized Controlled Trials of Exercise. Journal of aging and physical activity. 2001;9(2):91–114. doi: 10.1123/japa.9.2.91. [DOI] [Google Scholar]
- Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of Commercially Available Wearable Activity Trackers Among Adults Aged Over 50 and With Chronic Illness: A Mixed-Methods Evaluation. JMIR Mhealth Uhealth. 2016;4(1):e7. doi: 10.2196/mhealth.4225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mitzner TL, Boron JB, Fausset CB, Adams AE, Charness N, Czaja SJ, Sharit J. Older adults talk technology: Technology usage and attitudes. Computers in Human Behavior. 2010;26(6):1710–1721. doi: 10.1016/j.chb.2010.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miyamoto WS, Henderson S, Young MH, Pande A, Han JJ. Tracking Health Data Is Not Enough: A Qualitative Exploration of the Role of Healthcare Partnerships and mHealth Technology to Promote Physical Activity and to Sustain Behavior Change. JMIR Mhealth Uhealth. 2016;4(1):e5. doi: 10.2196/mhealth.4814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nyman SR, Victor CR. Older people's participation in and engagement with falls prevention interventions in community settings: an augment to the cochrane systematic review. Age and Ageing. 2012;41(1):16–23. doi: 10.1093/ageing/afr103. [DOI] [PubMed] [Google Scholar]
- Ortman JM, Velkoff VA, Hogan H. An aging nation: the older population in the United States. Washington, DC: US Census Bureau; 2014. pp. 25–1140. [Google Scholar]
- Rogers LQ, Courneya KS, Verhulst S, Markwell S, Lanzotti V, Shah P. Exercise barrier and task self-efficacy in breast cancer patients during treatment. Support Care Cancer. 2006;14(1):84–90. doi: 10.1007/s00520-005-0851-2. [DOI] [PubMed] [Google Scholar]
- Tudor-Locke C, Craig CL, Brown WJ, Clemes SA, De Cocker K, Giles-Corti B, Blair SN. How many steps/day are enough? for adults. The International Journal of Behavioral Nutrition and Physical Activity. 2011;8:79–79. doi: 10.1186/1479-5868-8-79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Valenzuela T, Okubo Y, Woodbury A, Lord SR, Delbaere K. Adherence to Technology-Based Exercise Programs in Older Adults: A Systematic Review. Journal of Geriatric Physical Therapy, Publish Ahead of Print. 2016 doi: 10.1519/jpt.0000000000000095. [DOI] [PubMed] [Google Scholar]
- Visser M, Brychta RJ, Chen KY, Koster A. Self-reported adherence to the physical activity recommendation and determinants of misperception in older adults. Journal of aging and physical activity. 2014;22(2):226–234. doi: 10.1123/japa.2012-0219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young H, Miyamoto S, Ward D, Dharmar M, Tang-Feldman Y, Berglund L. Sustained effects of a nurse coaching intervention via telehealth to improve health behavior change in diabetes. Telemed J E Health. 2014;20(9):828–834. doi: 10.1089/tmj.2013.0326. [DOI] [PMC free article] [PubMed] [Google Scholar]