Measurement of Self-Monitoring Web Technology Acceptance and Use in an e-Health Weight-Loss Trial

Abstract

Background: Research on technology acceptance and use in e-health weight-loss interventions is limited. Using data from a randomized controlled trial of two e-health interventions, we evaluated the acceptance and use of a self-monitoring Web site for weight loss. Materials and Methods: We examined eight theoretical constructs about technology acceptance using adapted 5-point Likert scales and the association of measured Web site usage and weight loss. Results: All scales had high internal consistency (Cronbach's alpha=0.74–0.97) in both interventions and at 3 and 15 months (end of intensive and maintenance intervention, respectively). From 3 to 15 months mean scores changed unfavorably for two constructs (compatibility and behavioral intention) among coach-led intervention participants, who received ongoing feedback on their self-monitoring entries. Among self-directed intervention participants, who received minimal coach support, mean scores changed unfavorably for five constructs (usefulness, ease of use, concern, compatibility, and behavioral intention). At 3 months, usefulness, ease of use, effect, compatibility, and behavioral intention in the coach-led group (Pearson r=0.33–0.5) and usefulness and affect in the self-directed group (r=0.43–0.46) were significantly correlated with Web site usage, which was correlated with weight loss (β=−0.02, p≤0.001 for both interventions). From 3 to 15 months, mean score changes for usefulness and behavioral intention correlated significantly with Web site usage in the coach-led group. Conclusions: The adapted acceptance measures showed acceptable psychometric properties and significant associations with actual Web site use, which correlated with weight loss. Better understanding of technology acceptance and use in e-health weight-loss interventions may improve participant adherence and outcome.

Introduction

More than two-thirds of U.S. adults are overweight or obese.¹ Lifestyle interventions for weight loss and maintenance that combine a balanced, calorie-deficit diet, increased physical activity, and behavioral skills training have proven effective to significantly reduce obesity-related health risks, such as type 2 diabetes mellitus and hypertension.^2–4 Traditionally, behavior therapy for obesity uses face-to-face interventions in individual or group settings, a delivery mode that, albeit effective, has notable limitations such as high participant and staff burden, associated costs, and limited reach. Technology-assisted intervention methods, or e-health interventions, may offer a cost-effective, scalable approach to behavioral weight management with potential for high public health impact.

“e-Health” refers to “the use of emerging information and communication technology, especially the Internet, to improve or enable health and healthcare.”⁵ Although e-health interventions for weight management, physical activity, and dietary changes have proliferated in recent years, reviews of the scientific literature find insufficient data to support their effectiveness, while noting low intervention adherence as a common problem.^6–9 These interventions typically operationalize and transform behavior change strategies (e.g., self-monitoring, goal setting, and behavioral feedback) proven in face-to-face interventions for delivery via use of technology. The number and type of technologies used, however, vary markedly among interventions, and it is unclear what specific technological features improve adherence and outcome.⁶ Small numbers of e-health weight management intervention studies have examined the association of technology use (e.g., number of logins and self-monitoring logs) and weight-loss outcome with equivocal results.^6,10 Furthermore, the field lacks theoretically grounded, validated instruments for measuring determinants of such technology usage behavior. Information systems research has yielded empirically supported theories and measures of technology acceptance and use.^11–13 To date, these measures have rarely been adapted for evaluation and validation in the context of e-health.

Using data from intervention participants in a primary care-based randomized controlled trial comparing two e-health weight-loss interventions and usual care,^14,15 this study examined the acceptance and use of a weight and physical activity self-monitoring Web site—an essential intervention component. The objectives were threefold. First, we analyzed the cross-sectional internal consistency and responsiveness to change over time of adapted scales from the information systems literature that measured eight theoretical constructs about technology acceptance (attitudes). Second, we assessed the association of the acceptance constructs with actual Web site usage (behavior). Third, we assessed the association of Web site usage and weight loss (outcome).

Materials and Methods

Evaluation of Lifestyle Interventions to Treat Elevated Cardiometabolic Risk in Primary Care (E-LITE) was a 15-month pragmatic clinical trial in which overweight or obese adults with prediabetes and/or metabolic syndrome were randomly assigned to one of three arms: (1) a coach-led weight-loss group intervention (n=79), (2) a self-directed DVD weight loss intervention (n=81), or (3) a usual-care control group (n=81). The coach-led and self-directed interventions led to significantly greater mean (95% confidence interval) weight loss (in kg) versus usual care through 15 months: −6.3 (−8.0, −4.5), −4.5 (−6.3, −2.7), and −2.4 (−4.2, −0.6), respectively. The complete trial protocol and primary findings were published previously.^14,15 The present study focuses on only participants in the coach-led and self-directed interventions who responded to the study Use of Technology Questionnaire, and pertinent methodological details are summarized below.

Participants

Participants were recruited at a 29-physician primary care clinic in a large, community-based, multispecialty group practice. Inclusion criteria included age 18 years or above, a body mass index of at least 25 kg/m², and the presence of prediabetes (defined by impaired fasting plasma glucose level of 100–125 mg/dL) and/or metabolic syndrome (defined by the 2005 joint criteria of the American Heart Association and the National Heart, Lung, and Blood Institute).¹⁶ Exclusion criteria included serious medical or psychiatric conditions (e.g., stroke, psychotic disorder), special life circumstances (e.g., pregnancy, planned move), and no regular access to a computer with Internet and e-mail capabilities.¹⁴

Interventions

The core of both interventions was the evidence-based, 12-session Group Lifestyle Balance™ (University of Pittsburgh Diabetes Prevention Support Center, Pittsburgh, PA) program,¹⁷ which was delivered in weekly group visits in the coach-led arm and via a take-home DVD in the self-directed arm. Lifestyle coach-participant communication continued for the remainder of the 15-month intervention period via secure e-mail within a comprehensive electronic health records system. Throughout the intervention, participants in both arms were instructed to set weight and physical activity goals and track their progress using the American Heart Association's free, secure Heart360^® Web site (https://www.heart360.org/). In the self-directed arm the coach e-mailed participants biweekly self-monitoring reminders but did not provide proactive, individualized coaching. In the coach-led arm, after 12 weeks of group sessions, the coach proactively reviewed the Heart360 self-tracking records of participants and provided individualized progress feedback, behavior change and maintenance coaching, and problem-solving assistance via secure e-mail every 2–4 weeks for 12 additional months. For more information on the interventions, see the protocol in Ma et al.¹⁴

Measurements

To assess participant acceptance of the Heart360 tracking tool, the E-LITE Use of Technology Questionnaire adapted relevant measures from the information systems field. Specifically, it adapted the measures of Venkatesh et al.¹¹ for five of a total of eight technology acceptance constructs: social influence, compatibility, affect, concern, and behavioral intention. In addition, it used the measures of perceived usefulness and perceived ease of use of Davis¹² as well as the computer self-efficacy scale of Compeau et al.¹³ The questionnaire included 4–10 items for each construct, all on a 5-point Likert response scale (ranging from 1=strongly disagree to 5=strongly agree). Construct definitions and measures are found in Table 1. E-LITE participants in the two interventions self-administered the questionnaire at 3 months (end of intensive intervention phase) and 15 months (end of maintenance phase). The overall score of a construct was calculated as the unweighted mean of all the constituent scale items.

Table 1.

Evaluation of Lifestyle Interventions to Treat Elevated Cardiometabolic Risk in Primary Care Use of Technology Questionnaire Constructs and Questions

CONSTRUCT, QUESTIONS
Perceived usefulness (the degree to which individuals believe that using the tool will enable them to manage their health)
1. The tool enables/would enable me to manage my health better.
2. The tool enhances/would enhance my effectiveness in managing healthcare.
3. The tool is/would be useful for managing my healthcare.
4. The tool makes/would make it easier for me to manage my health.
5. The tool offers/would offer additional health information.
6. The advantages of using the tool (would) far outweigh the disadvantages.
Perceived ease-of-use (the degree to which an individual believes that the tool will be easy to use)
1. My interaction with the tool is/would be clear and understandable.
2. It is/would be easy for me to become skillful at using the tool.
3. I (would) find the tool easy to use.
4. Learning to operate the tool is/would be easy for me.
Affect (the degree to which an individual likes using the tool)
1. Using the tool is/would be a good idea.
2. The tool makes/would make my effort to improve my health more interesting.
3. I (would) get frustrated using the tool.
4. I (would) like using the tool.
Social influence (the degree to which an individual believes that people who are important to him or her think he or she should use the tool)
1. People who influence me thought/would think that I should use the tool.
2. People who are important to me thought/would think that I should use the tool.
3. People who are important to me encouraged/would encourage me to use the tool.
4. People who influence me thought/would think that using the tool is a good idea.
Compatibility (the degree to which an individual believes that the tool fits his or her needs and available resources)
1. I have the resources necessary to use the tool.
2. I have the knowledge necessary to use the tool.
3. The tool is compatible with the way I like to do things.
4. A specific person (or group) is available for assistance with difficulties using the tool.
5. Using the tool fits well with my lifestyle.
Self-efficacy (the degree to which an individual feels confident that he or she could use the tool in different, specific conditions)
I could use the tool to manage my health…
1. if there was no one around to tell me what to do as I go.
2. if I had never used a tool like it before.
3. if I had only the user guide for reference.
4. if I had seen someone else using it before trying it myself.
5. if I could call someone for help if I got stuck.
6. if someone else had helped me get started.
7. if I had a lot of time to complete the tasks for which the tool was provided.
8. if I had just the built-in help facility for assistance.
9. if someone showed me how to do it first.
10. if I had used a similar tool before this one to perform the same tasks.
Concern (the degree to which an individual has concerns about using the tool)
1. I feel apprehensive about using the tool.
2. It scares me to think that I could lose a lot of information using the tool by hitting the wrong key.
3. I hesitate to use the tool for fear of making mistakes I cannot correct.
4. The tool is somewhat intimidating to me.
5. I am concerned about my privacy when using the tool.
Intention to use (the degree to which the individual feels he or she will use the tool in the future)
1. I will use the tool on a regular basis.
2. I predict I will use the tool.
3. I intend to use the tool in the future.
4. I will strongly recommend others to use the tool.

All constructs were assessed on a 5-point Likert scale, with 1=strongly disagree, 2=disagree, 3=neither disagree or agree, 4=agree, and 5=strongly agree. The overall score of a construct was calculated as the unweighted mean of all the constituent scale items.

Participant usage behavior was measured by the number of Heart360 self-tracking entries. Trained outcome assessors blinded to treatment assignment measured participants' weights in duplicate at baseline, 3 months, and 15 months with participants wearing light street clothes but no shoes.

Statistical Analysis

We analyzed the internal consistency, responsiveness to change, and association with Web site usage of the eight technology acceptance constructs and the association of Web site usage with weight loss by intervention arm. Internal consistency was assessed cross-sectionally at 3 and 15 months with Cronbach's alpha, which measures the extent to which each item of a questionnaire or scale is measuring the same construct.¹⁸ Cronbach's alpha coefficients of 0.70 or greater indicate good internal consistency.¹⁹ Responsiveness was assessed with paired Student's t tests, to determine the extent of change in construct mean scores from 3 to 15 months. Pearson's correlation coefficients measured the strength of relationship between construct mean scores and cumulative Heart360 usage at 3 months (contemporaneous correlation) and between changes in construct measure scores and cumulative Heart360 usage from 3 to 15 months (longitudinal correlation). Ordinary least-squares regression modeled the relationship between Heart360 usage and weight loss during the intensive intervention (i.e., baseline to 3 months) and maintenance (i.e., from 3 to 15 months) phases.

All analyses were conducted in SAS version 9.2 software (SAS Institute Inc., Cary, NC). Values of p are two-tailed with statistical significance defined as p<0.05.

Results

Participant Characteristics

Sixty-four of the 81 self-directed intervention participants and 69 of the 79 coach-led intervention participants completed the Use of Technology Questionnaire at 3 and/or 15 months. Similar to the entire E-LITE sample, questionnaire respondents had a mean±standard deviation age of 53.5±10.5 years and a mean±standard deviation body mass index of 31.7±5.0 kg/m² (weight, 92.7±17.4 kg), with 47% female and 77% non-Hispanic white. Fifty-two percent of respondents had prediabetes, 90% had metabolic syndrome, and 41% had both. Respondents and nonrespondents did not differ on any of these key characteristics. Respondents in the two interventions also had similar baseline characteristics except for age (Table 2).

Table 2.

Baseline Characteristics of the Use of Technology Questionnaire Respondents

CHARACTERISTIC	SELF-DIRECTED INTERVENTION (N=64)	COACH-LED INTERVENTION (N=69)	P VALUE
Age (years)	51.2±9.3	55.7±11.1	0.01
Female (%)	42.2	50.7	0.32
Race/ethnicity (%)			0.90
Non-Hispanic white	78.1	76.8
Asian/Pacific Islander	17.2	15.9
Latino/Hispanic	3.1	5.8
Income (%)			0.27
<$75,000	8.1	14.7
$75,000–$124,999	21.0	30.9
$125,000–$149,999	21.0	14.7
$150,000+	50.0	39.7
College level or above (%)	100	97.1	0.17
Weight (kg)	92.4±18.3	92.9±16.6	0.88
Body mass index (kg/m2)	31.6±4.8	31.8±5.2	0.79
Men	31.1±5.0	30.2±3.4	0.38
Women	32.2±4.5	33.3±6.1	0.41
Prediabetes (%)	50.0	53.6	0.68
Metabolic syndrome (%)	89.1	89.9	0.88
Prediabetes and metabolic syndrome (%)	39.1	43.5	0.61

Data are mean±standard deviation values unless indicated otherwise.

Internal Consistency and Responsiveness of the Technology Acceptance Scales

The measures of all constructs demonstrated high internal consistency with Cronbach's alpha coefficients ranging from 0.74 to 0.97, and the results were consistent between trial arms (coach-led and self-directed interventions) and time points (3 and 15 months) (Table 3).

Table 3.

Internal Consistency: Cronbach's Alpha Coefficients

	SELF-DIRECTED INTERVENTION		COACH-LED INTERVENTION
CONSTRUCT	3 MONTHS (N=57)	15 MONTHS (N=56)	3 MONTHS (N=56)	15 MONTHS (N=60)
Perceived usefulness	0.95	0.97	0.96	0.96
Perceived ease of use	0.94	0.91	0.94	0.96
Affect	0.86	0.87	0.82	0.84
Social influence	0.94	0.99	0.96	0.97
Compatibility	0.78	0.74	0.79	0.80
Self-efficacy	0.90	0.94	0.93	0.91
Concern	0.86	0.90	0.91	0.90
Intention to use	0.94	0.95	0.95	0.97

As shown in Table 4, mean scores decreased significantly from 3 to 15 months among respondents in the self-directed intervention for perceived usefulness, perceived ease of use, compatibility, and intention to use, whereas the mean score regarding concern increased significantly. Among respondents in the coach-led intervention, only changes in mean scores for compatibility and intention to use reached statistical significance—both declined from 3 to 15 months.

Table 4.

Responsiveness: Change in Construct Mean Scores

	SELF-DIRECTED INTERVENTION			COACH-LED INTERVENTION
COMPONENT	3 MONTHS (N=57)	15 MONTHS (N=56)	DIFFERENCE (N=46)	3 MONTHS (N=56)	15 MONTHS (N=60)	DIFFERENCE (N=46)
Perceived usefulness	3.55±1.01	3.20±1.06	−0.47±0.88a	3.65±0.97	3.68±0.95	−0.07±0.83
Perceived ease of use	3.96±0.92	3.56±1.01	−0.40±1.00a	3.98±0.87	3.89±0.92	−0.19±0.80
Affect	3.58±1.00	3.09±1.05	−0.52±0.91	3.52±0.92	3.43±0.95	−0.19±0.86
Social influence	3.25±0.73	3.10±0.82	−0.24±0.91	3.28±0.71	3.34±0.70	0.01±0.73
Compatibility	3.83±0.71	3.42±0.74	−0.39±0.72a	3.80±0.75	3.66±0.78	−0.22±0.68a
Self-efficacy	3.58±0.83	3.42±0.89	−0.17±1.05	3.56±0.78	3.56±0.75	−0.02±0.82
Concern	1.78±0.75	2.19±0.92	0.37±0.79a	1.93±0.90	1.99±0.84	0.07±0.65
Intention to use	3.35±1.16	2.53±1.06	−0.84±1.00a	3.43±1.07	2.96±1.16	−0.66±1.01a

Data are mean±standard deviation values.

^ap<0.05 by paired Student's t tests.

Association of Technology Acceptance and Use

The median (interquartile range) number of Heart360 self-tracking entries was 93 (40–181) among respondents in the self-directed intervention and 125 (68–171) in the coach-led intervention during the first 3 months, compared with 50 (0–188) and 46 (0–261), respectively, from 3 to 15 months. In the self-directed intervention, correlations between the cumulative number of Heart360 self-tracking entries and cross-sectional mean construct scores at 3 months reached statistical significance only for perceived usefulness (Pearson r=0.43) and affect (r=0.46), whereas none was significant between cumulative Heart360 entries and changes in mean construct scores from 3 to 15 months (Table 5). In the coach-led intervention, respondents' perceived usefulness and intention to use showed significant contemporaneous and longitudinal correlations with their actual usage behavior (r ranging from 0.33 to 0.37). In addition, the contemporaneous correlations for perceived ease of use, affect, and compatibility were statistically significant among these respondents.

Table 5.

Contemporaneous and Longitudinal Validity: Pearson Correlation Coefficients

	SELF-DIRECTED INTERVENTION		COACH-LED INTERVENTION
CONSTRUCT	CONTEMPORANEOUS	LONGITUDINAL	CONTEMPORANEOUS	LONGITUDINAL
Perceived usefulness	0.43a	0.12	0.33a	0.37a
Perceived ease of use	0.36	0.02	0.50a	0.33
Affect	0.46a	−0.25	0.39a	0.26
Social influence	0.21	0.36	0.18	−0.03
Compatibility	0.36	−0.12	0.36a	0.22
Self-efficacy	0.24	−0.06	0.03	0.20
Concern	−0.22	0.32	−0.16	−0.27
Intention to use	0.27	−0.01	0.33a	0.37a

^ap<0.05.

Association of Technology Use and Outcome

Mean (95% confidence interval) weight change from baseline to 3 months was −4.7 (−5.5, −3.9) kg in the self-directed intervention and −5.7 (−6.4, −4.9) kg in the coach-led intervention, and this change was maintained through 15 months. Figure 1 shows a significant linear relationship between the number of Heart360 self-tracking entries and the amount of weight loss from baseline to 3 months in self-directed (β=−0.02, p=0.001) and coached-led (β=−0.02, p<0.001) interventions. The relationship between Heart360 usage and weight loss from 3 to 15 months continued the same trend but was not statistically significant.

Fig. 1.

Correlation between Heart360 usage and weight change: (A) 0–3 months and (B) 3–15 months.

Discussion

e-Health weight management interventions have largely focused on repackaging effective, empirically validated face-to-face treatment strategies promoting healthy eating and physical activity for technology-assisted delivery, with little attention to understanding people's attitudes and behavior pertaining to the technology used in this context or their relationship to weight outcome. The need to address this gap is germane to promoting high-quality research in this rapidly growing area because the level of adherence (e.g., the frequency and duration of technology use) has been low and the evidence of effectiveness has been mixed in studies of e-health weight management interventions and related lifestyle change programs.^6–9 By drawing on prior technology acceptance and use research in the information systems discipline, this study examined participant acceptance and use of a weight and physical activity self-monitoring Web site that was an integral part of two effective lifestyle interventions for weight loss.¹⁵ Self-monitoring is a cornerstone of standard behavioral weight loss interventions,²⁰ and, not surprisingly, its automation has been a central feature of technology-assisted delivery of such interventions. Hence, the current findings make a useful contribution to the e-health weight management intervention literature.

We demonstrated that the adapted technology acceptance scales had consistently high internal consistency between interventions and time points for all the constructs. The finding that mean scores changed unfavorably from 3 to 15 months for five of the eight constructs in the self-directed intervention and for only two constructs in the coach-led intervention indicates scale responsiveness to differences in change of acceptance of the self-monitoring Web technology used over time between interventions. Although participants in both interventions were strongly encouraged to use Heart360 for self-monitoring, those in the coach-led arm received ongoing, personalized written behavioral feedback during the 12 weeks of group visits and subsequently via secure e-mail based on the coach's review of their self-monitoring records, but those in the self-directed arm did not. The differences in the modality and intensity of coach support for online self-monitoring might explain why self-directed intervention participants reported significantly less positive perceptions of usefulness and ease of use and significantly greater concern over time.

The behavior change model for Internet health interventions of Ritterband et al.²¹ posits that user attitudes and beliefs are among the factors that may explain and predict Web site use—a potential mediator of the intervention effect and hence target for change along with the identified health problem. We showed significant cross-sectional (at 3 months) and longitudinal (from 3 to 15 months) correlations of several acceptance constructs with actual use. The constructs we measured were based on the Unified Theory of Acceptance and Use of Technology (UTAUT) of Venkatesh et al.,¹¹ which had been validated in empirical evaluations involving different user groups and technologies in worksite settings. The UTAUT has five core acceptance constructs that determine use: perceived usefulness, perceived ease of use, compatibility, social influence, and behavioral intention. Venkatesh et al.¹¹ concluded that self-efficacy, affect, and concern were nonsignificant determinants of technology use whose effects were fully mediated by perceived usefulness and ease of use. In this study, we found that, with the exception of social influence, the UTAUT core constructs were significantly associated with cumulative Heart360 usage at 3 months in the coach-led arm and that the association persisted through 15 months for perceived usefulness and behavioral intention. The generally weaker correlations between the constructs and measured usage in the self-directed arm lend support to the scales' ability to detect differences between interventions.

The conceptual model of Ritterband et al.²¹ suggests that Web site use should lead to symptom improvement in Internet health interventions. This is supported by data from this and other studies¹⁰ showing that higher usage of the technology in e-health weight management interventions is associated with greater weight loss. At the same time, we observed that Heart360 self-monitoring entries declined notably from the 3-month intensive intervention phase to the 12-month maintenance phase. Decreasing technology use is commonly reported in e-health intervention studies.^6–9 Research is needed to identify strategies that will promote persistent technology use and hence possibly enhance the effectiveness of e-health interventions.

Several study limitations are noteworthy. First, this was a secondary analysis of a subset of participants in the intervention arms of a randomized controlled trial. As a result, selection bias is likely, even though questionnaire respondents and nonrespondents did not differ on any key characteristics. Second, the age difference among respondents in the two interventions could confound the current findings related to between-group differences. Third, the observed associations between the technology acceptance constructs and actual use and between technology use and weight loss do not prove causation. In fact, the relationships among these variables should not be presumed to be linear or unidirectional. Instead, they may be recurrent and cyclic and affected by other variables not measured in the current study. For example, Ritterband et al.²¹ conceptualized that Web site use in Internet interventions could be influenced by the combination of user characteristics, environmental factors, Web site characteristics, and user support. Future investigation of the relationships among these variables as well as between them and technology use and outcome in e-health interventions is needed. Finally, further validation of the current construct measures is warranted in e-health interventions involving other technologies, patient populations, and treatment targets.

Conclusions

This study finds that previously validated measures of technology acceptance in the information systems discipline can be adapted to evaluate user beliefs and attitudes regarding an automated self-monitoring technology in the context of e-health weight management interventions. In particular, user perceptions of usefulness, ease of use, and compatibility with existing values, needs, and resources, as well as intention to use the technology, show meaningful correlations with actual usage, whereas usage is correlated with weight loss. Future work to elucidate the role of these and other factors in determining technology use within e-health weight management interventions may help to improve adherence and effectiveness and to more firmly establish this form of treatment.

Acknowledgments

This study was supported by grant R34DK080878 from the National Institute of Diabetes and Digestive and Kidney Diseases, a Scientist Development Grant award (0830362N) from the American Heart Association, and internal funding from the Palo Alto Medical Foundation Research Institute. We also would like to acknowledge the Diabetes Prevention Support Center of the University of Pittsburgh for training and support in the Group Lifestyle Balance program; the current program is a derivative of this material. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Diabetes and Digestive and Kidney Diseases or the American Heart Association. No sponsor or funding source had a role in the design or conduct of the study, the collection, management, analysis, or interpretation of the data, or the preparation, review, or approval of the manuscript.

Disclosure Statement

No competing financial interests exist.