User Attitudes towards Physical Activity Websites in a Randomized Controlled Trial

Abstract

Objective

To better understand the mixed findings regarding the efficacy of Internet-based physical activity interventions, we examined the use and usefulness of particular website components that may lead to improvements in intervention efficacy.

Method

Participants were sedentary individuals from a 12-month randomized controlled physical activity trial conducted in Providence, Rhode Island and Pittsburgh, Pennsylvania from 2003–2006. The present study included participants from the Tailored Internet arm (n = 81; instantaneous web-based tailored feedback to participants) or the Standard Internet arm (n = 82; websites currently available to the public). We obtained objective data via the intervention websites and subjective usefulness data via questionnaires.

Results

The Tailored Internet arm logged onto their website significantly more times than the Standard Internet arm (median 50 vs. 38; p < .05). Among participants in the Tailored Internet arm, the self-monitoring feature (i.e., logging) followed by goal setting were rated as the most useful website components.

Conclusion

Logins in the current study were substantially higher compared to previous studies. Participants endorsed goal setting and self-monitoring as being most useful, which are critical components for health behavior change. Future studies should continue to examine these features and improve the perceived usefulness of other theory-based strategies.

Introduction

Despite the many health benefits associated with physical activity, approximately 55% of Americans do not engage in physical activity at the level recommended levels (Macera et al., 2005; United States. Public Health Service. Office of the Surgeon General., 1996). Given the cost and time associated with face-to-face and telephone interventions, it is not feasible to reach this large sedentary population using these channels. However, theory-based physical activity interventions delivered through the Internet may be a cost-effective channel for reaching this large population of sedentary adults. Research on the efficacy of Internet-based interventions for physical activity is mixed. Specifically, four studies have indicated positive results (Hurling et al., 2007; Napolitano et al., 2003; Rovniak, Hovell, Wojcik, Winett, & Martinez-Donate, 2005; Spittaels, De Bourdeaudhuij, & Vandelanotte, 2007) while other studies have found no differences between groups (Hageman, Walker, & Pullen, 2005; Marcus, Lewis, Williams, Dunsiger et al., 2007; Marshall, Leslie, Bauman, Marcus, & Owen, 2003; Spittaels, De Bourdeaudhuij, Brug, & Vandelanotte, 2007; Steele, Mummery, & Dwyer, 2007).

Sciamanna and colleagues (2002) examined user attitudes towards the intervention website used in the Napolitano and colleagues (2003) study listed above. In this study, 65 sedentary and women were randomly assigned to a three-month motivational stage-targeted, Internet plus email physical activity intervention or to a wait-list control group. The Internet group reported significantly more minutes of physical activity per week than the control group at the one-month follow-up; however, differences were no longer significant at the three-month follow-up. In this study, 79.2% of participants viewed the website as “somewhat helpful” and 12.5% as “very helpful.” Additionally, 83.3% of the participants used the website more than once during the three-month intervention. In another study, Leslie and colleagues (2005) examined user attitudes in a study examining the efficacy of an eight-week stage-targeted intervention delivered via the Internet or print (mailed) among 655 adults (Marshall et al., 2003). Results indicated a non-significant trend toward increased physical activity in both groups with no significant differences between conditions. A follow-up study reported that only 46% of the participants visited the website at least once (Leslie, Marshall, Owen, & Bauman, 2005). Furthermore, among the participants recalling the website, only 50% thought the website was moderately to extremely useful (Leslie et al., 2005). Additional research is needed to better understand user attitudes towards Internet-based physical activity interventions to help elucidate critical treatment components.

The purpose of this paper is to examine use patterns and user attitudes towards the Internet-based interventions in a recent study conducted by Marcus and colleagues (2007). In this study, participants were randomly assigned to one of three 12-month interventions: (1) Motivationally-Tailored Internet (i.e., Tailored Internet); (2) Motivationally-Tailored Print (i.e., Tailored Print); or (3) six researcher-selected websites currently available to the public (i.e., Standard Internet) (Marcus, Lewis, Williams, Dunsiger et al., 2007; Marcus, Lewis, Williams, Whiteley et al., 2007). Participants in all three groups increased their physical activity; however, there were no significant differences between groups (Marcus, Lewis, Williams, Dunsiger et al., 2007). Specifically, the Tailored Internet arm increased from a median of 0 minutes of physical activity per week at baseline to a median of 120.0 (mean=161.4, SD=217.5) at 6 months and 90.0 (125.1, SD = 159.0) at 12 months. The Tailored Print arm increased from a median of 0 at baseline to 112.5 (mean=123.1, SD=107.8) at 6 months and 90.0 (mean=123.2, SD 124.9) at 12 months. The Standard Internet arm increased from a median of 10.0 at baseline to 90.0 (mean=122.3, SD=126.9) at 6 months and 80.0 (mean=114.1, SD=132.5) at 12 months.

For this paper, the following four objectives were addressed: 1) To examine differences in use patterns and perceived website usefulness between the Tailored Internet and Standard Internet arms; (2) To examine if use patterns and perceived usefulness of the intervention websites predicted physical activity behavior change; 3) In the Tailored Internet arm, to determine which website components were used the most and which were rated as most useful; and 4) In the Standard Internet arm, to examine which of the six standard website were rated as most useful. Related to the first two objectives, we predicted that participants in the Tailored Internet arm would log onto the study website significantly more times than the Standard Internet arm and that higher rates of logging onto the study website would be significantly related to higher levels of physical activity.

Methods

Participants

Participants were healthy, sedentary (90 minutes or less of physical activity per week) adults, ages 18 and older (n = 249) who participated in a randomized controlled trial examining the efficacy of Internet and print-based physical activity interventions (see Figure 1). The sample size was based on the assumption of a 30-minute difference at 12 months between the Tailored Print and Tailored Internet arms. The sample was mostly women (82.7%) and Caucasian (76.3%). Participants were recruited primarily through newspaper advertisements from the Providence, Rhode Island area (75% of sample) and Pittsburgh, Pennsylvania. The study was conducted from January 2003 through May 2006. Participants read and signed a consent form approved by both sites’ institutional review boards. Additional details regarding the trial are available elsewhere (Marcus, Lewis, Williams, Dunsiger et al., 2007; Marcus, Lewis, Williams, Whiteley et al., 2007). Because the purpose of this paper was to examine user attitudes towards the physical activity websites, we did not include data in this paper from the 86 participants in the Tailored Print arm. Therefore, the total sample for the present analyses includes 163 participants.

Figure 1.

Participant Flow Chart

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

Measures

We assessed minutes of physical activity per week at baseline, and 6 and 12 months using the interviewer-administered 7-Day Physical Activity Recall (Blair et al., 1985; Sallis et al., 1985). Studies indicate that this interview is a reliable and valid assessment tool (Pereira et al., 1997). Objective data were downloaded from the intervention websites (e.g., frequency of logins, duration of website use).

Participants completed the 5-item Website Quality Questionnaire examining the quality of the Tailored Internet and Standard Internet websites. The items assessed how supportive the website was (5-point likert scale), how motivating it was (5-point likert scale), if some knowledge was gained (yes/no), how helpful the website was for increasing physical activity (3-point likert scale), and how enjoyable the website was (3-point likert scale α=0.845). The alpha coefficient for this questionnaire was .85 indicating adequate reliability.

Participants in the Tailored Internet arm completed the Tailored Internet Usefulness Questionnaire, which was an eight-item questionnaire examining frequency of use (based on a 5-point likert scale ranging from “never” to “very often”) for each of the eight website components which included: (1) “Tailored Report” based on responses to particular questionnaires; (2) “Stage of Change,” which included motivational messages based on stage of change; (3) “Physical Activity Logs” in which participants entered the type and duration of activity on an Internet-based log that looked like a calendar, (4) “Goal setting”; (5) “Resources” (e.g., local biking trails, simple stretches, how to take your heart rate); (6) “FAQ”-Frequently Asked Physical Activity Questions (e.g., “Can I break up physical activity throughout the day”); (7) “Tip of the day” (i.e., short tips on the homepage that changed each day); and (8) “Tip Sheets,” which were 13 emails sent during the intervention that included a link to a webpage covering a variety of topics (e.g., strategies to decrease boredom with exercise, fitting exercising in while traveling). The questionnaire included snapshots of each website feature. Participants also rated the usefulness of each component of the Tailored Internet intervention on a 5-point likert scale ranging from “not at all useful” to “extremely useful” for each of the eight components listed above. The alpha coefficient for the Tailored Internet Usefulness Questionnaire was .91 indicating good reliability.

Participants in the Standard Internet arm completed a similar questionnaire except the snapshots were of the six websites currently available to the public and our physical activity logging component. Participants also rated which of the six websites they used most frequently and which websites they found to be the most helpful.

Procedure

Participants in the overall trial were randomly assigned to the Tailored Internet, Tailored Print, or Standard Internet arm. The randomization was blocked on gender and stage of change and the sequence was randomly determined by a computer. Participants received a sealed envelope containing their group assignment at the time of randomization. Research assistants not involved in assessment generated the allocation sequence, participant enrollment, and assignment to groups. The research assistant conducting the assessments was blinded to treatment condition

Tailored Internet Intervention

Participants in the Tailored Internet intervention were regularly prompted to log onto the study website, which included monthly tailored feedback reports based on the transtheoretical model (Marcus & Lewis, 2003; Prochaska et al., 1993) and social cognitive theory (Bandura, 1977; Bandura, 1992), educational and motivational materials, self-monitoring, and a goal setting feature. The tailored reports included feedback that compared the individual’s assessment results with the results obtained from others in the same stage of motivational readiness (“normative comparisons”) and, in all reports after baseline, made comparisons with the individual participant’s previous assessment (“ipsative comparisons”). Tailored feedback targeted a number of cognitive and behavioral variables from Social Cognitive Theory and the Transtheoretical Model that are posited to influence behavior change. These constructs included stage of motivational readiness, decision making, self-efficacy, and a series of cognitive and behavioral processes of change. The tailored website intervention also had a new “tip of the day” posted each day to increase the likelihood of return visits.

Standard Internet Intervention

Participants in the Standard Internet arm were regularly prompted to access the Standard Internet website, which included links to the following six websites available to the public at the time of the study: American Heart Association (AHA; Just Move); Shape Up America; Mayo Clinic Fitness and Sports Medicine Center; American Academy of Family Physicians (physical activity section); American Council on Exercise (ACE); and American College of Sports Medicine (ACSM) Health and Fitness Information. We also prompted participants via email on a monthly basis to self-monitor on our website (i.e., document the frequency and type of activity each day) and complete the questionnaires (only the Tailored Internet intervention received feedback based on the questionnaires). Participants in all arms of the study received $10 for completing physical activity logs and monthly questionnaires.

Statistical Analyses

To examine the effect of usefulness rating on physical activity at a later timepoint, we used data from the 5-item Website Quality Questionnaire at 6 months to predict physical activity at 12 months. The number of times participants logged onto the study website over 12 months was calculated and summarized in terms of medians (since the data were skewed). A Wilcoxon two-sample test was used to compare number of logins between the Tailored Internet and Standard Internet arms. In addition, the association between number of logins and change in physical activity from baseline to 12 months was measured using quintile regression, controlling for treatment assignment and baseline physical activity. A log transformed version of number of logins was used in the regression, in order to make the measure more symmetric. We compared the Tailored Internet and Standard Internet arms with respect to the mean score on the Website Quality Questionnaire, using a two-sample t-test. Quantile regression was used to measure the association between website usefulness and change in physical activity from baseline to 12 months, while controlling for treatment assignment.

Results

Objective 1: Differences in Use Patterns and Perceived Website Usefulness for the Tailored Internet and Standard Internet Conditions

The median number of logins to the study website, regardless of group, was 44 times over the 12-month study. Participants in the Tailored Internet arm logged onto the study website significantly more times than the Standard Internet group, z = 2.21, p < .05 (median of 50 times for Tailored Internet and 38 times for Standard Internet). The number of logins by month is presented for both groups in Table 1 and Figure 1. Individuals in the Tailored Internet arm reported their website to be more useful, as measured by the 5-item Website Quality Questionnaire, than participants given the Standard Internet websites, 16.76 vs. 11.64; t(148) = 6.72, p < .001.

Table 1.

Median Number of Logins by Group and Month

Month	Tailored Internet	Standard Internet
1	8	7
2	9	5
3	5	4
4	5	3
5	4	3
6	3	2
7	3	2
8	2	2
9	2	2
10	1	2
11	1	2
12	1	2

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

Objective 2: Relationships among Use Patterns, Perceived Website Usefulness, and Physical Activity Behavior

The higher the number of logins, regardless of group, the greater the increase in physical activity from baseline to 12 months t = 3.39, p < .01. Additionally, higher usefulness ratings on the Website Quality Questionnaire were associated with greater increases in physical activity from baseline to 12 months, while controlling for group assignment, t = 2.32, p < .01. As can be seen in Figure 2, participants who continued to meet the physical activity recommendation of engaging in physical activity 150 minutes or more per week during months 6–12 tended to have more logins than participants not meeting this recommendation.

Figure 2.

Median number of Logins per Month by Meeting Physical Activity Criteria (150 Minutes or More of Physical Activity per Week).

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

The subjective self-report measure of total time spent on the website over 12 months was significantly correlated with the objective data obtained from the intervention website for both the Tailored Internet arm, r² = .54, p < .01 and Standard Internet arm, r² = .76, p < .001. Additionally, 43.8% of the participants over-reported their use and 56.2% under-reported their use. Among those over-reporting, the median number of minutes by which they over-reported use was 225 minutes during the 12-month intervention and among those under-reporting, the median number of minutes by which they under-reported use was 169 minutes.

Objective 3: Use Patterns and Perceived Usefulness of Tailored Internet Intervention

The average amount of time spent on the Tailored Internet intervention website per session was 7.1 minutes (total average over the year was 356 minutes). The number of times participants in the Tailored Internet arm clicked on particular links on the intervention website and the average amount of time spent on each feature within a session is summarized in Table 2. As can be seen from the table, the personalized report and stage of change information were the most used components of the Tailored website. Additionally, participants set goals an average of 6.35 (sd = 4.34) times over the course of the study. The higher the number of times participants set goals, the greater the odds of meeting national physical activity guidelines (>=150 min/week of PA) at 12 months; β=.26, OR=1.29, CI=(1.14,1.47), p<.001. The mean usefulness ratings at 6 months (based on a 5-point likert scale) for each of the eight features of the Tailored website are presented in Table 3. As is evident from the table, the Physical Activity Logging feature was rated as most useful followed by Goal Setting, the Feedback Reports, and the Physical Activity Resources webpage.

Table 2.

Frequency and Duration of Use for Website Features in Tailored Internet Arm.

Feature	Average number of clicks for each participant over 12 months	Average time per click (in minutes)
Personalized Report	18	20.76
Stage of Change	11	14.66
Info & Resources	3	4.57
Frequently Asked Questions	2	1.94
Did You Know (Tip of Day)	2	3.66
Monthly Tip Sheet	6	8.01

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

Table 3.

Average Usefulness Scores for Features of Tailored Internet Arm at 6 Months

Feature	Rating (ranges from 1–5)
Tailored Report	2.89
Stage of Change	2.57
PA logs	3.70
Goal Setting	3.21
Resources	2.79
FAQ	2.68
Tip of the Day	2.71
Tip Sheets	2.58

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

Objective 4: Use Patterns and Perceived Usefulness of Standard Internet Intervention

The average amount of time spent on the Standard Internet intervention website per session was 6.8 minutes (total average over the year was 260 minutes). Participants’ use of the six Standard Internet websites is summarized in Table 4. The American Heart Association website, Just Move, was used most often, with 49% of the participants reporting that they used this website the most. The average usefulness rating of the six websites was 2.25 (based on a 5-point likert scale).

Table 4.

Percent Reporting Use of Particular Website the Most at 6 Months in Standard Internet Arm

Website	Percent
AHA	49.3
Shape Up	19.7
Mayo	18.3
AAFP	4.2
ACE	2.8
ACSM	1.4

This study was conducted in Providence, Rhode Island, USA and Pittsburgh, Pennsylvania, USA from 2003–2006.

Discussion

As hypothesized, participants in the Tailored Internet arm logged on significantly more times throughout the year than participants in the Standard Internet arm. This difference could be attributed to the fact that the Tailored Internet intervention was specifically designed to promote return visits to the website. For example, new tips of the day were displayed on the homepage daily throughout the year. Additionally, the Tailored Internet website contained interactive and instantaneous features that were individually tailored such as the personalized feedback report. However, it is important to note that there were no physical activity differences between the Tailored Internet and Standard Internet arms. Therefore, this questions the importance of the interactivity and instantaneous features that were present in the Tailored Internet but not the Standard Internet arms. However, the higher login rate among the Tailored Internet participants relative to the Standard Internet participants could have perhaps been due to the instantaneous features and the interactive features of the Tailored Internet website. Given higher login rates were associated with higher physical activity levels, increasing login rates is important. Additional research is needed to better understand the importance of instantaneous and interactive features of the website.

Participants in this trial logged onto the study website substantially more times than in previous physical activity Internet-based trials (Marshall et al., 2003; Napolitano et al., 2003). This is important given higher login rates were associated with higher physical activity. There are several potential reasons for our higher login rate. First, participants were paid $10 each month to complete the online questionnaires and activity logs. Second, the website had a different “tip of the day” each day participants logged onto the website. Third, participants in our trial received tailored feedback in one of the interventions.

As can be seen in Table 1, login rates were most frequent at the beginning of the intervention phase and decreased throughout the 12-month intervention. This was not surprising given additional assistance may have been needed during the adoption compared to the maintenance phase. However, it is also important to note that based on Figure 2, it appears that participants who adopted and maintained their physical activity throughout the study kept a higher login rate throughout the entire 12 months than participants who adopted physical activity during the first six months but did not maintain their physical activity through 12 months. Therefore, it appears that a decline in website use over time may be detrimental to physical activity. However, it is important to note that website use declined among all user groups regardless of their degree of change. The initial high use might have been due to the novelty of the website and initial commitment to the study. Later use may reflect how beneficial the various users found the website. Regardless, it will be important for future studies to examine strategies to increase login rates during the maintenance phase of the intervention. For example, perhaps new features could be introduced during the maintenance phase such as moderated chat rooms and blogs. Additionally, the frequency of email prompts decreased during the maintenance phase in our study and therefore, future studies should consider maintaining a high level of email prompting during the maintenance phase.

Higher use of the study websites and higher usefulness rating were both related to higher physical activity, which is consistent with studies on other behaviors (e.g., weight loss, smoking cessation) (Cobb, Graham, Bock, Papandonatos, & Abrams, 2005; Tate, Jackvony, & Wing, 2003). As can be seen in Table 3, the Personalized Feedback Report and strategies based on the stage of change were the most popular sections of the website. These two sections were tailored and targeted to the individual, the most interactive, and had the strongest theoretical basis. Additionally, when asked to rate the usefulness of the features of the program, participants rated the physical activity logs most useful, followed by goal setting, and the tailored feedback report. This finding has important implications for future website development in that it may be important to include tailored, interactive, and theoretically-based components.

Although participants were not required to set goals nor did they receive incentives for goal setting, participants in the Tailored Internet arm set goals an average of 6.2 times during the course of the intervention. Specifically, participants set goals, recorded their activity, and then received a graph summarizing how their actual physical activity compared with their goal. These self-regulatory skills, in which participants received feedback and independently set goals based on the feedback, are considered to be critical for behavior change in social cognitive theory (Albert Bandura, 1986). Taken together, we recommend that tailored feedback, physical activity logging, physical activity feedback, and goal setting features be added to physical activity promotion websites.

We examined which of the study websites in the Standard Internet arm were rated as most useful. The Just Move program by the American Heart Association (AHA) was used nearly 2.5 times more than the next most popular site, Shape Up America. The Just Move program also received the highest ratings for perceived usefulness. In a review by Doshi and colleagues (Doshi, Patrick, Sallis, & Calfas, 2003), the Just Move website was rated by researchers and received the highest ratings for use of theory based strategies. It is important to note that the Just Move website was the first of the six websites listed on the Standard Internet homepage, which may have influenced its high use.

Although this study provides some data to help evaluate key features and user ratings of physical activity websites, there are a few limitations. First, it is possible that this sample was already primed and interested in using the Internet to receive physical activity information, advice, and support, and may have been biased towards this delivery channel. In fact, 61% reported at baseline that they would prefer to receive the Internet intervention rather than the print intervention. Second, although our study advertisements targeted both men and women, our final sample consisted of predominately women (82.7%). This is consistent with previous physical activity intervention studies (Marcus, Napolitano et al., 2007; Napolitano et al., 2003). Another limitation is that our study sample was predominantly Caucasian (76.3%), further limiting the generalizability.

Conclusions

In summary, our study indicated that participants found the various components of our tailored website useful. Additionally, participants logged onto our study website more times than in previous studies, perhaps due to the interactivity and change of content over time. Although participants found our study website to be more useful than websites currently available to the public, given the efficacy of our Standard Internet arm, we recommend that practitioners suggest that their patients access publically available websites as a tool to increase physical activity. Additionally, researchers should continue to develop and evaluate research-based interventions for wide-scale dissemination.