WALK 2.0: Examining the effectiveness of Web 2.0 features to increase physical activity in a ‘real world’ setting: an ecological trial

Abstract

Introduction

Low levels of health-enhancing physical activity require novel approaches that have the potential to reach broad populations. Web-based interventions are a popular approach for behaviour change given their wide reach and accessibility. However, challenges with participant engagement and retention reduce the long-term maintenance of behaviour change. Web 2.0 features present a new and innovative online environment supporting greater interactivity, with the potential to increase engagement and retention. In order to understand the applicability of these innovative interventions for the broader population, ‘real-world’ interventions implemented under ‘everyday conditions’ are required. The aim of this study is to investigate the difference in physical activity behaviour between individuals using a traditional Web 1.0 website with those using a novel Web 2.0 website.

Methods and analysis

In this study we will aim to recruit 2894 participants. Participants will be recruited from individuals who register with a pre-existing health promotion website that currently provides Web 1.0 features (http://www.10000steps.org.au). Eligible participants who provide informed consent will be randomly assigned to one of the two trial conditions: the pre-existing 10 000 Steps website (with Web 1.0 features) or the newly developed WALK 2.0 website (with Web 2.0 features). Primary and secondary outcome measures will be assessed by self-report at baseline, 3 months and 12 months, and include: physical activity behaviour, height and weight, Internet self-efficacy, website usability, website usage and quality of life.

Ethics and dissemination

This study has received ethics approval from the University of Western Sydney Human Research Ethics Committee (Reference Number H8767) and has been funded by the National Health and Medical Research Council (Reference Number 589903). Study findings will be disseminated widely through peer-reviewed publications, academic conferences and local community-based presentations.

Trial registration number

Australian New Zealand Clinical Trials Registry Number: ACTRN12611000253909, WHO Universal Trial Number: U111-1119-1755

Strengths and limitations of this study.

• This study is both novel and necessary as it examines the dissemination of a ‘real-world’ trial, in a natural setting, consisting of ‘everyday’ challenges that are not necessarily apparent in a randomised controlled trail.

• This intervention has a wide reach with the potential for it to be extended to broad, diverse populations throughout Australia and internationally.

• This study will identify the successful components and challenges associated with Web-based health promotion interventions, particularly concerning engagement and retention.

• Given the real world dissemination and wide reach of this intervention, we are limited in collecting objectively measured physical activity data, thus relying on self-reported measures alone.

Introduction

Regular physical activity (PA) plays an integral role in health promotion and disease prevention.¹ In addition to decreasing the risk of premature mortality, PA also reduces the risk of developing chronic disease, including cardiovascular disease, type 2 diabetes, some cancers, obesity, osteoporosis/osteoarthritis and poor mental health.^2–4 Despite the numerous physical and mental health benefits associated with PA, nearly 45% of adults (15 years or older) from Western countries do not engage in enough PA to confer health benefits.⁵ In Australia alone, 60% of adults (15 years or older) are not meeting minimum PA recommendations of 150 min of moderate-vigorous PA per week.⁶ It is estimated that these low levels of PA contribute to more than 8000 deaths in Australia each year, and that for every 1% of the Australian population that becomes sufficiently physically active some $7.2 million in healthcare costs could be saved annually.⁷ Motivating the population to attain higher levels of PA has been difficult, which emphasises the need for novel approaches that have the potential to reach broad populations.

One particular approach that has become increasingly popular within PA research is the use of the Internet to deliver health promotion and PA behaviour-change interventions. Given the unprecedented growth in Internet usage worldwide⁸ and across many populations, including women, elderly and those of low socioeconomic status,^9–11 it is not surprising that the Internet provides an attractive medium that has the potential to reach large diverse populations, providing greater accessibility to health promotion and behaviour change approaches. A number of literature reviews have demonstrated the effectiveness of Internet-based PA interventions for behaviour change, highlighting positive short-term behaviour change outcomes.^12–14 Few studies, however, have reported long-term behaviour change, or have examined the effectiveness of specific website components that may impact short-term and/or long-term behaviour change. The lack of maintenance of behaviour change has often been attributed to low levels of participant engagement and retention, and requires innovative approaches to address this challenge.^{13 15 16}

Web 2.0 features may be a promising approach to increasing user interaction and retention. Web 2.0 refers to a second generation of Internet-based features that are recognised to be highly interactive, and that have evolved from the traditional ‘read only’ website features to more participatory features, providing users with personal influence of how information is generated, created and shared collaboratively.^{17 18} Such features might include a wide range of interactive features such as blogs, wikis, podcasts, mash-ups and social networking, all of which have the potential to revolutionise website usage if utilised effectively.¹⁹ Further, there is growing acknowledgement that Web 2.0 features show promise in the field of health promotion and healthcare.^19–22 In order to fully understand the true effectiveness of Web 2.0 technologies in ‘real-world’ health promotion and healthcare practice, there is a need for large scale population-based studies that are evaluated in real-world settings.

In the past, PA intervention and behaviour change research has focused on the evaluation and outcomes of efficacy, in which efficacy of intervention is defined as its effect under ‘ideal conditions’.²³ These ‘ideal conditions’ commonly consist of tightly controlled, short-term randomised control trials (RCT) which are utilised in an effort to increase the internal validity of the intervention design.²⁴ Although RCTs are often considered the gold standard of trial design due to their ability to minimise the impact of selection and information biases, control for confounding variables and potentially rule out chance, they have often been challenged on the grounds of external validity.^23–26 Moreover, these types of interventions can be nearly impossible to adopt or implement in real-world settings, where participants are more likely to have a variety of health issues, numerous unforeseen external influences, and are often less motivated to engage in PA.^{24 27 28} As a result, there is a push to go beyond the use of RCTs alone, specifically in terms of public health and health promotion practice.²⁴ RCT's are an essential component of the research process, as a methodologically sound design will maximise internal validity, as well as provide a measure of efficacy. However, in order to understand effectiveness, generalisability and the true impact of large-scale public health and health promotion interventions on the general population, complementary approaches that go beyond RCTs and that are evaluated under normal, everyday conditions in a real-world setting (or as close to this as possible) are needed.^{24 27–29} This is particularly relevant to Web 2.0 features, as their spontaneous, viral and often uncontrollable nature makes it difficult to study their true dynamics in controlled RCT circumstances.³⁰ For example, if one cannot invite friends to join a social network, due to RCT-related restrictions, the social network is unlikely to be as functional and effective as it would be in real-world circumstances where such artificial barriers are not present. Therefore, given the proliferation of Web 2.0 features on the Internet, understanding the translation of Web-based PA intervention research into the real-world is becoming increasingly important.^{31 32}

The over-arching aim of the present study is to investigate the difference in PA behaviour between individuals using a traditional Web 1.0 website with those using a novel Web 2.0 website. Second, this study will investigate the effectiveness of Web 2.0 features (eg, involving social networking) to engage and retain individuals to a PA promotion website, as well as examine differences in quality of life between the intervention groups. A two-group randomised trial will be applied to compare a website using the novel Web 2.0 features with a traditional Web 1.0 intervention, and investigate the effectiveness of both interventions in a ‘real-world’ setting. The efficacy and internal validity of the interventions described in this study is also being investigated in a separate RCT, which has been described elsewhere.³³

Primary hypothesis

H1: Participants in the Web 2.0 condition will display significantly higher levels of PA at 3 months and 12 months, when compared with the Web 1.0 condition.

Secondary hypotheses

H2: There will be significantly higher retention and engagement in the Web 2.0 condition at 3 months and 12 months, when compared with the Web 1.0 condition, measured using website statistics and a usability survey.

H3: Participants in the Web 2.0 condition will display significantly higher improvements in quality of life at 3 months and 12 months, when compared with the Web 1.0 condition.

Methods

Trial design

This is the second phase of the larger WALK 2.0 project and aims to build on the earlier RCT³³ by investigating the comparative effectiveness of two web-based PA interventions in a real-world setting (figure 1). Participants will be randomly assigned to one of the two interventions, a Web 1.0 condition and a Web 2.0 condition. Outcomes will be assessed at baseline, and at 3 and 12 months. The study will be reported according to CONSORT guidelines.³⁴

Figure 1.

Flow chart of study protocol.

Participants, recruitment and eligibility

The study aims to recruit 2894 participants (approximately 1447 participants per condition) from self-selected new registrants to the existing and freely available 10 000 Steps website. This website is a key component of the 10 000 Steps Australia project, which is a community-level PA promotion project originally started in 2001.^{35 36} The 10 000 Steps website is used to promote PA through the use of Web 1.0 features, with several interactive features, such as ‘i-challenges’, ‘workplace challenges’, ‘virtual journeys’ and ‘step logs’. Over the past 13 years, the 10 000 Steps website has attracted over 260 000 registered members, an average of 2184 new registrants per month. Further details regarding the 10 000 Steps website are described elsewhere.^{33 37} All adults (18 years of age and older) who seek to register on http://www.10000Steps.org.au of their own impetus will be offered the opportunity to participate in the current study. Participants will be excluded from the current study if they: (1) are under 18; (2) are seeking to participate in a 10 000 Steps workplace challenge³⁸; (3) are, or have been, a participant of the WALK 2.0 RCT³³; and (4) have an existing medical condition which could be exacerbated through increasing PA assessed using the Physical Activity Readiness Questionnaire, (PAR-Q).³⁹

Study procedure

All new users who register with the 10 000 Steps website will be invited to participate in this trial. They will be provided with a brief overview of the project and interested users will then be screened for eligibility using a self-administered online survey, while non-interested users will be returned to 10 000 Steps, as they had initially intended. All interested individuals who meet inclusion criteria will then be invited to provide informed consent. On providing informed consent, all eligible participants will automatically be uniformly randomly assigned to one of the two trial conditions using a computer-generated algorithm. Participants will gain access to the respective intervention websites and will be prompted to complete a brief online baseline survey. All these steps are fully automated, and there will be no interaction with the research team at any point. Participants randomised to the Web 1.0 condition will gain access to the original 10 000 Steps website (http://www.10000steps.org.au), which features Web 1.0 technologies. Participants randomised to the Web 2.0 condition will be assigned to a newly developed website featuring Web 2.0 technologies (http://www.walk.org.au).

Primary and secondary outcome measurements will be assessed by online self-report questionnaires at baseline, 3 and 12 months. These measures will include PA behaviour, height and weight (to calculate the body mass index), Internet self-efficacy, website usability (only assessed at 3-month and 12-month follow-up periods), and quality of life. Website usage statistics will be collected throughout for the entire duration of the study. Demographic data will also be collected at baseline. Participants will receive email invitations to complete the online baseline and follow-up outcome measurements. Participants who have not completed the survey after a 3 week period will be sent a total of three reminder emails (sent 1 week apart) encouraging them to complete the online assessments.

Interventions

Web 1.0 condition

Participants allocated in the Web 1.0 condition will be given access to the existing 10 000 Steps website, using Web 1.0 features. This website was originally developed to promote the community-based 10 000 Steps Australia project³⁶ and includes features that support individual self-monitoring (eg, step log), communication exchange (eg, discussion forums) and access to information and educational resources (eg, benefits of PA). Participants allocated to this condition will be able to log their daily activities, in terms of type and duration of activity and/or number of steps as well as access the website library for information concerning PA and other health behaviours. Participants will also have the opportunity to share stories, ask questions or make comments in the discussion forum. This is a public forum in which all information posted can be viewed by all participants on the website. The Web 1.0 condition does not have the functionality of individualised personal pages or forums.

Web 2.0 condition

Participants allocated to the Web 2.0 condition will be given access to a newly developed website that will provide content and basic functionality similar to the Web 1.0 condition (step log and library), but will be supplemented with Web 2.0 features. These features include annotation, messaging and group publishing tools implemented in a configurable social networking setting where individual user environments may be different. Participants in the Web 2.0 condition will still have access to self-monitoring features and information and educational resources, however, these will have advanced functionalities (eg, status updates, internal emails, requesting ‘friends’, personalised profile pages) providing opportunities for greater interactivity and participatory communication between users. With respect to the personalised profile pages, participants will have the opportunity to add and/or upload content to their own page, share their information with others, invite outside individuals who are not part of the trial to become their friend and use the website (these individuals will be able to use the website, but will not be involved any aspect of the research trial) and access their ‘friends’ profile pages.

Participants of both conditions will not receive any instruction from the research team on the use of the features of the intervention to which they have been assigned, on the basis the website tools provided are not complex or exceptional by comparison with other widely known websites such as Facebook.

Measures

Physical activity

PA will be evaluated through self-report, using the Active Australia Survey.⁴⁰ The Active Australia Survey evaluates frequency (number of sessions) and duration (hours and/or minutes) of walking for transport and recreation and participation in other moderate-intensity and vigorous-intensity PAs during the previous week. The Active Australia Survey has established acceptable test–retest reliability and validity in the Australian adult population, and has been documented as a useful evaluation tool for detecting intervention-related change in PA behaviours.^41–43 The Active Australia Survey will be used to determine: (1) sufficient PA (150 min of moderate to vigorous PA per week), (2) total minutes of PA per week, and (3) total sessions of PA per week. Total PA time will be calculated as the sum of time spent walking (if continuous and ≥10 min), the time spent doing moderate level PA, plus double the time spent engaging in vigorous PA, which accounts for the higher volume of energy expenditure per unit time that is associated with vigorous PA.⁴⁰

Anthropometric measurements

Participants will be requested to provide self-reported measurements of height (in centimeters) and weight (in kilograms). BMI will be calculated from these measurements. Participants will receive information to assist them to convert imperial measurements to metric, but will not be provided with instructions on how to measure their height and weight.

Other measures

Website engagement and retention will be assessed using various website statistics, self-reported internet self-efficacy and website usability. Website statistics will be collected via Google analytics, a commonly available web traffic analysis tool, and the website databases (10 000 Steps, Walk 2.0). Google analytics allows extracting of group-level and individual-level data on a wide range of variables. Specific to this trial, Google analytics will be used to extract information concerning the number of visits to both websites over time (Web 1.0 and Web 2.0), the number of pages accessed for each website per visit, and the time spent on the website for each visit. In addition to the data extracted from Google analytics, specific information will also be extracted from the database of the intervention websites. Data concerning the number of days with a step entry, number of steps logged, and number of friends/walking buddies will be extracted from both the Web 1.0 and Web 2.0 conditions. Additionally, data concerning the number of step entries with a comment, number of stream posts (ie, status updates), number of comments on friends’ streams, number of goals set (by goal type-daily, weekly, etc) and the number of blog entries will be extracted from the Web 2.0 condition.

Internet self-efficacy will be assessed using the Internet Self-Efficacy Scale (ISES), which has previously shown good reliability and internal consistency.⁴⁴ The ISES is an 8-item survey instrument used to assess participants’ confidence in their ability to execute Internet tasks, gather information and troubleshoot problems with the Internet. A 7-point Likert scale is used to assess participant responses, in which a 7 corresponds with ‘strongly agree’ and a 1 corresponds with ‘strongly disagree’. Website usability will be explored using the System Usability Scale (SUS).⁴⁵ The SUS is a brief survey tool used to access participant usability of a range of interface technologies. It is a 10-item scale, scored on a 5-point scale of strength of agreement. Final scores can range from 0 to 100, where higher scores indicate better usability.⁴⁵ The SUS is a highly robust and versatile tool for accessing usability, reporting good reliability and concurrent validity.⁴⁶

Lastly, the RAND 36 Short Form Health Survey (RAND-36) will be used to assess quality of life.^{47 48} This instrument measures quality of life via 8 health-related concepts, including physical functioning, bodily pain, role limitations due to physical health problems, role limitations due to personal or emotional problems, emotional well-being, social functioning, energy/fatigue and general health perceptions. RAND 36 was developed from the original SF-36 Medical Outcomes Study Survey.⁴⁸ Researchers from the original Medical Outcomes Study-MOS⁴⁹ released a commercial version of SF-36 while the original RAND-36 has been made available license-free from the RAND Corporation. Both survey instruments contain the same items developed for MOS,⁴⁹ however the scoring for the body pain and general health sections are slightly different between the RAND and SF-36.⁴⁸ In scoring RAND 36, precoded numeric values are given to each scale item. All items are then scored on a 0 to 100 range, with a high score representing a more favourable health state. Additionally, items in the same scale are averaged together to create 8 scale scores. Any items left blank are referred to as missing data and are not taken into account when calculating the scale scores.⁴⁸ The RAND 36 has been validated in Australian populations,⁵⁰ has demonstrated suitability for use in the general population,^{51 52} and has been associated with the stage of motivational readiness to changes in PA.^{53 54} All outcome measures are detailed below and in table 1.

Table 1.

Summary of outcome measures

	Data collection instrument	Collection points (months)
Primary outcome measures
Physical activity levels	Active Australia Survey40	0, 3 and 12
Secondary outcome measures
Self-reported anthropometric  measurements	Height	0, 3 and 12
	Weight	0, 3 and 12
Other measures
User engagement and retention	Internet self-efficacy scale44	0 (Baseline only)
	System usability scale45	3 and 12
Self-reported quality of life	RAND-3649	0, 3 and 12
Descriptive information	Demographics questionnaire	0 (Baseline only)
Website usage	Google analytics Website database	3 and 12 3 and 12

Statistical power and sample size

This trial is powered to detect a 4% difference in the prevalence of sufficient PA as defined by National Physical Activity Guidelines between the Web 1.0 and Web 2.0 groups. To achieve this aim with an 80% power with an α level of 0.05, a minimum of 1034 participants per group will be required.⁵⁵ A review of web-based PA interventions suggests that studies that do not include aspects of Web 2.0 had a small effect on change in PA status of participants, and had a dropout rate of approximately 30%.¹⁶ However, given that this study is not a RCT and that there is no contact with participants whatsoever, the number of participants per group has been inflated by 40% to account for the effects of participant drop-out. Across the intervention period, the study will aim to recruit 1447 participants per group, resulting in a total of 2894 participants recruited into the study.

Statistical analysis

All analyses will follow ‘intention to treat’ principles. Main comparisons between groups will be performed using general linear mixed modelling. The impact of missing data will be addressed using multiple imputation. The sample size is inflated to take into account the possible uncertainty arising from data that is not missing at random. All analyses will be conducted using SPSS for Windows (V.20.0). The level of significance (α) will be set at 0.05.

Discussion

Going beyond a traditional efficacy RCT³³ and implementing this intervention to a larger audience, in ‘real-world’ conditions is both novel and necessary. It is beyond any doubt that implementing and evaluating RCTs is still very much warranted, as the outcomes of these tightly controlled studies provide valuable information concerning efficacy and internal validity.^{24 26 56} However, in relation to population-based interventions that are expected to reach a large proportion of the population in a real-world setting, RCTs alone are limited in yielding valid and generalisable evidence.²⁴ By extending our intervention implementation to a ‘real-world’ trial, we are able to gain a greater understanding of the true effectiveness of an intervention in a natural setting which may be influenced by natural external variables that are not necessarily apparent in a RCT. Variables related to health status, motivation, compliance and resources associated with long-term sustainability are just a few of the external variables not taken into account when carrying out a RCT.²³ Being aware of these variables and any other challenges (that may occur due to chance) in a real-world setting, provides us with the opportunity to address them and develop alternative best practice approaches that are logical, plausible and meet the needs of the larger population. The current study provides an opportunity to do this. This real-world trial will provide us with a greater understanding of how to engage and retain participants from the general population, under everyday conditions, in web-based interventions aimed at increasing PA and other health-related behaviours. More importantly, this trial is the first step to understanding the factors associated with population-level dissemination and research translation. Being able to identify the successful components and challenges associated with of Web 2.0 features, particularly concerning engagement and retention, will assist with intervention refinement and further intervention delivery, ultimately impacting PA behaviours and subsequently preventing chronic disease and improving global health.