A Randomized Controlled Trial of an Educational Intervention Using an Online Risk Calculator for Knee Osteoarthritis: Effect on Risk Perception

Abstract

Objective

Young adults, in general, are not aware of their risk of knee osteoarthritis (OA). Understanding risk and risk factors is critical to knee OA prevention. We tested the efficacy of a personalized risk calculator on accuracy of knee OA risk perception and willingness to change behaviors associated with knee OA risk factors.

Methods

We conducted a randomized controlled trial of 375 subjects recruited using Amazon Mechanical Turk. Subjects were randomized to either a) use a personalized risk calculator based on demographic and risk factor information (intervention) or b) view general OA risk information (control). At baseline and after the intervention, subjects estimated their 10-year and lifetime risk of knee OA and responded to contemplation ladders measuring willingness to change diet, exercise, or weight-control behaviors.

Results

Subjects in both arms had an estimated 3.6% 10-year and 25.3% lifetime chance of developing symptomatic knee OA. Both arms greatly overestimated knee OA risk at baseline, estimating a 10-year risk of 26.1% and a lifetime risk of 47.8%. After the intervention, risk calculator subjects’ perceived 10-year risk decreased by 12.9 percentage points to 12.5% and perceived lifetime risk decreased by 19.5 percentage points to 28.1%. Control subjects’ perceived risks remained unchanged. Risk calculator subjects were more likely to move to an action stage on the exercise contemplation ladder (RR = 2.1). There was no difference between the groups for diet or weight-control ladders.

Conclusions

The risk calculator is a useful intervention for knee OA education and may motivate some exercise-related behavioral change.

Introduction

Knee osteoarthritis (OA) is a highly prevalent disease that affects around 14 million adults in the United States (1). Although knee OA is generally viewed as a disease related to aging, recent data suggest that half of diagnoses are made in patients younger than 55 (2). Numerous risk factors for developing OA have been identified, including obesity (3–6), knee injury (4,5), occupational exposure (7), older age (4,5), and female sex (4,5). OA prevention efforts have traditionally targeted populations that have already accumulated major risk factors (8,9). As disease-modifying drugs are not currently marketed, prevention strategies focused on risk factors are essential for reducing the future burden of knee OA (10). Weight loss over about 5kg over a decade halved the risk of knee OA in women in the Framingham knee OA study (11). Exercise therapies have also been shown to reduce knee injury in younger cohorts (10,12).

Few studies have focused on how well individuals understand their risk and how risk perception may affect OA prevention efforts (13). Young adults are generally unaware that some risk factors for OA are modifiable (13). Furthermore, they may over- or under-estimate their risk due to lack of information. Accurate risk assessment is important, as people who underestimate their risk may lead a less healthy lifestyle, while those who overestimate their risk may perceive of the disease as inevitable and may not take preventative measures (14–16). Prevention efforts should therefore focus on a younger population prior to accrual of modifiable and preventable risk factors.

Risk calculators have been developed for chronic diseases such as diabetes, cancer, stroke, and chronic heart disease (17–21) in order to provide patients with accurate information about their risk, and, potentially, to make them more amenable to prevention. Risk calculators have improved risk perception accuracy for colorectal cancer (22) and have been shown to be effective in increasing intention to adhere to periodontal treatment (23). Personalized risk communications also have been shown to enhance informed decisions about screening procedures (24).

We developed an online risk calculator for knee OA that uses demographic and risk factor information to estimate risk. Its acceptability was tested in a clinical setting (25) and we have also used it to assess the perception of risk in the general population (13). This study tested the effectiveness of the calculator in a randomized trial that examined whether use of the calculator could educate young adults regarding their individualized risk of knee OA. Study participants also responded to contemplation ladders, which are instruments used to measure willingness to change. Participants ranked their willingness to change diet, exercise, and weight-control before and after the intervention. These data provide insight as to whether education regarding the risk of knee OA affects willingness to engage in behavioral change related to obesity, a major risk factor for knee OA. We hypothesized that subjects presented with personalized risk information would improve their understanding of their knee OA risk and be more willing to alter behavior associated with OA risks factors than subjects shown generalized information about OA.

Subjects and Methods

Risk Calculator Development

We developed an online risk calculator that provides personalized risk estimates for OA based on data entered by the user. The risk estimates used in the calculator were developed from the Osteoarthritis Policy Model (OAPol), a validated, state-transition, Monte Carlo simulation of population characteristics and treatments for knee OA (2,26). Risk factors were incorporated into the model using a prediction model published by Zhang et al (27). We carried out 1,920 model simulations using combinations of the following characteristics: age (25–45), sex (male or female), race (White, Black, Hispanic), obesity status (obese, not obese), occupational exposure (yes, no), family history of OA (yes, no), and history of knee injury (yes, no). After calculator users input their demographic and risk factor data, they are matched with the corresponding OAPol output, which is used to show them their personalized risk of developing OA. Details on the development of the OA Risk Calculator (OA Risk C) are published elsewhere (25). The OA Risk C is available at oariskc.arthritisrisk.org.

Online platform and screening

We conducted the study using Amazon’s Mechanical Turk, an online crowd-sourcing platform (28). Workers on Amazon Mechanical Turk are paid to complete Human Intelligence Tasks (HITs), which are small jobs posted by other users. Mechanical Turk is often used to identify study participants, as it provides a large number of individuals at a low cost. Mechanical Turk workers find HITs by searching through a database that they can sort by payment and estimated completion time. Studies examining Mechanical Turk demographics have concluded that Mechanical Turk workers more closely represent the US population than convenience samples (29), and Mechanical Turk data has test-retest reliabilities similar to traditional social science collection methods (30). Mechanical Turk participants also have exhibited the same biases as subjects recruited in a university setting (31). The Mechanical Turk platform has been frequently used in research including studies on behavioral psychology, knowledge of ovarian cancer, perception of sexual risk in older adults, and perception of research risks among cocaine users (32–35).

We identified workers on Mechanical Turk with a screening HIT using a survey to exclude those who did not meet inclusion criteria (age 25–45, not diagnosed with OA, and US resident). To ensure quality data, participants were required to have at least 1000 previously completed HITS and an approval rate of 98% from requesters of previous tasks. Workers were paid $0.05 for completing the screening HIT. Eligible workers were then given access to the main risk calculator study.

Study Design

When eligible participants were given the link to the risk calculator study, they were sent to an external site with the main study, a parallel two-arm randomized trial. All subjects filled out demographic and risk factor information, completed contemplation ladders, and indicated their perceived risk for OA prior to viewing the trial materials. These data provided the baseline assessment. Subjects were then randomized to one of the two arms using a variable size block randomization. The control arm viewed general information about OA. The materials included a general description of osteoarthritis and its symptoms (knee pain and functional limitation) and an explanation that patients with knee OA often undergo total knee replacement, which can improve symptoms. The materials also contained information on three risk factors for knee OA: obesity, knee injury, and occupational hazards. The intervention arm viewed both the general information about OA and used the personalized risk calculator.

After using either the control or intervention materials, users were asked again to estimate their perceived risk of knee OA and rank their willingness to undergo lifestyle changes using contemplation ladders. Study participants in both arms were paid $3.00 upon completion of the study.

Contemplation Ladders

Contemplation ladders were developed based on the transtheoretical model of change, in which people have varying levels of readiness to change their behaviors. They were originally developed and validated for smoking cessation (36–39) and have been since validated for substance use disorders (40). Level of willingness to change, as indicated by ladder responses, is predictive of behavioral changes for quitting smoking or responsiveness to addiction treatment (36,39–41). Contemplation ladders have been modified for a variety of activities, such as going to the dentist (42) and decreasing drug use (40,41).

The ten rungs on the ladder correspond to stages of change, with 0 (lowest rung) representing pre-contemplation and 10 (highest rung) representing advanced action (39). Alternating rungs on the ladder contain a statement anchoring the rung to the appropriate stage. For example, on the diet ladder rung 0 says “No thoughts about monitoring my diet” and rung 10 says “I am taking action to monitor my diet”. Participants marked the appropriate height on contemplation ladders for increasing exercise, monitoring diet, and controlling weight.

Outcomes

The primary outcome was the change in reported perception of 10-year and lifetime risk of knee OA. The secondary outcome was measured by the percentage of participants who were not in an action stage, a score of 8 or higher, at baseline and who reached an action stage after the intervention. We compared responses, on average, between the control (general OA information) and intervention (personalized risk calculator and general OA information) arms.

Statistical Methods

We estimated that a sample of 173 subjects per group would provide 90% power to observe an effect size of 0.35 standard deviations or greater in change in risk perception, assuming an alpha level of 0.05. We evaluated the success of randomization between the intervention and control arm using t-tests for continuous variables and chi-square tests for categorical variables. We used t-tests for primary analysis and generalized linear models (implemented as Poisson regression in SAS v9.4) to estimate the increases in reaching action stages in intervention compared to control arms across the three contemplation ladders, after adjusting for the obesity status and region.

Results

Cohort Characteristics

One thousand participants responded to the screening survey posted on Amazon Mechanical Turk. Of these, 591 were eligible for the trial. Three hundred and seventy-seven participants were randomized and 375 provided complete data that were used for analysis (Figure 1). Screening for the study began on April 20, 2015 and the study finished on May 17, 2015 after the target number of responses was met.

Figure 1.

Consort flow diagram of study participants

Study participant characteristics are provided in Table 1. Both the control and intervention arms had a mean age of 32 and a mean BMI of 27. Fifty-two percent of both arms were male. Eighty percent of the control arm was White, 5% Black, 6% Hispanic, and 9% other. In the intervention arm, 77% of the arm was White, 6% Black, 5% Hispanic, and 12% other.

Table 1.

Study participant characteristics

	Control	Risk Calculator	Overall
Age	32 (5)	32 (5)	32 (5)
BMI	27 (6)	27 (6)	27(6)
Obesity Group
Normal weight (BMI<25)	80 (44%)	84 (44%)	164 (44%)
Overweight: BMI 25–29.9	57 (31%)	58 (30%)	115 (31%)
Obese: BMI ≥ 30	46 (25%)	50 (26%)	96 (26%)
Race/Ethnicity
White	147 (80%)	148 (77%)	295 (79%)
Black	9 (5%)	11 (6%)	20 (5%)
Hispanic	10 (6%)	10 (5%)	20 (5%)
Other	17 (9%)	23 (12%)	40 (11%)
Education
Some High School or High School Degree	19 (10%)	20 (10%)	39 (10%)
2 Year College Degree or Some 4 Year College	59 (32%)	66 (34%)	125 (33%)
Bachelor’s Degree or higher	105 (57%)	106 (55%)	211 (56%)
Region
1 (Northeast)	24 (13%)	39 (20%)	63 (17%)
2 (Midwest)	34 (19%)	34 (18%)	68 (18%)
3 (South)	68 (37%)	76 (40%)	144 (38%)
4 (West)	57 (31%)	43 (22%)	100 (27%)
Risk Factors
Family History	128 (70%)	122 (64%)	250 (67%)
Occupational Exposure	94 (51%)	85 (44%)	179 (48%)
Injury	37 (20%)	39 (20%)	76 (20%)
Number of Risk Factors
0	18 (10%)	16 (8%)	34 (9%)
1	38 (21%)	53 (28%)	91 (24%)
2	52 (28%)	54 (28%)	106 (28%)
3	75 (41%)	69 (36%)	144 (38%)

The study sample was geographically diverse. Thirteen percent of the control group was from Northeastern states, 19% was from Midwestern states, 37% was from Southern states, and 31% was from Western states. Similar geographic distribution was present in the intervention group, though a smaller percentage was from the West and a larger percentage was from the Northeast.

The two study arms had similar distributions of knee OA risk factors. Seventy percent of the control group and 64% of the OA Risk C group reported family history of arthritis, hip or knee replacement, or finger nodes. Fifty-one percent of the control group and 44% percent of the risk calculator group had a history of occupational exposure to OA risk factors such as kneeling, and 20% of both groups had a history of knee injury. Forty-one percent of the control group and 36% of the risk calculator group had all three risk factors.

Primary Outcome: Change in Risk Perception

As estimated by the OA Risk C (25), subjects in both arms had on average a 3.6% chance of developing symptomatic knee OA in the next 10 years and a 25.3% chance of knee OA development in their lifetime.

At the baseline, control subjects had a mean 10-year risk perception of 26.9% (95% CI: 23.2 – 30.6%) and a mean lifetime risk perception of 48.0% (95% CI: 43.9 – 52.1%). Similarly, at the baseline, intervention arm participants had a mean 10-year risk perception of 25.4% (95% CI: 22.0 – 28.8%) and a mean lifetime risk perception of 47.6% (95% CI: 43.5 – 51.7%).

However, after using the risk calculator or viewing general OA information, depending on the arm assignment, the perception of OA risk differed. For subjects in the intervention arm (who had seen their personal risk by using the OA Risk C), perceived 10-year risk decreased by 12.9 percentage points (95% CI: −15.7 – −10.1) to 12.5% (95% CI: 10.0 – 15.0%) and perceived lifetime risk decreased by 19.5 percentage points (95% CI: −22.9 – −16.0) to 28.1% (95% CI: 25.2 – 31.0%). In contrast, the control arm subjects who viewed the general OA information did not report change in their risk perception; perceived 10-year risk increased by 0.8 percentage points (95% CI: −1.3 – 2.8) to 27.7% (95% CI: 23.7 – 31.6%) and perceived lifetime risk decreased by 0.9 percentage points (95% CI: −3.5 – 1.6) to 47.1% (95% CI: 42.7 – 51.4%) (Table 2).

Table 2.

Efficacy of the online, personalized OA risk calculator (OA Risk C) in changing OA risk perception

Risk Perception		Control	Risk Calculator	Difference*
10 Year Risk	Before	26.9 (23.2 – 30.6)	25.4 (22.0 – 28.8)
	After	27.7 (23.7 – 31.6)	12.5 (10.0 – 15.0)
	Change	0.8 (−1.3 – 2.8)	−12.9 (−15.7 – −10.1)	13.8 (10.3 – 17.3)
Lifetime Risk	Before	48.0 (43.9 – 52.1)	47.6 (43.5 – 51.7)
	After	47.1 (42.7 – 51.4)	28.1 (25.2 – 31.0)
	Change	−0.9 (−3.5 – 1.6)	−19.5 (−22.9 – −16.0)	18.6 (14.2 – 22.9)

^*Adjusted for region

Secondary Outcome: Willingness to Change

Willingness to change was measured by the percentage of subjects who were not at an action stage prior to the intervention and moved to an action stage, measured by contemplation ladder, after the intervention. Subjects in the intervention arm were more likely to reach an action stage on the exercise ladder; 26.9% of risk calculator subjects moved into an exercise action stage, compared to 13.6% of control subjects (RR = 2.1, 95% CI: 0.9 – 4.7). For weight control and diet contemplation ladders, the data did not suggest that the intervention increased subjects’ willingness to move to an action stage (RR = 1.0 and 1.3 respectively) (Table 3).

Table 3.

Efficacy of the online, personalized OA risk calculator (OA Risk C) in increasing willingness to engage in behaviors preventing accumulation of knee OA risk factors

	Percent Moving to Action Stage		Relative Risk*
	Control	OA Risk C
Weight Management
Contemplation Ladder N=155	17.7%	17.1%	1.0 (0.5–2.2)
Diet Contemplation Ladder N=150	23.7%	29.7%	1.3 (0.7–2.5)
Exercise Contemplation Ladder N=133	13.6%	26.9%	2.1 (0.9–4.7)

^*Adjusted for region and obesity

Discussion

We have presented results of a randomized controlled trial comparing subjects’ perceptions of knee OA risk before and after using a personalized, online knee OA Risk Calculator (OA Risk C). We also compared willingness to engage in three behavioral changes, increasing exercise, monitoring diet, and controlling weight, that would prevent weight gain, a well-established risk factor for knee OA. Prior to the intervention, both groups highly overestimated their risk of developing knee OA in 10 years and over the course of their lifetime. Use of the personalized risk calculator facilitated more accurate perception of knee OA risk. Moreover, the subjects who viewed their personalized risk were more likely to move to an action stage on a contemplation ladder for increasing exercise. However, there was no difference between the arms in the percentage of subjects who moved to an action stage for monitoring their diet or controlling their weight.

Providing personalized risk estimates has been shown to increase knowledge and decision making for a variety of illnesses (22–24,43). One systematic review of randomized controlled trials of personalized risk information in screening procedures found that participants who were provided with personal risk information made more informed choices about screening compared to control subjects (24). Another study of online risk calculators for prediabetes found that in subjects who overestimated their risk, personalized risk information reduced their perceptions by 16%. This corresponds to our finding that OA Risk C subjects overall decreased their 10 year perception of risk by 13%. However, subjects in the prediabetes study on average estimated a lower risk than predicted by the prediabetes calculator, and the risk calculator did not improve perceptions of risk for subjects who originally underestimated their risk (43). In our study, subjects greatly overestimated their risk for knee OA prior to using the OA Risk C, which may indicate a difference in perceptions regarding the risk of prediabetes and knee OA.

This study suggests that viewing personalized risk information for knee OA motivates subjects to be willing to change their exercise habits. As the risk calculator intervention is brief, implementing further interventions after viewing personalized risk may allow subjects to act on this momentum.

The study has several limitations. The risk calculator subjects had more exposure to knee OA information, as they received both the educational materials and the calculator estimates. This may partially explain their more accurate risk perception and willingness to increase exercise after the intervention. The intervention was brief and may not have had a lasting impact on the subjects. Due to the cross-sectional nature of our study, our data do not indicate whether subjects will act on increased motivation to change behavior nor do our data indicate how long subjects’ increased motivation will last. Studies in other diseases have found that risk perception interventions have a sustained effect on accurate risk perception. In longitudinal studies, improved perception of breast cancer risk has lasted at least 6 months (44) and 18 months (45). Future studies should include a longitudinal component to address the impact on behavior change over time.

Prior to the intervention, many subjects had willingness to change scores that were already in the action stage, so we were only able to measure a change in willingness to change in a smaller sample. The population was limited to persons between the ages of 25 and 45, and consisted of users of Amazon’s Mechanical Turk, who may have somewhat higher educational attainment by virtue of their comfort with the internet-based platform. However, studies examining Mechanical Turk demographics have concluded that Mechanical Turk workers more closely represent the US population than convenience samples (29).

Another concern about participants recruited on Mechanical Turk is that they are habitual survey-takers, which may bias their responses. The data provided by Berinsky et al. showed that of the seven studies they posted on Mechanical Turk, only 2% of MT workers responded to five or more. Moreover, there was no significant difference in study responses between those who responded to many studies and those who did not (29). Our study was only posted once and had no other iterations, making it unlikely that users had previously taken a similar survey.

Finally, the risk estimates used by the OA Risk C were derived using the OAPol model and are informed by published data from multiple sources.

People who viewed personalized estimates of 10-year and lifetime OA risk appear to have a more accurate perception of their risk and are more willing to change their exercise habits. Accurate risk perception is important, as underestimation of risk can lead to less healthy behavior and increased risk-taking due to a false sense of security (16,22,46,47). Conversely, overestimation of risk can be associated with a disinclination to modify risky behaviors, as the risk is perceived as inevitable (15). Overestimation of risk is also associated with increased anxiety and distress (48), and overestimation may increase use of health resources unnecessarily (22,49). While it is conceivable that users who overestimated their risk might increase risk taking after learning that their risk is lower than expected, the OA Risk C allows users to view their risk with or without risk factors. This may discourage additional risk-taking behaviors, as users can see what their increase in risk would be if they accumulated risk factors.

This study suggests that the knee OA Risk C may be a successful public health tool both for raising awareness of knee OA risk and increasing motivation to change exercise-related behaviors that are associated with OA risk. To determine the optimal way to implement the OA Risk C, future studies may examine if its effect on motivation changes based on the user’s age or other demographic factors. Since most of weight accumulation occurs in early middle age (50), the OA Risk C tool could be most helpful for the population between 25 and 45 years of age. Some risk prevention, especially related to injury prevention, could and should also occur earlier in life.

Significance and Innovations.

• The knee osteoarthritis risk calculator (OA Risk C) is an effective educational tool for improving accuracy of 10-year and lifetime perception of knee OA risk

• The OA Risk C increases willingness to change exercise behaviors