The impact of computer self-efficacy, computer anxiety, and perceived usability and acceptability on the efficacy of a decision support tool for colorectal cancer screening

Abstract

Objective

This study investigated the efficacy of an internet-based personalized decision support (PDS) tool designed to aid in the decision to screen for colorectal cancer (CRC) using a fecal occult blood test. We tested whether the efficacy of the tool in influencing attitudes to screening was mediated by perceived usability and acceptability, and considered the role of computer self-efficacy and computer anxiety in these relationships.

Methods

Eighty-one participants aged 50–76 years worked through the on-line PDS tool and completed questionnaires on computer self-efficacy, computer anxiety, attitudes to and beliefs about CRC screening before and after exposure to the PDS, and perceived usability and acceptability of the tool.

Results

Repeated measures ANOVA found that PDS exposure led to a significant increase in knowledge about CRC and screening, and more positive attitudes to CRC screening as measured by factors from the Preventive Health Model. Perceived usability and acceptability of the PDS mediated changes in attitudes toward CRC screening (but not CRC knowledge), and computer self-efficacy and computer anxiety were significant predictors of individuals' perceptions of the tool.

Conclusion

Interventions designed to decrease computer anxiety, such as computer courses and internet training, may improve the acceptability of new health information technologies including internet-based decision support tools, increasing their impact on behavior change.

Introduction

Colorectal cancer (CRC) is a major public health problem. It is the most commonly diagnosed internal cancer in Australia¹ and the second most common cause of cancer in the Western world.² Given the slow progression of the disease, current strategies focus on the early detection of curable lesions through screening using endoscopic means or fecal occult blood tests (FOBTs). Population screening with FOBT has been shown to reduce mortality by 15–35% assessed on an intention-to-screen basis relative to an unscreened population.³ However, the efficacy of population-based screening programs depends in part on high participation rates, but in many countries these have not been achieved.⁴ This highlights the need to develop strategies to increase participation rates.

Cancer decision aids have been designed to overcome some of the problems associated with poor participation in cancer prevention programs. The purpose of these aids is to assist the decision making of people facing health treatments or screening decisions through increasing knowledge and awareness of personal risk and response alternatives. A systematic review of health-related decision aids concluded that they improve knowledge, reduce decisional conflict, and stimulate individuals to be more active in their decisions, without increasing anxiety.⁵ Another review article focusing exclusively on cancer decision aids confirmed that they improved knowledge compared with usual practice, and led to a significant reduction in anxiety when used in a screening context.⁶ Thus, decision aids for cancer screening hold promise as a means of improving participation in CRC screening, while minimizing negative side effects such as anxiety and decisional conflict.

Early research on cancer decision aids used simple, paper-based pamphlets and booklets to help people decide whether to screen for CRC. However, more recently widespread use of the internet has allowed more cost-effective, high-quality, dynamic, and interactive resources to be developed and utilized.⁷ Over the past 11 years, the rate of internet access across the world has increased by approximately 400%.⁸ In December 2009, 27% of the world's population was estimated to have internet access, with much higher proportions in Europe, the United States, and the Oceania/Australia region (58%, 78%, and 60%, respectively). This increase in internet access over the past decade has led to a plethora of internet-based tools designed to help patients make decisions about their health. Importantly, there is some evidence that web-based interventions are more successful at achieving knowledge and targeted behavior change than non web-based interventions.⁹

A recent review¹⁰ found that internet-based decision support tools have a small but statistically significant effect on health behavior (Cohen's d=0.16). The review also found that the efficacy of these tools is improved if they incorporate behavior change techniques and their development is based on the extensive use of theory. Another factor that may be important to the efficacy of decision support tools is the perceived usability and acceptability of the tool in the target group. Perceived usability refers to the perceived ease of using the tool.¹¹ For example, if the internet-based tool is difficult to navigate, the font is too small, or the links are ineffective, then one might posit that users would gain less benefit from the tool than if otherwise so. Perceived acceptability refers to the willingness of the members of the target group to utilize the website and has been associated with individual differences in computer self-efficacy and computer anxiety,¹² particularly in the early stages of utilizing a decision support tool. Specifically, people who have lower computer self-efficacy or higher computer anxiety might be expected to perceive the tool as less useful and acceptable, and therefore conceivably benefit less from its utilization than those more competent with computers.

Consideration of usability and acceptability is critical for decision support related to CRC screening because the targeted users are people aged over 50 years. Research indicates that computer anxiety tends to be higher in older adults¹³ and it is therefore likely that computer anxiety and computer self-efficacy may be important mediators of the benefits of internet-based cancer decision support tools for CRC management.

The main aim of this study was to test the efficacy of a recently developed, prototype internet-based decision support tool designed to optimize decision making around CRC screening participation, and to test the extent to which efficacy is affected by usability, acceptability, and attitude to computers. The tool was developed utilizing the Preventive Health Model¹⁴ and the Precaution Adoption Process Model.¹⁵ It provides a measure of current knowledge of screening and attitude to FOBT use (ie, the Preventive Health Model; variables salience and coherence of screening, perceived susceptibility to CRC, cancer worries, social influence on decision making, and response efficacy) and readiness to screen (ie, Precaution Adoption Process Model stage).

This study investigated the efficacy of an internet-based personalized decision support (PDS) tool designed to aid in a decision to screen for CRC using an FOBT. We tested whether the efficacy of the tool in influencing attitudes to screening was moderated by perceived usability and acceptability, and considered the role of computer self-efficacy and computer anxiety in these relationships. Our principal hypothesis was that the tool would improve knowledge of and attitudes toward CRC screening and that these effects would be more pronounced in people who perceived the tool as highly useable and acceptable. We predicted that people with low computer self-efficacy and/or high computer anxiety would perceive the tool as less useable and acceptable and consequently would experience less change in attitudes and knowledge.

Methods

Participants and procedure

Participants (N=81; 45 women, aged 50–76 years) were recruited from the general public using snowball sampling. About half of the participants had completed some degree of post-secondary education, which is comparable to the general population in this age group.¹⁶ People who expressed interest in the study and had internet access were emailed an information sheet, which provided instructions to access websites containing the on-line questionnaires and the decision support tool. People who were interested but lacked internet access were contacted by telephone and invited to complete the study at a residence located close to the participants' home. Once they arrived they were provided with a printed version of the information sheet and access to a computer in order to complete the questionnaires and decision support tool.

Data were collected in three parts. First, participants completed an on-line questionnaire to assess computer anxiety, computer self-efficacy, and CRC knowledge. Second, participants worked through the on-line PDS tool, which assessed the Preventive Health Model variables described above. They were then presented with tailored messages based on their responses to these variables (see Intervention section below for more details). Third, participants completed a follow-up on-line questionnaire to capture demographic information, assess perceived usability and acceptability of the PDS, and gather post-exposure measures of CRC knowledge and scores on the model variables. This study was approved by the Ethics Committee of the School of Psychology at the University of Adelaide.

Intervention

The PDS tool used in this study is an internet-based application that collects user information in real time and delivers instantaneous, personalized messages aimed at moving individuals through the decision stages relevant to CRC screening (http://bowelcancerscreening.csiro.au). An extensive user model drives a series of algorithms that underlie an educational message library. These algorithms have been developed to ensure that messages delivered to an individual are united with natural language so that they read in a coherent, logical manner. After log-in, individuals completed a baseline survey that incorporated Preventive Health Model and Precaution Adoption Process Modelⁱ variables and collected demographic details. These variables were assessed online; 17 items measured all five Preventive Health Model factors and one measured Precaution Adoption Process Model stage.

Baseline survey items are rated on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree.’ Before describing in more detail the personalized messages provided by the PDS, we will first discuss the Preventive Health Model.

The Preventive Health Model¹⁴ is a continuum model that has been used extensively in the area of CRC screening. The model focuses on five variables: salience and coherence (the extent to which performing CRC screening is consistent with beliefs about how to protect and maintain health); cancer worries (concerns about the consequences of CRC); response efficacy (beliefs that undertaking CRC screening will be effective in reducing disease threat); social influence (beliefs about, and desire to comply with, the attitudes of key others to CRC screening); and perceived susceptibility (subjective personal risk for developing CRC). All five factors have been linked to intention to screen for CRC and/or screening behavior in previous research,^{14 17} and a recent confirmatory factor analytic study by Flight et al¹⁸ has established cross-cultural validation for the five-factor model for use in Australia.

Once the Preventive Health Model variables are measured, the computer program operates on these data to determine the individualized content to be delivered. The resulting web interface consists of tailored, personalized messages that address relevant knowledge deficits and reinforce perceptions when favorable to screening, or motivate people to change perceptions when unfavorable. For example, a library of messages has been created to address response efficacy. The response efficacy statement reads ‘When colorectal polyps are found, colorectal cancer can be prevented.’ Participants who respond ‘disagree’ receive a personalized, ‘motivating’ message reading: ‘(Participant's Name), you really don't believe that colon cancer screening is effective. In fact, it's very effective—that's why the Australian Cancer Council recommends yearly screening for people over 50 who are of average risk. As you are (participant's age), screening can save your life by finding early, curable cancer.’ Participants who respond ‘agree’ to the statement would receive a ‘reinforcing’ message: ‘(Participant's Name), you've told us that you believe colon cancer screening is effective. You're absolutely right … (continues as per above).’

The intervention and questionnaire measures were examined in a pilot study with 42 adults between 50 and 75 years of age, recruited through snowball sampling, who attended the CSIRO laboratories. The study assessed changes in scores on the Preventive Health Model variables before and after exposure to the PDS tool and compared a group who received tailored messages (n=22) with those who received untailored, web-based education (n=20). The participants also completed questionnaire measures of computer anxiety, CRC and screening knowledge, and perceived usability and acceptability. Subsequent focus groups with participants indicated that the materials were acceptable and understandable. The PDS tool was also deemed to be both usable and acceptable to this sample.

Questionnaires

The potential influence of a number of mediating and moderating variables was assessed. The details of these variables are provided below.ⁱⁱ The total of 82 items required to assess these had the potential to create a significant respondent burden, although assessment of reliability data indicates that the data have good fidelity.

Computer self-efficacy

The Murphy Computer Self-Efficacy Scale¹⁹ was adapted for use in the current study. Fifteen questions from the 32-item scale were selected as they were deemed most relevant and appropriate to measure computer self-efficacy in this age group (items available from the corresponding author). The 15-item scale had high internal consistency (Cronbach's α>0.90). All items were scored on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree,’ with higher scores representing higher computer self-efficacy. The scale included items such as ‘I feel confident copying a file’ and ‘I feel confident learning to use different computer programs.’

Computer anxiety

Computer anxiety was measured using the Computer Anxiety Rating Scale.²⁰ Participants responded to 18 items on a 5-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree.’ The scale includes items such as ‘I hesitate to use a computer for fear of making mistakes that I cannot correct’ and ‘I am confident that I can learn computer skills’ (reversed item) and higher scores indicate greater computer anxiety. This scale had high internal consistency (Cronbach's α>0.90).

Colorectal cancer and screening knowledge

A 13-item scale consisting predominantly of true/false questions was developed to assess knowledge of CRC and screening. The scale included items such as ‘bowel cancer screening requires the completion of a colonoscopy,’ ‘if you have bowel cancer you will have obvious symptoms,’ and ‘the risk of developing colon polyps increases with age.’ Given that most items were dichotomous (true/false), it is not valid to examine the internal consistency of this scale.

Perceived usability and acceptability of the PDS

A 36-item questionnaire was developed to assess perceived usability and acceptability. Twenty-nine items were adapted from a study that evaluated the perceived usability of a computer-tailored intervention designed to promote physical activity.²¹ These items were re-worded to focus on the PDS and its advice about CRC screening. A further seven questions were developed for a pilot study of the PDS and included items such as ‘I felt it was easy to find the information I was looking for’ and ‘I believe the tool's content was up to date.’ All items were measured on a 5-point Likert scale from ‘strongly agree’ to ‘strongly disagree.’ This scale had high internal consistency (Cronbach's α>0.90).

Results

Table 1 presents scores on CRC knowledge, and attitudes toward CRC screening before and after exposure to the PDS. Exposure to the PDS led to a significant increase in knowledge about CRC and screening for CRC (partial η²=0.28), and a significant improvement in attitudes toward CRC screening as shown by an increase in salience and coherence (partial η²=0.07), perceived susceptibility (partial η²=0.17), social influence (partial η²=0.05), and response efficacy (partial η²=0.22). Conversely, PDS exposure led to a significant reduction in cancer worries (partial η²=0.08). Taken overall, these changes suggest that the PDS is effective in modifying important attitudes to and beliefs about CRC and CRC screening in the desired direction.

Table 1.

Knowledge of CRC and screening, and attitudes toward CRC screening before and after exposure to the PDS

Scales	Mean before	SD	Mean after	SD	F value	Effect size
CRC and screening knowledge	9.57	2.74	11.23	1.47	30.72**	0.28
Salience and coherence	15.72	4.09	16.99	2.72	6.32*	0.07
Cancer worries	5.59	2.42	5.04	2.02	7.14**	0.08
Perceived susceptibility	11.96	4.47	14.17	4.06	16.05**	0.17
Social influence	14.83	4.04	15.91	3.48	4.30*	0.05
Response efficacy	7.47	1.73	8.51	1.24	22.71**	0.22

Pre and post-exposure scores were compared using repeated measures ANOVA. Effect size is partial η².

*p<0.05; **p<0.01.

CRC, colorectal cancer; PDS, personalized decision support tool.

Table 2 presents descriptive statistics for perceived usability and acceptability, computer self-efficacy, and computer anxiety. To test the moderating effect of perceived usability and acceptability on changes in knowledge and attitudes, usability and acceptability was transformed from a continuous variable to a factor using a median split of scores. Thus, participants were classified as having high perceived usability and acceptability (n=42) if this score was above the median score of 131 and low (n=39) if below the median. Repeated measures ANOVAs were run and the time×perceived usability and acceptability interaction effect was examined to test whether change in attitudes and knowledge about CRC were mediated by perception (see figure 1).

Table 2.

Descriptive statistics for perceived usability and acceptability, computer self-efficacy and computer anxiety

	PUA	CSE	CA
Perceived usability and acceptability (PUA)
Computer self-efficacy (CSE)	0.498**
Computer anxiety (CA)	−0.514**	−0.952**
Mean	129.99	45.25	46.52
SD	20.44	21.89	19.53
Range	74–162	15–75	18–79

**p<0.01.

Figure 1.

Colorectal cancer (CRC) knowledge and attitudes toward screening before and after exposure to the personalized decision support tool, by level of perceived usability and acceptability (PUA).

Perceived usability and acceptability had no effect on the changes in knowledge about CRC and CRC screening (F (1, 79)=0.24, p>0.05, partial η²=0.01) with both groups showing near identical improvements in knowledge after exposure to the PDS (see figure 1). However, perceived usability and acceptability had a significant effect on the changes in salience and coherence (F (1, 79)=10.1, p<0.01, partial η²=0.11), cancer worries (F (1, 79)=7.63, p<0.01, partial η²=0.09), perceived susceptibility (F (1, 79)=12.1, p<0.001, partial η²=0.13), response efficacy (F (1, 79)=9.29, p<0.01, partial η²=0.11), and social influence (F (1, 79)=4.42, p<0.05, partial η²=0.05). In all cases, people who perceived the PDS as highly usable and acceptable experienced changes in attitudes in the desired direction, whereas people with low perceived usability and acceptability showed little to no change.

Table 2 also presents correlations between perceived usability and acceptability, computer self-efficacy, and computer anxiety. People with higher computer self-efficacy had lower computer anxiety and this correlation was extremely high (r=−0.95). As hypothesized, both constructs were significantly associated with perceived usability and acceptability of the PDS tool. Overall, people with higher computer self-efficacy and lower computer anxiety tended to perceive the PDS tool as more usable and acceptable.

Discussion

The main aim of the PDS is to improve people's attitudes to and beliefs about CRC screening, thus increasing the likelihood that they will engage in CRC screening. The results of this study show that this aim was only achieved in those people who perceived the tool as being usable and acceptable. People who perceived the tool as unacceptable or non-user friendly showed no change in their attitudes toward CRC screening, although they did experience an increase in knowledge about CRC and screening. Computer anxiety and computer self-efficacy were found to be key determinants of the perceived usefulness and acceptability of the PDS, and are therefore important factors influencing the potential efficacy of the tool.

One intriguing finding in this study was that the perceived usability and acceptability of the PDS moderated the changes in all of the preventive health model factors but not CRC knowledge. Rather, all participants experienced an increase in their knowledge of CRC and CRC screening, regardless of whether they perceived the tool as useful and acceptable or not (see figure 1). One plausible explanation for this effect is that people who felt the tool was not acceptable or user friendly may have read the tailored feedback and accepted the factual information (eg, the risk of developing CRC increases with age), but they may not have changed their attitudes toward screening itself. This could explain why their knowledge about CRC and screening increased but their attitudes toward screening did not improve.

Studies have shown that knowledge about CRC and screening predicts screening behavior,^{22 23} suggesting that the PDS should lead to an increase in CRC screening, even in people who perceive the tool as unacceptable or non-user friendly. Nonetheless, the ultimate goal of the PDS is to improve people's attitudes to CRC screening and to move them to a higher stage of readiness to screen for CRC. As such, further attempts to improve the perceived usability and acceptability of the tool are warranted.

Low computer self-efficacy and high computer anxiety explained about 25% of the variance in the perceived usability and acceptability of the PDS and, consequently, are key factors influencing users' perceptions of the tool. It has been proposed that interventions aimed at improving computer self-efficacy may be more effective in improving perceived usability and acceptability than modifications to the underlying technology (ie, the PDS tool).²⁴ It is important to note that computer self-efficacy and computer anxiety were correlated almost perfectly in our study and therefore interventions designed to address either one will almost certainly impact on the other factor (ie, through improving computer self-efficacy, for example, computer anxiety is reduced).

Computer anxiety has been shown to be lower in people who have more experience with computers.²⁵ Given the huge increase in computer use, computer ownership, and internet access in recent times, computer anxiety at a population level should be declining due to increased interaction with this technology. With respect to interventions, computer anxiety has been shown to decrease in people who have completed computer courses or training.^{26 27} with more intensive courses leading to more substantial improvements in computer anxiety.²⁸ Publicly funded computer courses for people aged over 50 years may help decrease computer anxiety, leading to an increase in the perceived usability and acceptability of new technologies, and thus the efficacy of health-related decision support tools. Future research is needed to assess whether computer anxiety, computer self-efficacy, and the perceived usability and acceptability of the PDS mediate changes in stage of readiness to screen for CRC and screening behavior.

In conclusion, it is important to acknowledge that this study did not measure actual screening behavior. Although attitude to screening is a significant predictor of intention and intention, in turn, partially predicts behavior, there is a significant amount of unexplained variance in these associations. Consequently, it is not possible to conclude that provision of the PDS would necessarily impact on uptake of screening and research is currently underway to address this issue.²⁹