Mode effects in assessing cancer worry and risk perceptions: Is social desirability bias at play?

Abstract

Background

Risk perceptions and worry are important determinants of health behavior. Despite extensive research on these constructs, it is unknown whether people’s self-reports of perceived risk and worry are biased by their concerns about being viewed negatively by others (social desirability).

Methods

In this study, we examined whether reports of perceived risk and worry about cancer varied across survey modes differing in the salience of social desirability cues. We used data from the National Cancer Institute’s 2007 Health Information National Trends Survey, which assessed perceived cancer risk and worry in one of two survey modes: an interviewer-administered telephone survey (higher likelihood of socially desirable responding; n = 3,678) and a self-administered mail survey (lower likelihood of socially desirable responding; n = 3,445). Data were analyzed by regressing perceived risk and worry on survey mode and demographic factors.

Results

Analyses showed no effect of survey mode on cancer risk perceptions (B = 0.02, p = .55, d = .02). However, cancer worry was significantly higher in the self-administered mode than in the interviewer-administered mode (B = 0.24, p < .001, d = .26). Education moderated this effect, with respondents lower in education exhibiting a stronger mode effect. When cancer worry was dichotomized, the odds of reporting cancer worry were approximately twice as high in the self-administered mode compared to the interviewer-administered mode (OR = 2.13, p < .001).

Conclusions

These results bolster the veracity of self-reported cancer risk perceptions. They also suggest that interviewer-administered surveys may underestimate the frequency of cancer worry, particularly for samples lower in socioeconomic status. Studies are needed to test for this effect in clinical contexts.

INTRODUCTION

Accurately measuring perceived risk and worry about particular diseases is important for research and clinical purposes. Risk perceptions occupy a central role in many theories of health behavior and decision-making (1–3) and are important motivators of behavior. (4, 5) Attempts to gauge public knowledge also often focus on the measurement of risk perceptions. (6–11) Similarly, affective responses to risk – including worry about particular diseases – are important components of decision making (12, 13) and critical motivators of health behavior, independent of risk perceptions. (14, 15) However, excessive worry can interfere with daily functioning and reduce quality of life. (16, 17) Thus, some health promotion efforts seek to induce a low level of worry, (14, 18) while counseling of patients at high risk for certain diseases may seek to alleviate unnecessary worry. (19)

Despite the importance of accurately assessing risk perceptions and worry, it is unknown whether self-reports of these constructs may be subject to social desirability bias. Social desirability bias refers to the tendency to under- or overreport particular behaviors, beliefs, or attitudes in order to avoid being viewed negatively by others. (20) One common method for assessing social desirability bias in survey responding is to capitalize on studies where data are collected using two or more survey modes (e.g., interviews vs. self-administered questionnaires), given that questions asked in interviewer-administered surveys (e.g., face-to-face or telephone) are more likely to elicit socially desirable responses than those asked in self-administered surveys. (21–23)

Previous multi-mode surveys have uncovered evidence of social desirability in self-reports of constructs potentially related to risk perceptions and worry. For instance, people reported higher health-related quality of life and fewer long-standing illnesses when asked in an interviewer-administered telephone survey compared to a self-administered mail survey. (24, 25) Similarly, survey mode effects consistent with social desirability have also emerged in reports of trait anxiety, with people reporting lower general levels of anxiety in a telephone interview compared to a self-administered mail survey. (26) Together, these findings suggest that it may be socially undesirable to report high perceived risk of disease – an indicator of possible ill-health – and worry about disease – a construct related to anxiety.

Research objectives

The purpose of this study was to test for survey mode effects consistent with social desirability in the reporting of cognitive risk perceptions and worry about cancer, constructs that have received a great deal of attention in behavioral research across the cancer control continuum. (14–17, 27–31) Also, this study examined whether survey mode effects in reports of perceived cancer risk and worry were particularly strong for members of certain sociodemographic groups. For example, education may play a role, as education can increase people’s awareness of relevant social norms and thereby increase social desirability effects. (32) Alternatively, education can diminish socially desirable responding if it reduces the extent to which people actually possess an undesirable belief or engage in an undesirable behavior. (33) Studies also report increased scores on tests of socially desirable responding (34) as people age. (35) Social desirability can also vary across genders, in part because there may be different norms and expectations for males vs. females in particular domains (e.g., social desirability leads men to overreport, and women to underreport, number of lifetime sex partners). (21) Finally, members of various racial or ethnic groups may be differentially susceptible to social desirability bias. Previous research has shown that people with individualistic cultural backgrounds are often more willing to reveal information about themselves to out-group members than are people with collectivist backgrounds. (36)

METHODS

Data source and participants

Data were obtained from the National Cancer Institute’s (NCI) 2007 Health Information National Trends Survey (HINTS), a national survey designed to monitor trends in the use of health information and communication technologies in the U.S., as well as access to and use of cancer-related information. The survey collects data from a nationally representative sample of the U.S. civilian, non-institutionalized adult population. Details of survey development, design, and methodology have been published elsewhere. (37–39)

Data were collected from January 2008, through April 2008. In an effort to address declining response rates for random digit dialing (RDD) telephone surveys, (38) HINTS 2007 used a dual-frame sampling design: One frame used RDD techniques to identify households for computer-assisted telephone interviews, and the other frame used the U.S. Postal Service listing of residential addresses to identify a stratified cluster sample of households to receive a mail survey. The overall response rates for the RDD and address frames were 24.2% and 31.0%, respectively. In both survey frames, sample weighting procedures were used to account for respondents’ probability of being selected, to adjust for nonresponse bias, to reduce the sampling variance of estimators, and to produce statistically valid standard errors for estimators. (38) This study is based on data obtained from both the RDD and mail surveys, which allowed us to compare responses presumably provided under pressure to present oneself in a socially desirable light (RDD telephone survey) to responses provided under less pressure to give a socially desirable response (self-administered mail survey). (21–26)

Measures

Perceived cancer risk was measured with the item, “How likely do you think it is that you will develop cancer in the future? Would you say your chance of getting cancer is…” Responses were made on a five-point scale ranging from 1 = very low to 5 = very high. Cancer worry was assessed using the item, “How often do you worry about getting cancer? Would you say…” Response options included 1 = rarely or never, 2 = sometimes, 3 = often, and 4 = all the time. The survey did not provide explicit “don’t know” options (40) for the perceived risk and worry measures.

In addition to analyzing raw (continuous) cancer worry scores, the ratings were recoded into a dichotomous variable, such that responses of rarely or never were recoded as 0 and responses of sometimes, often, and all the time were recoded as 1. This recoding method has been used in previous studies of cancer worry to compare respondents who report worrying about cancer any amount with those who do not. (31, 41–43) Sociodemographic characteristics were also assessed, including respondents’ gender, age, highest level of educational attainment, and race or ethnicity.

Statistical analyses

Analyses were conducted using SUDAAN version 10.0.1 (RTI International). Hierarchical weighted linear regressions were used to examine whether perceived cancer risk and worry were significantly different between the mail and RDD modes. In the first step, all sociodemographic factors and a dummy variable for RDD survey mode were entered. In the second step, interactions between all sociodemographic characteristics and the RDD dummy variable were entered. To account for the complex sampling design of HINTS 2007, jackknife replicate weights were used. A set of 50 weights was applied to the RDD data and a separate set of 50 weights was applied to the mail data, as recommended for examining mode effects in HINTS 2007. (44)

A parallel set of analyses was also conducted on the dichotomized measure of cancer worry. Specifically, a hierarchical weighted logistic regression was used to examine the effects of sociodemographic factors and survey mode (entered in the first step) and the interactions between sociodemographic factors and survey mode (entered in the second step) on whether or not people reported worrying about cancer. For each analysis, the analytic sample included respondents who had data on all predictor variables as well as the relevant outcome measure (i.e., either perceived cancer risk or cancer worry).

RESULTS

Sociodemographic characteristics

Table 1 shows the demographic characteristics of respondents in the two sample frames (N = 7,123; mail n = 3,445; RDD n = 3,678). There were no significant differences in sociodemographic characteristics across the two survey modes.

Table 1.

Demographic characteristics of respondents (N = 7,123) by survey mode: Unweighted counts (n) and weighted percentages (%)

Characteristic	Mail		RDD		OR	p value
	n	%	n	%
Gender
Male	1336	48.4	1425	48.5	1.00
Female	2109	51.6	2253	51.5	1.00	.90
Age
18–34	610	30.6	433	30.8	1.00
35–49	885	29.8	838	28.9	0.96	.21
50–64	1145	23.2	1191	23.9	1.02	.43
65–74	443	8.3	658	8.3	0.98	.56
>74	362	8.1	558	8.1	0.98	.44
Education
Some High School or Less	297	13.8	360	14.2	1.00
High School Graduate	1056	32.6	1083	32.7	0.96	.28
Some College	838	28.6	906	28.5	0.95	.21
College Graduate	1254	24.9	1329	24.6	0.94	.06
Race / ethnicity
Non-Hispanic White	2463	69.2	2948	69.8	1.00
Black/African American	435	11.4	241	11.1	0.96	.25
Hispanic	322	13.2	294	12.7	0.94	.14
Other	225	6.2	195	6.4	1.04	.62
n	3,445		3,678

Note. Odds ratios and p values are from a weighted binary logistic regression predicting survey mode = RDD.

Perceived cancer risk

Table 2 shows predictors of cancer risk perceptions. Reports of perceived risk were similar in the mail (M = 2.66, SEM = 0.03) and RDD modes (M = 2.65, SEM = 0.03). The effect of survey mode did not approach significance in a linear regression controlling for sociodemographic factors (p = .55, d = .02). Respondents in the two oldest age groups had lower perceived cancer risk than did those in the youngest age group, and respondents who were Black, Hispanic, or Other had lower perceived risk than did non-Hispanic Whites. When interaction terms between survey mode and sociodemographic characteristics were tested in Step 2 of the model, none of the interactions reached significance (all ps ≥ .10).

Table 2.

Predictors of perceived cancer risk and cancer worry

Characteristic	Perceived cancer risk		Cancer worry
	B	p	B	p
Gender
Male	…		…
Female	0.00	.94	0.19	<.001
Age
18–34	…		…
35–49	0.03	.57	0.10	.01
50–64	0.02	.72	0.10	.02
65–74	−0.19	.002	−0.05	.21
>74	−0.40	<.001	−0.17	.001
Education
Some High School or Less	…		…
High School Graduate	−0.06	.32	−0.11	.02
Some College	0.02	.74	−0.16	.001
College Graduate	−0.03	.69	−0.19	<.001
Race / ethnicity
Non-Hispanic White	…		…
Black/African American	−0.43	<.001	−0.21	<.001
Hispanic	−0.34	<.001	0.07	.24
Other	−0.47	<.001	−0.09	.10
Survey mode
RDD	…		…
Mail	0.02	.55	0.24	<.001

Note. B and p values are from weighted linear regressions predicting cancer risk perceptions and cancer worry.

Cancer worry

As shown in Table 2, there was a significant effect of response mode on reports of cancer worry, with people in the RDD mode reporting less frequent cancer worry (M = 1.51, SEM = 0.02) than those in the self-administered mail mode (M = 1.75, SEM = 0.02) (d = .26). Other predictors of cancer worry included female gender, lower educational attainment, and age. Respondents between the ages of 35 to 49 and 50 to 65 reported worrying significantly more about cancer than did those under age 35, while those over age 74 reported worrying significantly less than did those under age 35. Black respondents reported worrying less about cancer than did non-Hispanic Whites.

When interactions between sociodemographic characteristics and survey mode were added to the model in Step 2, a significant interaction emerged between survey mode and the dummy variable for Education = College graduate (B = 0.20, p = .04), and a marginally significant interaction emerged between survey mode and Education = Some college (B = 0.17, p = .06). Follow-up analyses revealed that the effect of survey mode was larger among those with less than a high school diploma (n = 644) (B = 0.36, p < .001, d = .43) than among college graduates (n = 2574) (B = 0.19, p < .001, d = .28) and those who completed some college (n = 1737) (B = 0.23, p < .001, d = .32). No other interaction approached significance (all ps > .13).

Dichotomized measure of worry

When cancer worry was recoded into a dichotomous variable, 59% of respondents reported worrying in the mail mode whereas only 41% reported worrying in the RDD mode. In a logistic regression controlling for all sociodemographic characteristics, the effect of survey mode emerged as significant. Respondents were more likely to report worrying about cancer in the mail mode than in the telephone mode (OR = 2.13, p < .001).

Sociodemographic predictors of the dichotomized worry measure were consistent with those of the continuous measure. Males were less likely than females to report worrying (OR = 0.59, p < .001), as were Black respondents (OR = 0.49, p < .001) and those in the “Other” racial/ethnic group (OR = 0.58, p = .001) compared to non-Hispanic Whites. Findings related to age were also consistent: Respondents between the ages of 35 to 49 and 50 to 64 were more likely than respondents under age 35 to report worrying (OR = 1.36, p = .009 and OR = 1.40, p = .001, respectively), while respondents over age 74 were less likely than those under age 35 to report worrying (OR = 0.60, p = .001). Also consistent with the continuous measure, respondents with some college were less likely to report worrying than were respondents without a high school diploma (OR = 0.76, p = .04). The difference was in the same direction but did not reach significance for college graduates compared to those without a high school diploma (OR = 0.79, p = .09). When the interactions between sociodemographic characteristics and survey mode were added to the model in Step 2, none of the interactions approached significance (all ps > .20).

Missing values: refused and “don’t know”

Respondents in the RDD mode were more likely to refuse to answer the question about perceived cancer risk compared to respondents in the mail mode, χ² = 118.29, df = 1, p < .001. Among respondents who provided information on all sociodemographic characteristics, 227 people in the RDD mode either refused to provide an estimate of their perceived cancer risk (23 respondents) or responded “don’t know” (204 respondents). On the other hand, only one person in the mail mode failed to estimate their perceived risk. A similar, although less dramatic, effect was observed for the item concerning cancer worry, χ² = 7.02, df = 1, p = .009. Among respondents providing information on all sociodemographic characteristics, 12 people in the RDD mode failed to provide information on their frequency of cancer worry (3 people refused to answer and 9 responded “don’t know”) whereas no respondents in the mail mode failed to estimate their frequency of worry.

DISCUSSION

This study took advantage of the dual response modes of a nationally representative U.S. survey to examine whether self-reported cancer risk perceptions and worry were subject to mode effects consistent with socially desirable responding. The emergence of mode effects in reports of cancer worry suggests that people may be more comfortable revealing that they worry about cancer via a self-administered survey rather than via a telephone interview, and this effect may be stronger among less educated people. These results are clinically relevant given the difficulties often experienced by healthcare professionals in helping patients disclose their feelings and psychosocial concerns about cancer, (19, 45, 46) a critical step in patient-provider communication. (47, 48) If self-administered questionnaires help facilitate patients’ disclosure of cancer worry, this may be a useful complement to communication training programs designed to help healthcare professionals elicit patients’ concerns. (45, 46)

The mode effects observed here are also relevant to empirical research and theoretical advancement in medical decision making. Measures of cancer worry are often highly positively skewed – with few scores at the high end of the worry scale – which limits researchers’ ability to explore theorized associations between cancer worry and behavior across the full range of worry levels. (14, 31, 41–43). For example, a recent review on the role of cancer worry in cancer screening behavior found “low to moderate levels of cancer worry even among those at high [cancer] risk,” which has “hampered theoretical and empirical advancement” by limiting statistical power at higher levels of worry. (43) Similarly, a meta-analysis concerning breast cancer worry found evidence that “high levels of cancer worry are uncommon” and that cancer worry may not be “generally problematic, even among high-risk women.” (14) Our results suggest the possibility that this skewed distribution of worry may be due, in part, to survey administration effects consistent with social desirability. In turn, it may be helpful for future studies to assess cancer worry using self-administered techniques and to examine worry-behavior associations stratified across different data collection modes.

We did not find survey mode effects for reports of cancer risk perceptions, suggesting that mode effects should not necessarily be added to the many complicating factors in elicitation of perceived risk. (9, 11, 49–51) However, the finding of a mode effect on “don’t know” responding suggests the need for continued research. Some respondents may be reluctant to judge their risk in an interview because they fear giving responses that the interviewer knows to be inaccurate. Alternatively, there may be more time pressure in a telephone interview than in a mail survey. Given that members of underserved populations are more likely to respond “don’t know” to cancer risk perception measures, causing them to be disproportionately excluded from studies of medical decision making, (40) there may be value in using self-administered surveys in order to maximize the probability of capturing responses from these important populations.

Limitations and strengths

This study involved only single measures of perceived cancer risk and worry. Alternative measures may yield different effects. The two survey modes also differed on characteristics aside from degree of social desirability pressures. The RDD mode had a lower response rate than the mail mode (24% vs. 31%) and was unable to sample individuals who lacked landline telephones. Although the data were weighted to adjust for nonresponse bias and underrepresentation, and analyses controlled for sociodemographic factors, follow-up studies are needed in which respondents are randomized to survey mode. Caution should also be taken in generalizing our results to clinical contexts, as people may be more willing to disclose cancer worry to healthcare providers than to anonymous telephone interviewers (e.g., due to greater trust). Research is needed to test for mode effects in clinical contexts and to examine alternative ways of eliciting patients’ concerns such as by avoiding the term “worry” (e.g., “how often are you concerned about the possibility of getting cancer?”).

Strengths of this study include its use of a large, nationally representative sample that allowed for generalizable findings and the assessment of interactions between survey mode and sociodemographic factors. The extensive cognitive testing of the HINTS survey (38) provides assurance that the items were likely to be understood by most respondents. Finally, the frequent use of similar measures in studies of cancer affect and cognition ensures the relevance of the findings to contemporary medical decision making research. (15, 30, 31, 40, 41, 51–53)

Conclusions

There may be benefits of assessing cancer worry using self-administered questionnaires rather than interviews, including potentially receiving more veridical patient reports and obtaining a wider distribution of worry scores in medical decision making research. (43) Using self-administered techniques may also reduce the frequency of “don’t know” responding to cancer risk perception items, which could be helpful in understanding the risk perceptions of members of underserved populations. (40) These findings require replication in clinical contexts and experimental studies that randomly assign participants to response mode.