Don’t know responses to cognitive and affective risk perception measures: Exploring prevalence and socio-demographic moderators

Abstract

Objectives

Many people report uncertainty when appraising their risk of cancer and other diseases, but prior research about the topic has focused solely on cognitive risk perceptions. We investigated uncertainty related to cognitive and affective risk questions. We also explored whether any differences in uncertainty between cognitive and affective questions varied in magnitude by item-specific or socio-demographic characteristics.

Design

Secondary analysis of data collected for a 2×2×3 full-factorial risk communication experiment (N=835) that was embedded within an online survey.

Methods

We investigated the frequency of “don’t know” responses (DKR) to eight perceived risk items that varied according to whether they assessed (a) cognitive vs. affective perceived risk, (b) absolute vs. comparative risk, and (c) colon cancer vs. “any exercise-related diseases”. Socio-demographics were: sex, age, education, family history and numeracy. We analyzed the data using multilevel logistic regression.

Results

The odds of DKR were lower for affective than cognitive perceived risk (OR=0.64, p<.001). This difference occurred for absolute but not comparative risk perceptions (interaction effect, p=.004), but no interactions for disease type or demographic characteristics were found (ps>.05).

Conclusions

Lower uncertainty for affective (versus cognitive) absolute perceived risk items is consistent with research stating: (1) risk perceptions are grounded in people’s feelings about a hazard, and (2) feelings are easier for people to access than facts. Including affective perceived risk items in health behavior surveys may reduce missing data and improve data quality.

Introduction

Risk perception is a central construct in many health behavior theories (Janz & Becker, 1984; Rogers, 1975; Schwarzer, 1999). Ample empirical evidence suggests that people who perceive that they are at higher risk of disease are more likely to take health protective actions (Sheeran, Harris, & Epton, 2013). However, a growing body of research shows that many people report that they don’t know their risk of cancer and other diseases (Hay, Orom, Kiviniemi, & Waters, 2015; Lipkus, Rimer, & Strigo, 1996; Waters, Hay, Orom, Kiviniemi, & Drake, 2013; Waters, Kiviniemi, Orom, & Hay, 2016). This “don’t know responding” (DKR) also seems to be more prevalent in populations affected by health disparities (e.g., those with less formal education) (Hay et al., 2015; Waters et al., 2013; Waters et al., 2016) and among people who have limited health knowledge (Hay et al., 2015). Consistent with the theoretical premise that perceived risk predicts behavior, one recent study reported that people who indicated that they didn’t know their cancer risk were less likely to engage in some cancer prevention and detection behaviors (Waters et al., 2016). This body of research suggests that DKR does not occur at random and may be a meaningful signifier of a need for additional health education and/or behavioral intervention among socio-demographically disadvantaged populations.

However, one limitation with the current DKR literature is that it focuses solely on cognitive risk perceptions and does not address affective risk perceptions. This is important because recent research shows affective risk beliefs (i.e., reported feelings about a risk) are more predictive of intentions and health behavior than cognitive risk beliefs (i.e., reported thoughts about a risk) (Dillard, Ferrer, Ubel, & Fagerlin, 2012; Janssen, van Osch, de Vries, & Lechner, 2010; Janssen, van Osch, de Vries, & Lechner, 2013; Janssen, van Osch, Lechner, Candel, de Vries, 2012; Janssen, Waters, van Osch, Lechner, & de Vries, 2014; Weinstein et al., 2007). Furthermore, early risk perception research (Slovic, Fischhoff, & Lichtenstein, 1980) and modern theoretical perspectives (Cameron, 2003; Finucane, Alhakami, Slovic, & Johnson, 2000; Loewenstein, Weber, Hsee, & Welch, 2001) emphasize the importance of considering emotions, feelings, and other affective aspects of health risk beliefs when predicting health choices and behaviors. Consequently, understanding whether and under what conditions DKR occurs among affectively-oriented risk beliefs could lend preliminary insight into the psychological processes driving DKR.

This paper addresses this gap in the “don’t know” literature by investigating the frequency of DKR for cognitive and affective risk questions. Affect and emotions influence decision making via a variety of mechanisms (for a discussion, see Peters, Västfjäll, Gärling, & Slovic, 2006). Of particular relevance to this paper is its role as information; that is, an affective reaction provides insight about a situation, and those feelings – which are based on prior similar experiences and/or associations – influence risk judgments (Gilovich, Griffin, & Kahneman, 2002; Peters, Västfjäll, Gärling, & Slovic, 2006). This “reliance on affect is a quicker, easier, and more efficient way to navigate in a complex and uncertain world” (Slovic, Peters, Finucane, & MacGregor, 2005) (p. S35). Moreover, research has shown that people have difficulties interpreting and understanding cognitively-oriented risk information (Nelson, Reyna, Fagerlin, Lipkus, & Peters, 2008). Therefore, we hypothesize that DKR will be more common for items assessing cognitive than affective risk perceptions.

Differences in DKR by type of risk appraisal may be exacerbated among populations who 1) have difficulty evaluating cognitive risk information (e.g., people with lower education and people with low numeracy (Bruine de Bruin, Fischhoff, Millstein, & Halpern-Flesher, 2000; Nelson et al., 2008)), 2) use affect more in the decision making process (e.g., women have been found to be more affectively oriented than men (Booth-Butterfield & Booth-Butterfield, 1990; Maio & Esses, 2001)), and 3) have developed more accessible affective reactions (e.g., people who have closely experienced cancer are likely to have developed more accessible affective reactions to the disease compared to people who lack this experience (Peters, McCaul, Stefanek, & Nelson, 2006)). To explore this further, we investigated whether the effect of a cognitive vs. affective risk appraisal on DKR was moderated by these socio-demographic characteristics (i.e. education, numeracy, gender, family history).

We also explored the moderating impact of two risk perception item characteristics. One item characteristic was whether it assessed absolute or comparative risk perceptions. We identified this characteristic as important because people interpret these questions in different ways (French & Marteau, 2008; Helweg-Larsen & Shepperd, 2001) and they may be differentially associated with affect (French & Marteau, 2008; French & Hevey, 2008; Helweg-Larsen & Shepperd, 2001). The second item characteristic examined the moderating impact of the disease that is the topic of the risk perception question (e.g. a dreaded disease like cancer elicits strong negative affect (Peters et al., 2006)).

To the best of our knowledge, no other studies have investigated DKR related to affective risk perceptions. Comparing DKR towards cognitive and affective risk questions could have important value for understanding the psychological processes driving DK responding (Waters et al., 2013). It may also provide preliminary insight into why risk feelings are more strongly related to health decision making than are risk cognitions. This is important because little research has examined possible reasons for these effects. Furthermore, elucidating these questions will help researchers clarify the choices they must make when assessing risk perceptions in situations where time and survey space are limited (e.g. in clinical and community settings) since, for example, choosing the items resulting in less DKR (and consequently fewer missing data points) might be beneficial.

To summarize, the aim of the present study was twofold. First, to investigate the frequency of don’t know responding to cognitive and affective risk questions. Second, to investigate the moderating impact of sociodemographic characteristics (i.e. sex, age, education, family history, and numeracy) and risk perception item characteristics (i.e. absolute vs. comparative risk and disease type).

Methods

This paper describes a secondary analysis of data collected for the purpose of investigating the separate and combined effects of three different risk communication strategies (BLINDED, under review). That study explored whether combining recommended risk communication strategies when presenting information about the effect of physical inactivity on the risk of multiple diseases improves key cognitive or affective precursors of health behavior change. In that study, risk ladders were used to simultaneously illustrate the hypothetical risk of four diseases: heart disease, stroke, diabetes, and colon cancer.

Participants

The survey was conducted between November 11, 2015 and December 7, 2015 using GfK KnowledgePanel^®, a probability-based Internet panel designed to be representative of the U.S. population. GfK randomly selected a subsample of individuals from its English language database who met the age requirement for the original risk communication experiment (30–65 years old) and emailed them an invitation to participate. Those who agreed completed an online eligibility screener assessing their physical activity. Respondents who did not meet current guidelines for aerobic activity (i.e. at least 150 minutes per week of moderate intensity aerobic physical activity) were invited to complete the survey. Recruitment was stratified so that at least 50% of the sample had no more than vocational-technical training and at least 50% were racial/ethnic minorities.

A total of 1161 participants completed the questionnaire (BLINDED, under review). The dataset for this study included people who completed all items needed for the analyses and who completed the questionnaire within an acceptable time frame (i.e. between the 3^rd and 97^th percentiles, or 7.00–7144.34 minutes) resulting in a final dataset of 835 participants. Participants were on average 48 years old (SD=10.22) and 57% of the respondents were female. Almost half of the sample was nonwhite (47%) and had no college training (46%). The average numeracy score was 1.3 (SD=0.76; range=0–2) and the great majority (87%) reported a family history of exercise-related diseases. The median completion time was 21 minutes.

Procedure and Measures

After consenting, participants provided information about their physical activity behavior. Participants were then randomly assigned to view one of 12 experimental conditions that varied the strategies used to communicate the risks of heart disease, stroke, diabetes, and colon cancer (BLINDED, under review) in a 3 (social comparison information: absent vs. above average vs. much above average) × 2 (risk reduction information: present vs. absent) × 2 (numerical format: words vs words plus numbers) factorial design. Next, they completed survey items assessing absolute and comparative cognitive perceived likelihood (National Cancer Institute, 2009) and absolute and comparative affective perceived likelihood (adapted from Janssen et al., 2010). To reduce burden, these items were asked only for colon cancer and for “any exercise-related diseases” (including colon cancer) instead of all four diseases separately. All eight perceived likelihood items included an explicit DK response option. Participants then completed questions about the following socio-demographics: sex, age, education, racial background, family history of disease, and numeracy (Lipkus, Samsa, & Rimer, 2001; National Cancer Institute, 2014). See Table 1 for wording of items.

Table 1.

Description of the measures

Don’t know responding
Cognitive risk uncertainty	How likely do you think it is that you will get [colon cancer/sick from ANY of the diseases shown in the picture in the next 10 years], if you do not get regular physical activity? [1=not likely; 4=very likely; I don’t know];
	Compared to other people your age and sex, how likely do you think it is that you will get [colon cancer/sick from ANY of the diseases shown in the picture] in the next 10 years, if you do not get regular physical activity? [1=much less likely; 5=much more likely; I don’t know].
	Four dichotomous variables were created indicating whether people responded “I don’t know” (i.e. perceived risk uncertainty).
Affective risk uncertainty	How much do you agree with the following statement: "I feel like I could easily get [colon cancer/sick from ANY of the diseases shown in the picture] in the next 10 years if I do not get regular physical activity [1=do not agree; 4=strongly agree; I don’t know];
	Compared to other people your age and sex, how easily do you feel you could get [colon cancer/sick from ANY of the diseases shown in the picture] in the next 10 years if you do not get regular physical activity? [1=much less easily; 5=much more easily; I don’t know].
	Four dichotomous variables were created indicating whether people responded “I don’t know” (i.e. perceived risk uncertainty).
Socio-demographic factors1
Numeracy	Imagine that we flip a fair coin 1,000 times. What is your best guess about how many times the coin would come up heads in 1,000 flips? _____ times;
	Imagine that the chance of getting a disease is 1%. If there were 1,000 people, about how many would be expected to get the disease? _____ people.
	All correct answers were summed for a possible range of 0–2.
Family history	Have any of your family members or friends ever had [diabetes, heart disease, stroke, breast cancer, colon cancer, other cancer]? [1=no; 2=yes).
	A dichotomous variable was created indicating whether family of friends had a history of at least 1 chronic disease.

¹Information about gender, age, educational level [1=vocational technical training or less; 2=education beyond vocational technical training], and racial-background [1=non-white; 2=white] were provided by GfK.

Other psychosocial variables that were assessed included information seeking, knowledge about physical activity-related diseases, interest and intentions in engaging in physical activity, message comprehension, message acceptance, response-efficacy, worry, severity, self-efficacy, anticipated regret, attitudes about physical activity, present/future time orientation, and defensive processing. These questions were not included in the present analyses because they were outside the scope of the research question. The full questionnaire can be obtained from the corresponding author.

Statistical analysis

Perceived risk uncertainty was created by dichotomizing each perceived likelihood item to indicate whether participants responded “I don’t know” or provided a valid response (e.g., not likely, a little likely, somewhat likely, very likely) (Hay et al., 2015; National Cancer Institute, 2014; Waters et al., 2013). A multilevel logistic data analytic approach was adopted with eight items (level 1 or item level, n=6680) nested within participants (level 2 or person level, N=835). To each multilevel model, we added a random intercept to take possible person-level dependencies in DKR into account. Item characteristics were entered as fixed effect predictors at level-1 of the multilevel models¹ whereas person characteristics were entered as level-2 predictors. Predictors were not centered. Both regression weights (expressing predicted change in log odds with a one unit increase in the predictor) and (95% confidence intervals of) odds ratios are reported.

First, we predicted DKR (1=don’t know, 0=any other response) by risk appraisal type (1=affective, 0=cognitive). Second, we predicted DKR by the other items characteristics (i.e. absolute vs. comparative, colon cancer vs. other diseases) and person characteristics (i.e., gender, age, education, numeracy, and family history) in separate multilevel models to explore possible main effects of these characteristics. Finally, we examined whether the effect of risk appraisal type on DKR was moderated by any of these item or person characteristics by adding interaction terms to the multilevel models.

Since experimental condition did not have an influence on DKR, experimental condition was not controlled for in the analyses. Multilevel analyses were conducted in SAS version 9.4 and results were considered to be significance when p<.05.

Results

Risk appraisal type as predictor of DKR (main effects)

Risk appraisal type was a significant predictor of DKR (see Table 2). The odds of DKR were lower for affective risk appraisals compared to cognitive risk appraisals (B=−0.45, t(6678)=−5.31, p<.001, OR=0.64, 95% CI of OR =0.54–0.75).

Table 2.

DKR by levels of risk appraisal type, other item-level characteristics and demographic-level characteristics (main effects).

Predictor	DKR (%)	P
Risk appraisal type
Cognitive	17.3	<.001
Affective	13.4
Item-level characteristics
Absolute vs. comparative		.28
Absolute	15.8
Comparative	15.0
Disease type		<.001
Colon cancer	18.8
Any of the diseases	11.9
Demographic-level characteristics
Sex		.59
Men	14.6
Women	15.9
Education		<.001
Vocational-technical training or less	19.2
More than vocational-technical training	12.1
Numeracy		<.001
0	20.9
1	17.7
2	11.6
Family history		.11
No	20.0
Yes	14.6
Age	---	.77

Item and socio-demographic characteristics as predictor of DKR (main effects)

Disease type was the only item characteristic that significantly predicted DKR (see Table 2). DKR was higher for items assessing risk perception of colon cancer compared to other diseases (B=−0.83, t(6678)=−9.57, p<.001, OR=0.44, 95% CI of OR=0.37–0.52). Regarding participant characteristics, only education and numeracy were predictive of DKR (see Table 2). Higher levels of education (B=−0.69, t(6678)=−4.20, p<.001, OR=0.50, 95% CI of OR=0.37–0.69) and numeracy (B=−0.43, t(6678)= −4.10, p<.001, OR=0.65, 95% CI of OR=0.53–0.80) were both associated with lower levels of DKR. Absolute vs. comparative risk, sex, age, and family history did not significantly predict DKR (all ps>.05).

Item and socio-demographic characteristics moderating the impact of risk appraisal type on DKR (interaction effects)

Regarding item characteristics, the effect of risk appraisal on DKR was moderated by absolute vs. comparative risk (B=0.49, t(6676)=2.92, p=.004). Follow-up simple slope analyses revealed that lower levels of DKR for affective risk appraisal (vs. cognitive risk appraisal) occurred only for absolute risk items (B=−0.69, t(6676)=−5.80, p<.001, OR=0.50, 95% CI of OR=0.40–0.63) but not for comparative risk items (B=−0.20, t(6676) = −1.67, p=.10, OR=0.82, 95% CI of OR=0.65–1.04). No interaction between risk appraisal type and disease type was found (p=0.08). No evidence for moderation was found for any of the socio-demographic characteristics (all ps>.05).

Discussion

We found support for our hypothesis that DKR will be more common for items assessing cognitive risk perceptions compared to affective risk perceptions. This is in line with research showing the difficulties people have with interpreting and understanding cognitively oriented risk information (Hoffrage, Lindsay, Hertwig, & Gigerenzer, 2000; Lipkus et al., 2001; Nelson et al., 2008). The difference in DKR between cognitive and affective risk might be related to the fact that affective processing of risk is hypothesized to be easier and faster (Epstein, Lipson, Holstein, & Huh, 1992; Loewenstein et al., 2001; Slovic et al., 2005) and found to be more accessible (Verplanken, Hofstee, & Janssen, 1998) than cognitive processing of risk. Future research should examine the extent to which affective processing and accessibility drives the difference in cognitive and affective DKR in research specifically designed for that purpose.

Although previous research has shown that cognitive and affective risk items load on different factors (Ferrer, Klein, Persoskie, Avishai-Yitshak, & Sheeran, 2016; Janssen et al., 2012) and are more strongly associated with other affective constructs (e.g., worry; Janssen et al., 2012) suggesting conceptual distinctiveness, it might be interesting to investigate if affective risk questions indeed reflect affective processes using objective measures (e.g., fMRI technology) and are not merely different from the cognitive questions due to specificity differences (i.e. cognitive questions require more specific and objective information). This will inform interventions that help people understand and use cognitive aspects of risk perceptions that are necessary for engaging in shared medical decision making (e.g., by weighing the probabilities of a medication’s risks and benefits (Zikmund-Fisher, 2013). For future research it might also be interesting to investigate if excluding a DK response option may prompt respondents to choose the option closest to their intuitive or gut reactions (Waters et al., 2013). Moreover, since affective risk perceptions have been found to be more strongly associated with health behavior and behavioral intentions, it would be interesting to investigate if DKR towards affective risk perceptions is more or less strongly associated with health behavior compared to cognitive risk perceptions.

The moderation analyses showed that the differential effect between affective and cognitive risk perceptions was only found for absolute risk and not for comparative risk of disease. Additional research is needed to better understand these contextual differences to advance theory and the development of more effective communication practices that better serve the information needs of the general public. It could be that comparative risk taps more into the affective risk component such as that comparative risk information informs the affective risk judgments, resulting in less DKR differences between cognitive and affective comparative risk. It has been suggested that social comparison information might be more affective than objective information (Fagerlin, Zikmund-Fisher, & Ubel, 2007; Klein, 2003; Schwartz, 2009). However, there is also evidence showing absolute risk to be more strongly associated with affect compared to comparative risk (French & Marteau, 2008). Another explanation could be that comparative risk judgments reflect a biased process as has been found in previous research about unrealistic optimism (Kruger, 1999; Shepperd, Klein, Waters, & Weinstein, 2013; Shepperd, Waters, Weinstein, & Klein, 2015) that operates the same for both cognitive and affective risk questions. Replication of these results and an in-depth examination of the underlying processes explaining this difference are required.

No other moderating factors were found. The difference in DKR between cognitive and affective risk perceptions was similar in magnitude for colon cancer and for other diseases related to physical activity. Furthermore, the effect was consistent in magnitude across many demographic characteristics. Perhaps these demographic factors have a stronger influence on affective risk levels (i.e. correlate more strongly with affective risk compared to cognitive risk) or on the relationship between affective risk and health behavior (i.e. certain groups may rely more on their feelings), but do not impact whether people are less uncertain about their feelings compared to their risk cognitions. Future research should examine this further.

The present study is in line with previous research showing that DKR is more prevalent in populations affected by health disparities (i.e. low education and low numeracy) (Waters et al., 2013). The results further showed that DKR is more prevalent for risk estimates concerning colon cancer compared to any diseases associated with physical inactivity (i.e. colon cancer, diabetes, stroke, and heart disease). This is in accordance with research showing that knowledge and risk perceptions of behavioral risk factors for cancer are generally low, especially among relatively new risk factors that have received little attention in risk communication practices yet, such as physical activity (Breslow, Sorkin, Frey, & Kessler, 1997; van Kann, Janssen, van Osch, & de Vries, 2010; Wardle, Waller, Brunswick, & Jarvis, 2001). Therefore, even if people did not know their risk for colon cancer, they could answer the question about any diseases based on their risk estimate for the other three diseases. For future research it would be interesting to investigate if actual risk status is associated with DKR in general and with differences in DKR between cognitive and affective risk questions.

The differences in don’t know responding between cognitive and affect risk questions are in line with recent theoretical perspectives suggesting that beliefs about health risks consists of cognitive and affective risk components (e.g., the TRIRISK model (Ferrer et al., 2016), the self-regulation model of illness cognition and behavior (Cameron, 2003), risk as feelings framework (Loewenstein et al., 2001), the affect heuristic (Finucane et al., 2000)) and it adds to the understanding of why risk feelings are more strongly related to health decision making than are risk cognitions (Dillard et al., 2012; Janssen et al., 2010; Janssen et al., 2013; Janssen et al., 2012; Janssen et al., 2014; Weinstein et al., 2007). Specifically, risk feelings might reflect a more natural and accessible way of how people think of probabilities compared to cognitive probability judgments. For this reason, in situations where time and survey space is limited, the inclusion of affective risk perception items might be preferable to cognitive risk perception items. Moreover, the effect adds to the evidence that DK responses are not random (e.g. due to a lack of motivation or study fatigue). Other explanations might be more plausible such as that a subset of the population may not be able to appraise their risk with certainty due to limited knowledge as has been shown in previous research (Hay et al., 2015). This might be especially the case for absolute cognitive risk estimates resulting in higher DKR compared to affective risk. Cognitive risk questions might be more likely to elicit feelings of embarrassment about providing a wrong answer due to the perception that their knowledge does not meet the accuracy requirements for that question. For items assessing feelings of risk, this concern might be less salient as there is no real “right or wrong” answer about how someone feels. For future research it would be interesting to investigate whether lack of knowledge is more strongly associated with cognitive DKR than with affective DKR.

Limitations

The use of a large national sample including a large proportion of people from underserved populations allows generalizability to population segments that have had relatively limited representation in traditional risk perception research. Furthermore, the inclusion of affective risk extends previous DK research from solely focusing on cognitive risk. Nevertheless, a few limitations should be considered. First, cognitive and affective risk perceptions were measured with single items. Although multi-item measurement might improve the reliability of these constructs (but see Weinstein et al., 2007), experimental evidence suggests that longer questionnaires reduce data quality (Galesic & Bosnjak, 2009). Therefore, we limited the number of survey questions by adapting single measures used in nationally representative surveys and in published empirical research. Moreover, the response scales for the perceived absolute and comparative risk items were not the same (i.e. 4 point scales were used for absolute questions and 5 point scales for comparative questions). As it might be difficult for people to switch between scales especially for similar constructs, using similar scales may benefit the reliability of these constructs. In addition, to reduce participant burden, the different perceived risk questions (i.e. absolute vs. comparative and cognitive vs. affective) were asked only for colon cancer and “any of the diseases shown in the picture” instead of all four diseases separately. The use of non-specific risk questions is uncommon in the risk perception literature and may limit comparability with other DKR studies. Finally, the cognitive and affective risk questions were measured in a fixed order. For future research it might be important to counterbalance these questions to prevent question order effects.

Conclusions

Participants responded “don’t know” less often when asked to indicate their affective (versus cognitive) absolute perceived risk. This effect was consistent in magnitude across many socio-demographic characteristics. In health behavior research situations where time and survey space is limited, including affective risk perception items might be preferable to cognitive risk perception items to reduce missing data points.