Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jul 1.
Published in final edited form as: Psychol Health. 2020 Sep 2;36(7):862–878. doi: 10.1080/08870446.2020.1788714

Examining strategies for addressing high levels of “I don’t know” responding to risk perception questions for colorectal cancer and diabetes: An experimental investigation

Jennifer L Hay 1, Elizabeth Schofield 1, Marc Kiviniemi 2, Erika A Waters 3, Xuewei Chen 4, Kimberly Kaphingst 5, Yuelin Li 1, Heather Orom 4
PMCID: PMC7952023  NIHMSID: NIHMS1660166  PMID: 32876479

Abstract

Objective:

Many people say they “don’t know” their risk for common diseases (DK responders). Inadequate health literacy and higher disease information avoidance may suppress risk knowledge and thereby increase DK responding. Study goals were to examine two plausible interventions to address the health education needs of DK responders.

Design:

Participants were identified in a pre-screener as DK responders for either diabetes or colorectal cancer (CRC) risk perception questions (N=1276; 35% non-white; 49% inadequate health literacy). They were randomly assigned to read either standard or low literacy risk information about diabetes or CRC, and to undergo a self-affirmation intervention or not.

Main outcome measure:

DK responding following reading the risk information.

Results:

Neither intervention reduced DK responding. Multivariable analyses showed that health literacy, information avoidance, and believing the disease is unpredictable – but not risk factor knowledge and need for cognition – best predicted participants’ conversion from a DK response to a non-DK scale point response.

Conclusion:

Results confirm that both inadequate health literacy and higher information avoidance are associated with DK responding. DK responders are also disproportionately underserved and less adherent to health behaviors. Because galvanizing risk perceptions are central to public health, addressing their information needs is a priority.

Keywords: risk perceptions, uncertainty, health literacy, health information avoidance

Introduction

Change in people’s perceptions of disease risk through provision of health risk information is a central goal of public health initiatives, health behavior interventions, and health care provider communication. Increasing risk perceptions when they are too low or improving risk perception accuracy can improve engagement in health behaviors, and thus reduce actual disease risks, particularly when done in conjunction with addressing other critical motivating beliefs such as self-efficacy and response-efficacy. This principle is supported by several health behavior theories (Conner & Norman, 2005) and decades of empirical work confirming that risk perceptions for disease consistently motivate health protective and risk reducing behaviors (Sheeran, Harris, & Epton, 2014; Waters, McQueen, & Cameron, 2014).

However, changing risk perceptions is quite difficult. Educational interventions (e.g., (Weinstein & Klein, 1995)) and even genetic testing feedback (Aspinwall, Taber, Kohlmann, Leaf, & Leachman, 2014) have not necessarily shifted risk perceptions in expected directions, with individuals showing no change, or only short-term changes, in their perceptions of risk over time. Additionally, a significant proportion of the general population reports that they “don’t know” their risk for common diseases such as diabetes or colorectal cancer (CRC). Between 5% and 10% of the population chooses this response even when an explicit “don’t know” (DK) response is not provided to them (Waters, Hay, Orom, Kiviniemi, & Drake, 2013; Waters, Kiviniemi, Orom, & Hay, 2016), and the proportion is several times higher and in some cases exceeds 50% when an explicit DK response option is available (Orom et al., 2018; Waters et al., 2013). As such, the goal of changing risk perceptions may be made even more complex by the fact that a significant proportion of the population does not start with a risk perception at all.

The preponderance of risk perception research has simply eliminated DK responses from analyses. Yet, individuals who report DK responses tend to be more poorly informed about risk, have less formal education, have inadequate health literacy (Hay, Orom, Kiviniemi, & Waters, 2015; Orom et al., 2018; Waters et al., 2013), and to be less adherent to some health protective behaviors (Waters et al., 2016). As such, efforts to address the underlying causes of DK responses could help potentially vulnerable populations improve their risk awareness and motivate them to engage in health-protective behaviors such as physical activity, dietary improvement, sun protection, and appropriate cancer screening.

Empirically-supported explanations for DK responses provide guidance concerning how DK responses may be changed. Orom and colleagues (Orom et al., 2018) have provided evidence for two primary explanations for DK responding in the context of diabetes and CRC risk perceptions. Specifically, people with inadequate health literacy and people with a tendency to avoid disease risk information have lower risk factor knowledge and, consequently, have higher rates of DK responses. While a common explanation for “don’t know” responses for survey questions in general involves respondents expending low cognitive effort (Krosnich, 1991; Krosnick et al., 2002), Orom and colleagues (Orom et al., 2018) did not find evidence supporting this explanation in the context of risk perception questions, see also (Kiviniemi, Ellis, Orom, Waters, & Hay, 2020).

This prior work (Orom et al., 2018) suggests two intervention approaches for addressing lack of risk knowledge for DK responders - targeting messages for audiences with inadequate health literacy, and disrupting defensive avoidance of threatening health information. In the current study, we target these two pathways by experimentally testing corresponding intervention strategies to reduce DK responding. Reductions in DK responding correspond to the increased use of one of the item response options (e.g., not at all likely, somewhat likely, very likely; hereafter as scale point responses). First, to address the fact that individuals with inadequate health literacy have lower risk knowledge and thus higher DK responses, we tested intervention messages developed to reduce literacy demands (Berkman et al., 2011); the control condition involved standard risk information regarding risk factors for diabetes and CRC that can be obtained in the public domain (National Institute of Diabetes and Digestive and Kidney Diseases, 2016; Prevent Cancer Foundation, 2013). Second, to address the fact that individuals with higher disease information avoidance have lower risk knowledge and higher DK responses, we tested a personal self-affirmation intervention. Self-affirmation is defined as the act of affirming one’s moral and adaptive adequacy (Steele, 1988). By enhancing the salience of personal strengths and values, self-affirmation interventions (Steele, 1988) can reduce information avoidance and other forms of defensive processing (Howell & Shepperd, 2012; Jessop, Simmonds, & Sparks, 2009; McQueen & Klein, 2006; Napper, Harris, & Epton, 2008; Sherman et al., 2009; van Koningsbruggen & Das, 2009). If either or both experimental interventions alter DK responses in the hypothesized direction, it will also be a strong indication that the investigated factors are causal mechanisms that drive DK responding to disease risk perception questions.

We pose two hypotheses and an exploratory research question. 1) Among people with high disease information avoidance, engaging in a self-affirmation intervention (versus not engaging in a self-affirmation intervention) will result in greater scale point responding. 2) Among people with inadequate health literacy, presenting risk information in the form of a low literacy message (versus a standard literacy message) will result in greater conversion to scale point responding. We also wanted to obtain additional empirical evidence for the causal mechanisms that result in DK responses to risk perception questions. Therefore, we ask the exploratory research question, “what are the associations between demographics, disease information avoidance, health literacy, disease-specific risk factor knowledge, beliefs about the unpredictability of disease risk, and need for cognition and converting to scale point responses?”

We investigate these questions in the context of diabetes and CRC risk perceptions. We chose these two health contexts given that diabetes and CRC are exceedingly common in U.S. men and women (American Cancer Society, 2019; Gregg et al., 2014) and amenable to behavioral risk reduction (American Diabetes Association, 2018; Tuomilehto et al., 2001; Whitlock, Lin, Liles, Beil, & Fu, 2008).

Methods

Participants

Participant recruitment and data collection for this online survey study was conducted by GfK, a market research firm. (The GfK survey panel has since been acquired by Ipsos). Individuals were eligible for participation if they were age 18 or older and fluent in English, indicated that they did not know their risk of either diabetes or CRC, and reported that they had no personal history of either diabetes or CRC. To ensure adequate representation of individuals from across the spectrums of health literacy and disease information avoidance, we worked to recruit equally into four blocks for each disease. For diabetes, 25% of participants had low avoidance and low health literacy, 20% had high avoidance and low health literacy, 28% had low avoidance and high health literacy, and 27% had high avoidance and high health literacy. For colorectal cancer the percentages were 28%, 27%, 23% and 23%, respectively. Data were collected between September 2017 and January 2018. Study procedures were approved by the University at Buffalo Institutional Review Board. Participants were consented as part of GfK’s umbrella IRB-approved consent process.

GfK used an equal probability selection method to recruit participants from among members of its standing Internet survey panel (KnowledgePanel). The panel is designed to be nationally representative; it is comprised of 55,000 individuals who are selected by address-based probability sampling based on the most recent Delivery Sequence File of the United States Postal Service, which avoids non-coverage issues associated with telephone-based sampling. Panel participants are compensated by either receiving points that can be redeemed for prizes and awards, or by receiving free Internet access and/or a web-enabled device if they do not already have access or a device. Typical earnings are equivalent to $4-$6 per month. To boost participation toward the end of our study, non-responders were offered additional incentives; 25 participants received $5 and 41 received $10.

Of 22,522 participants who GfK invited to complete an eligibility screener, 14,545 (64.58%) did so. Among participants completing the screener, 1,588 were eligible to take part in the study, satisfied our recruitment stratification criteria, and were invited to take part in the main study. Of these, 1,313 (82.7%) agreed to participate in the study.

To maintain data quality, GfK excluded 37 (2.8%) cases because they met two or more criteria for extreme low quality responding. The goal was to drop participants who were likely responding in nonsensical ways and whose responses could obscure patterns in the data. Data were excluded for participants who 1) completed the survey in 3.5 min or less (1/4 median completion time of 14 min), 2) answered attention checks incorrectly (e.g., “Survey validation item, please select ‘Somewhat agree’ ”), 3) straight-lined or selected the same response for at least half of the question grids, or 4) inconsistently answered a repeated question about their insurance status (Curran, 2016). We were conservative in dropping participant data; the modal proportion of responders with low enough quality responding to impact study findings is reported to be 8–12% (Curran, 2016).

Study Design

We conducted a 2 × 2 × 2 × 2 experiment with two individual difference variables and two experimentally manipulated variables: (assessed disease information avoidance: high vs low) × (assessed health literacy: adequate vs inadequate) × (manipulated self-affirmation condition: self-affirmation vs. control) × (manipulated literacy message condition: written for people with low literacy vs standard message). To balance the study design cells, we first selected participants based on their screener scores for health literacy and information avoidance to fill the four cells for each combination of participant health literacy and avoidance. Then, from within each of these four cells, participants were randomly assigned to the four conditions for the two experimental interventions (i.e., the self-affirmation or literacy message interventions).

Study intervention conditions

Self-affirmation versus control condition. A number of different self-affirmation interventions reduce disease information avoidance and other forms of defensive processing (Howell & Shepperd, 2012; Jessop et al., 2009; McQueen & Klein, 2006; Napper et al., 2008; Sherman et al., 2009; van Koningsbruggen & Das, 2009). We selected an intervention (Napper, Harris, & Epton, 2009) that could be implemented via an online survey. Participants who were randomized to the self-affirmation condition were asked to write a brief essay about a value that is important to them about themselves (self-affirmation) or a value that is not important to themselves but could be important for someone else (control condition matched for time and attention). This standard value affirmation task (McQueen & Klein, 2006) has been successfully used in large GfK samples previously (Ferrer, Klein, & Graff, 2017). Participants were exposed to the self-affirmation intervention prior to viewing a risk-based message.

Standard literacy versus low literacy message. Participants randomized to the standard literacy condition viewed a widely disseminated risk-based message for either diabetes (National Institute of Diabetes and Digestive and Kidney Diseases, 2016) or CRC (National Cancer Institute, 2019a, 2019b) that contains information about risk factors and prevention for the relevant disease. Participants randomized to the low literacy condition viewed a risk-based message with the same content as the standard message but was adapted for a lower health literacy audience. To develop the low literacy message, we adapted versions of the standard messages that follow best practices for low health literacy information such as short sentences, plain language, and the use of visuals (Doak, Doak, & Root, 1996; Kaphingst et al., 2012). Assessed with the Fry Graph readability formula, readability grade level for the standard and low literacy CRC messages were 11th and 6th grade, respectively, and 16th and 8th grade for the diabetes messages.

Pilot testing messages. The low health literacy and self-affirmation messages underwent pilot testing. Participants were seven Mechanical Turk (MTurk) workers (all age ≥18, <high school educational attainment, English fluent). They were asked open-ended questions to assess understanding and perceptions of the low literacy and self-affirmation interventions (e.g., comprehension, credibility, relevance). Pilot test participants received $2 for their time and participation. They unanimously described the messages as highly understandable. In response to participant feedback we changed some of the graphic elements to be more explanatory or appealing.

Procedure

KnowledgePanel members were screened to determine eligibility. An average of 13.5 days later (to avoid demand characteristic effects), individuals who were deemed eligible and had the health literacy/disease information avoidance characteristics necessary for recruitment stratification were invited to participate in the main study. Experimental intervention and assessments were administered in the following order: self-affirmation intervention or control intervention, standard literacy or low health literacy message, assessment of risk perception (primary outcome), assessment of self-concept (self-affirmation manipulation check), assessment of risk factor knowledge, assessment of unpredictability beliefs, and assessment of need for cognition. Participants received a message regarding only diabetes or CRC, not both. Assignment to health issue was based on whether they had provided a DK response regarding diabetes or CRC risk perceptions. Those who answered DK to both diabetes and CRC risk perceptions were randomly assigned a health issue.

Measures

Eligibility Screen.

Individuals who did not know their disease risk were identified using an absolute perceived risk item from the National Cancer Institute’s Health Information National Trends Survey (Nelson et al., 2004), modified to include a don’t know response option. Response options were on a 4-point Likert-type scale: “not at all likely,” “somewhat likely,” “fairly likely,” and “very likely,” plus a fifth “don’t know” option. For each question, we counterbalanced the order of the response options so that half of respondents saw the DK option first and half saw it last, as in our pilot work we found that people are more likely to select DK if it is presented first rather than last. A DK responding variable was created for each risk question by dichotomizing responses into either any scale point response or a DK response.

Disease Information Avoidance.

During the eligibility screen, information avoidance was separately assessed for each disease with 7 items from Howell and Shepperd’s information avoidance scale (Howell & Shepperd, 2016) adapted to enquire about avoiding information about diabetes (α= .84) or about CRC (α= .86). Participants rated the extent to which they agreed with items such as, “I would rather not know about diabetes/CRC” (1 = strongly disagree to 4 = strongly agree). Mean scores for each disease were calculated for each participant. To allow for our blocked randomization design, we classified individuals with scores greater or equal to the median as having high avoidance for research question 2 and treated the measure continuously otherwise. There is no empirical evidence establishing whether information avoidance is best characterized as by a continuous or categorical scale (Miles, Voorwinden, Chapman, & Wardle, 2008).

Health Literacy.

Health literacy was also assessed during the eligibility screen. We used standard cut-off scores for inadequate (≤3, combining both the two lower categories (“high likelihood of limited literacy” and “possibility of limited literacy”) and adequate literacy (>3) for our measure of health literacy, the Newest Vital Sign (Weiss et al., 2005).

Primary outcome measure.

After delivery of the intervention, we assessed DK responding on a perceived absolute risk question with an explicit don’t know option. The outcome of interest was whether participants continue to respond don’t know after exposure to the intervention or do they now respond by using one of the scale point responses for the risk item (“not at all likely,” “somewhat likely,” “very likely” and “very likely”).

Self-Concept.

We assessed self-concept with five items that assess salience of positive aspects of self and values (Napper et al., 2009) as a manipulation check in order to test whether positive aspects of self-concept were more salient among those who received the self-affirmation intervention compared to those in the control condition.

Risk knowledge.

We assessed risk factor knowledge by asking participants to indicate whether each of 5 risk factors increased/lowered/made no difference to risk for CRC or diabetes; knowledge was scored as the total number of correct responses for test items (possible range = 0–5). Risk factors for CRC were older age, eating a lot of red meat, regular physical activity, smoking, and having a blood relative with CRC (Center for Disease Control and Prevention (CDC), 2017a). Risk factors for diabetes were having high blood pressure, regular physical activity, having a blood relative with diabetes, older age, and staying a healthy weight (Center for Disease Control and Prevention (CDC), 2017b). Alpha coefficients were good for both assessments (α = 0.76 for diabetes and 0.78 for CRC). Answers to all risk factor knowledge questions were addressed in the risk messages.

Unpredictability beliefs.

We used a 3-item measure to assess participants’ belief that developing diabetes/CRC is unpredictable (Hay et al., 2014). Each item (e.g., “Anybody can get diabetes, no matter what they do”) had a four-point Likert response (1 = strongly disagree to 4 = strongly agree). The mean of the 3 items scaled up to a 0–100 scale (Baser, Li, Brennessel, Kemeny, & Hay, 2017) served as the measure of unpredictability (α = 0.80).

Need for Cognition.

We used the Need for Cognition Scale short form (7 items, α=.80) (Cacioppo, Petty, & Kao, 1984). An example item includes “Thinking is not my idea of fun.” Responses were on a four-point Likert scale from 1) strongly agree – (4) strongly disagree and were summed.

Demographics.

We used demographic characteristics assessed annually by GfK. These included sex, age, race/ethnicity, annual household income, education, employment status, and marital status. We also assessed family history of diabetes and CRC.

Statistical Approach

First, we describe participant baseline characteristics and compare them across the two survey groups (diabetes and CRC) using independent samples t-tests for continuous and Chi-square tests for categorical variables. Next, we conducted a manipulation check for the self-affirmation manipulation separately for each survey group. For participants who were avoidant with respect to the survey disease at baseline, we assess the effect of affirmation by examining self-concept at post-intervention using independent sample t-tests. We then examined the effect of the experimental interventions on conversion from a DK response to a scale point response in each survey using logistic regression. Finally, we examine demographics, avoidance, knowledge, unpredictability beliefs, and need for cognition as potential predictors of conversion from a DK response to scale point response in each survey, using a series of unadjusted logistic regression models, and significant predictors in these models were added to the adjusted models. In these logistic regression models assessing associations to conversion to a scale point responding, survey weights developed by GfK were applied, which make each of the surveys representative of the US population. All analyses were conducted in SAS software version 9.4.

Results

Descriptive findings

Seven hundred and nine participants were assigned to the diabetes study arm, and 567 were assigned to the CRC study arm. More than half of participants in each survey (67% in the diabetes survey; 61% in the CRC survey) converted from a don’t know response to a scale point response for the applicable disease following receipt of the risk information. Participant characteristics overall and by survey are shown in Table 1.

Table 1.

Participant Characteristics by Survey Group

Characteristic All n (%) Diabetes n (%) CRC n (%) Characteristic All n (%) Diabetes n (%) CRC n (%)
All 1276 709 567 Employment Status*
Sex***  Employed 598 (47%) 350 (49%) 248 (44%)
 Male 604 (47%) 365 (51%) 239 (42%)  Self-employed 85 (7%) 54 (8%) 31 (5%)
 Female 672 (53%) 344 (49%) 328 (58%)  Retired 362 (28%) 191 (27%) 171 (30%)
Age  Unemployed 231 (18%) 114 (16%) 117 (21%)
 18 – 24 73 (6%) 39 (6%) 34 (6%) Marital Status**
 25 – 34 199 (16%) 105 (15%) 94 (17%)  Married/ Partnered 701 (55%) 417 (59%) 284 (50%)
 35 – 44 177 (14%) 98 (14%) 79 (14%)  Single 277 (22%) 151 (21%) 126 (22%)
 45 – 54 195 (15%) 116 (16%) 79 (14%)  Divorced/ Separated 198 (16%) 96 (14%) 102 (18%)
 55 – 64 268 (21%) 156 (22%) 112 (20%)  Widowed 100 (8%) 45 (6%) 55 (10%)
 65–74 234 (18%) 130 (18%) 104 (18%) Family History***
 75+ 130 (10%) 65 (9%) 65 (11%)  Yes 200 (16%) 170 (24%) 30 (5%)
Race / Ethnicity  No 1069 (84%) 534 (75%) 535 (94%)
 White, Non-Hispanic 824 (65%) 464 (65%) 360 (63%)  Missing 7 (1%) 5 (1%) 2 (0%)
 Black, Non-Hispanic 192 (15%) 106 (15%) 86 (15%) Need for Cognition
 Other, Non-Hispanic 53 (4%) 27 (4%) 26 (5%)  Low 592 (46%) 325 (46%) 267 (47%)
 Hispanic 169 (13%) 94 (13%) 75 (13%)  Median 141 (11%) 78 (11%) 63 (11%)
 2+, Non-Hispanic 38 (3%) 18 (3%) 20 (4%)  High 542 (42%) 305 (43%) 237 (42%)
Annual Income*** Health Literacy**
 <$25k 450 (35%) 180 (25%) 270 (48%)  Inadequate 630 (49%) 323 (46%) 307 (54%)
 $25k to <$50k 234 (18%) 138 (19%) 96 (17%)  Adequate 646 (51%) 386 (54%) 260 (46%)
 $50k to <$75k 186 (15%) 115 (16%) 71 (13%) Affirmation Messaging
 $75k to <$100k 142 (11%) 104 (15%) 38 (7%)  Control 599 (47%) 333 (47%) 266 (47%)
 $100k to <$125k 106 (8%) 64 (9%) 42 (7%)  Affirmation 677 (53%) 376 (53%) 301 (53%)
 $125k and up 158 (12%) 108 (15%) 50 (9%) Literacy Messaging
Educational Attainment  Standard 667 (52%) 366 (52%) 301 (53%)
 Less than HS 131 (10%) 70 (10%) 61 (11%)  Low Literacy 609 (48%) 343 (48%) 266 (47%)
 HS 403 (32%) 246 (35%) 157 (28%)
 Some College 412 (32%) 206 (29%) 206 (36%)
 College or higher 330 (26%) 187 (26%) 143 (25%)
Characteristic All mean (SD) Diabetes mean (SD) CRC mean (SD)
Information Avoidance 1.48 (0.5) 1.47 (0.5) 1.50 (0.5)
Risk Knowledge*** 3.63 (1.3) 3.93 (1.5) 3.26 (1.1)
Unpredictability*** 66.60 (23.3) 63.10 (23.2) 70.96 (22.7)
Need for Cognition 18.98 (3.7) 19.08 (3.6) 18.86 (3.8)
Open in a new tab

Note: Significant differences in distribution by survey group are indicated with

*

(p < 0.05)

**

(p < 0.01)

***

(p < 0.001). Participant demographics did not differ by assignment to condition (literacy messaging and affirmation messaging). Avoidance, family history, unpredictability, avoidance, and knowledge are disease-specific. The sums for need for cognition is less than column totals due to missing responses.

Manipulation Check

For self-affirmation, the positive self-concept salience measure was used as a manipulation check. The manipulation was confirmed for self-affirmation. For participants who endorsed greater disease information avoidance during eligibility screening, subsequent self-affirmation was associated with increases in salience of positive aspects of self-concept. In the diabetes intervention, those who were self-affirmed had a mean positive self-concept salience score 0.3 points higher than those who were not self-affirmed (mean for affirmation: 2.77, mean for no affirmation: 2.44, t = 3.62, p < .001). In the CRC intervention, those who were self-affirmed had a mean self-concept score 0.3 points higher than those who were not self-affirmed (mean for affirmation: 2.82, mean for no affirmation: 2.55, t = 2.77, p = .01).

Experimental Intervention Effects

As discussed above, in both disease conditions more than half of the participants converted from a DK to a scale point response following the interventions. However, rates of conversion did not differ by experimental condition either overall or in the specific groups expected to convert at a higher rate. That is, for participants who had inadequate health literacy, the lower literacy message did not result in greater conversion to a scale point response than the standard literacy message for either CRC risk or diabetes risk (both p > .05). For more avoidant participants, self-affirmation compared to control did not result in greater conversion for either CRC risk or diabetes risk (both p > .05). These experimental effects were also assessed after adjustment for important covariates (e.g., health literacy, education, disease knowledge, unpredictability), with no change in interpretation. Results of the test of the experimental effects within the targeted subset are shown in Table 2.

Table 2.

Experimental Effects on DK Conversion within Targeted Subsets

Participant Subset Message Comparison Disease n Unadjusted OR (95% CI) Adjusted OR (95% CI)
Low literacy Low Literacy Diabetes 323 0.82 (0.53, 1.28) 0.68 (0.43, 1.08)
CRC 307 0.92 (0.59, 1.44) 0.95 (0.60, 1.52)
Avoidant Self-affirmation Diabetes 335 1.16 (0.74, 1.81) 1.37 (0.86, 2.19)
CRC 283 1.34 (0.83, 2.18) 1.36 (0.82, 2.25)
Open in a new tab

Note: Adjusted models for diabetes are adjusted for disease knowledge and (for self-affirmation) health literacy. Adjusted models for CRC are adjusted for unpredictability, disease knowledge, and education.

Predictors of Conversion to a Scale Point Response

In bivariate associations, more education (diabetes p=.03; CRC p<.001), lower unpredictability beliefs (diabetes p=.001; CRC p=.004), and more risk factor knowledge (both p <.001) were all associated with greater conversion rates in both surveys. For diabetes, higher income (p=.01), higher health literacy (p<.001), and lower disease information avoidance (p<.001) were also significantly associated with greater conversion to a scale point response. CRC survey participants with a college degree had over four times the odds of conversion compared to participants with only some college (OR = 4.40, 95% CI = 2.61 – 7.40). In the overall samples (including those not explicitly hypothesized to convert), neither of the experimental conditions were associated with conversion to a scale point response. Full bivariate results are given in Table 3.

Table 3.

Weighted, Unadjusted Associations to Conversion from DK Responding

Demographic Diabetes DK Conversion CRC DK Conversion
% convert OR (95% CI) % convert OR (95% CI)
All 70% 66%
Sex
 Male 68% REF 63% REF
 Female 71% 1.17 (0.85, 1.61) 68% 1.20 (0.85, 1.70)
Age
 18 – 24 75% 0.76 (0.29, 1.98) 56% 0.49 (0.20, 1.17)
 25 – 34 62% 0.40 (0.18, 0.92) 73% 1.02 (0.47, 2.24)
 35 – 44 72% 0.65 (0.28, 1.54) 64% 0.67 (0.30, 1.48)
 45 – 54 73% 0.65 (0.28, 1.55) 62% 0.64 (0.28, 1.43)
 55 – 64 66% 0.49 (0.21, 1.14) 69% 0.84 (0.38, 1.85)
 65–74 72% 0.62 (0.26, 1.49) 60% 0.57 (0.25, 1.28)
 75+ 80% REF 72% REF
Race / Ethnicity
 White, Non-Hispanic 69% REF 67% REF
 Black, Non-Hispanic 60% 0.66 (0.42, 1.04) 64% 0.88 (0.51, 1.52)
 Other, Non-Hispanic 79% 1.69 (0.83, 3.46) 75% 1.52 (0.71, 3.28)
 Hispanic 74% 1.28 (0.79, 2.07) 58% 0.68 (0.43, 1.08)
 2+ Races, Non-
Hispanic		 87% 2.86 (0.40, 20.31) 81% 2.13 (0.34, 13.36)
Annual Income
 <$25k 55% REF 59% REF
 $25k to <$50k 70% 1.89 (1.13, 3.15) 66% 1.33 (0.77, 2.29)
 $50k to <$75k 73% 2.20 (1.29, 3.78) 60% 1.04 (0.60, 1.80)
 $75k to <$100k 70% 1.87 (1.08, 3.23) 67% 1.36 (0.71, 2.59)
 $100k to <$125k 78% 2.95 (1.47, 5.93) 76% 2.16 (1.12, 4.17)
 $125k and up 74% 2.36 (1.40, 3.98) 69% 1.54 (0.86, 2.75)
Educational Attainment
 Less than HS 57% 0.45 (0.27, 0.77) 51% 0.79 (0.45, 1.38)
 HS 70% 0.78 (0.51, 1.19) 61% 1.18 (0.76, 1.84)
 Some College 75% REF 57% REF
 College Grad or higher 71% 0.82 (0.52, 1.28) 85% 4.40 (2.61, 7.40)
Employment Status
 Employed 70% 1.26 (0.82, 1.95) 68% 1.71 (1.08, 2.73)
 Self-employed 67% 1.10 (0.54, 2.24) 76% 2.49 (1.10, 5.63)
 Retired 75% 1.66 (0.98, 2.81) 65% 1.49 (0.85, 2.60)
 Unemployed 64% REF 55% REF
Marital Status
 Married/ Partnered 70% 0.97 (0.42, 2.23) 65% 0.66 (0.29, 1.49)
 Single – Never Married 71% 1.02 (0.43, 2.44) 61% 0.55 (0.23, 1.32)
 Divorced/ Separated 65% 0.79 (0.31, 2.02) 74% 0.99 (0.38, 2.62)
 Widowed 71% REF 74% REF
Family History
 Yes 73% REF 68% REF
 No 68% 0.79 (0.53, 1.17) 66% 0.91 (0.35, 2.40)
Need for Cognition
 Low 67% 0.79 (0.56, 1.11) 66% 0.95 (0.66, 1.37)
 Median 73% 1.09 (0.61, 1.92) 61% 0.77 (0.42, 1.39)
 High 72% REF 67% REF
Health Literacy
 Inadequate 54% REF 58% REF
 Adequate 74% 2.42 (1.66, 3.51) 68% 1.52 (0.99, 2.33)
Affirmation Messaging
 Control 67% 0.76 (0.55, 1.05) 65% 0.94 (0.66, 1.33)
 Affirmation 73% REF 66% REF
Literacy Messaging
 Standard 71% 1.15 (0.83, 1.58) 63% 0.78 (0.55, 1.10)
 Low Literacy 68% REF 69% REF
Characteristic M (SD) OR (95% CI) M (SD) OR (95% CI)
Information Avoidance 1.41 (0.5) 0.57 (0.41, 0.79) 1.43 (0.5) 1.22 (0.86, 1.74)
Risk Knowledge 4.29 (1.2) 1.35 (1.21, 1.51) 3.63 (0.9) 1.58 (1.29, 1.93)
Unpredictability 59.72 (23.5) 0.99 (0.98, 1.00) 67.46 (21.3) 0.99 (0.98, 1.00)
Need for Cognition 19.35 (3.6) 1.03 (0.99, 1.08) 19.49 (4.2) 1.03 (0.98, 1.07)
Open in a new tab

Note: All conversion rates, means, and odds ratios are weighted to the US population. For continuous variables, the means presented are for the participants that converted to a scale point response. For diabetes, health literacy, unpredictability, avoidance, and disease knowledge remained significant after adjustment for income and education. For CRC, unpredictability, disease knowledge, and education remained significant after adjustment.

After adjustment for covariates, higher health literacy (OR = 1.62, 95% CI = 1.08 – 2.45), lower unpredictability (OR = 0.99, 95% CI = 0.98 – 1.00), lower avoidance (OR = 0.65, 95% CI = 0.47 – 0.92) and higher risk factor knowledge (OR = 1.27, 95% CI = 1.13 – 1.43), but not education (p=.25) or income (p=.07), remained significantly associated with lower DK responding for diabetes. For CRC, lower unpredictability (OR = 0.99, 95% CI = 0.98 – 1.00), higher education (overall p<.001), and higher risk factor knowledge (OR = 1.55, 95% CI = 1.26 – 1.91) remained significantly associated with conversion to a scale point CRC risk response.

Discussion

Up to 50% of the general population reports that they “don’t know” their risk for common diseases (Waters et al., 2013; Waters et al., 2016). Based on theoretical support for the mechanistic importance of risk perceptions in health behavior change (Conner & Norman, 2005) and empirical support for this premise (Sheeran et al., 2014), a common intervention strategy centers on increasing risk perceptions in order to motivate behavioral action. Given the high rate of DK responding, the documented challenges associated with changing risk perceptions (Weinstein & Klein, 1995) are likely more complex, and may be most difficult in those with inadequate health literacy and high disease information avoidance (Orom et al., 2018). In this study, we examined whether interventions designed to address these distinct groups would reduce DK responding. We found no evidence for our intervention approach, however we did find some confirmation of the mechanisms involved in providing DK responses, as well as additional insight into these potential mechanisms.

We found no significant evidence to support our first hypothesis. The self-affirmation experimental intervention was successful in increasing the salience of positive aspects of self-concept, yet it did not differentially lead to greater conversion from a DK response to a scale point response in those with higher disease information avoidance. It may be that heightened self-concept in avoiders did not specifically or adequately increase their disease risk message receptivity. That disease information avoidance remained strongly associated with lower rates of conversion in the diabetes condition only confirms the importance of this factor in DK responding. Therefore, it will be important to consider other approaches to addressing DK responding in those who endorse disease risk avoidance. Other recent studies have found that self-affirmation interventions do not always function as expected (e.g., they may be conditioned on affective state or timeliness of psychological threat and resources to address threat) (Ferrer & Cohen, 2018; Taber, McQueen, Simonovic, & Waters, 2019). Our second hypothesis was also not supported; the literacy intervention did not differentially increase conversion from a DK to a scale point response in those with inadequate health literacy. While we successfully adapted standard messaging to substantially lower readability grade level (i.e., low literacy messaging at 8th and 6th grade levels, standard literacy messaging 16th and 11th grade levels for diabetes and colorectal cancer, respectively), it may be that 8th and 6th grade levels were not low enough to differentially impact those with inadequate health literacy. Indeed, reductions in reading level are not always sufficient to address literacy limitations (Berkman et al., 2011). Other methods to reach those with inadequate literacy, beyond improvement in message readability per se, will be important to consider in future research. Finally, it is possible that the interventions did successfully target the proposed mechanisms (engaging in high disease information avoidance; having inadequate health literacy), but that there is a third variable related to both DK responding and these proposed mechanisms. Such a third variable could be a stronger or more proximal mechanism than either information avoidance or health literacy. This also would require further study.

The high rate of overall sample conversion from DK responses to a scale point response - 67% and 61% for diabetes and CRC, respectively - might have further detracted from our ability to test our first two study hypotheses. In fact, large proportions of the population may be amenable to shifting away from the use of DK responses if they are provided with some type of relevant disease risk information. Furthermore, exposure to risk perception items may over time lead to risk judgements (Windschitl, 2002). It might also be useful to examine level of response certainty of those who convert from DK to scale point responses, which could act as another potential discriminator of different “strengths” of DK responses, and has been found to be quite common even in individuals who are not using a DK response to convey their risk perceptions (Orom et al., 2017). It is possible that this additional dimension – level of response certainty – has some important implications for risk perception stability, as well as motivation for behavior change, that warrants further research.

Regarding our exploratory research question, we confirmed and extended our understanding of DK responding. In prior work we showed that lower health literacy, higher disease risk avoidance, and reduced risk knowledge are important factors in DK responding (Orom et al., 2018). In the current study, we extended this to the important role for these factors in persistent DK responding – continuing to respond DK even after receipt of health risk information. We also identified some additional factors that appear to be quite important in this process. In adjusted analyses, the belief that risk for disease (diabetes, CRC) is unpredictable (Hay et al., 2014) was associated with reduced conversion from a DK response to a scale point response, reflecting that for some a DK response may be quite consistent with the belief that disease risk is not predictable, or not knowable. Importantly, those individuals who embrace these beliefs may be relatively resistant to risk messages and may require alternative pathways to persuasion to adopt health protective behavior. Adjusted analyses also revealed some important differences between disease contexts. For diabetes but not CRC, avoidance remained an important multivariable predictor. For CRC but not diabetes, educational attainment remained an important multivariable predictor. It is possible that these two common diseases elicited differential barriers to conceptualization of a risk perception; with avoidance more a barrier in the diabetes context; and educational attainment more a barrier in the CRC context. Future research should explore this possibility. Importantly, our current mechanistic findings differ somewhat from prior work in the CRC context; in prior work, both higher avoidance and lower health literacy were important (Orom et al., 2018), and while unpredictability, knowledge, and education were significant in this prior work, they were the most important correlates of DK responding in the current study. These findings, along with the confirmed importance of health literacy and risk factor knowledge, are all important areas for future research and intervention development. Another area for future research is to extend this work beyond examination of the cognitive, deliberative risk perceptions assessed in the current study, to risk perceptions that draw on intuition and affect, which have been recently shown to be important elements of the risk perception process, and with significant relationships to behavior (Ferrer, Klein, Persoskie, Avishai-Yitshak, & Sheeran, 2016; Hay et al., 2016; Janssen, Waters, van Osch, Lechner, & de Vries, 2014; Riley et al., 2019).

The study included substantial strengths as well as limitations. Study strengths included our large and diverse population sample, the use of publicly available information regarding the risks of diabetes and CRC, and our experimental intervention design paired with in-depth exploratory follow-up analyses. Further, given that manipulation checks can influence responding to subsequent survey questions (Hauser & Schwarz, 2015), we included compliance checks subsequent to primary outcome assessments in this study. Our use of an online survey panel may introduce limitations, especially in the generalizability of our participants with inadequate literacy. Some of our cell sizes were small.

In summary, we have confirmed and extended some important insights into the core mechanisms underlying high levels of DK responding to risk perception questions in the general population. While the experimental interventions to reduce DK responding were not effective, our results confirm prior work establishing inadequate health literacy and higher disease information avoidance as important correlates of greater DK responding to risk perception questions. Addressing DK responding is an important priority given the centrality of public health efforts to galvanize health risk perceptions, and the fact that those who respond DK to risk perception questions are disproportionately underserved and less-adherent to health protective behaviors.

Acknowledgements

We acknowledge the support of NCI R01 CA197351 (Orom, Hay, MPIs) and the MSK Support/Core Grant (P30 CA008748). We also want to thank Theodore Smith, BA, for his administrative support in manuscript development. These data have been presented, in part, as a poster at the Society of Behavioral Medicine Annual Meeting 2019, Washington DC.

Footnotes

Disclosure Statement

No potential conflict of interest was reported by the authors

References

  1. American Cancer Society. (2019). Cancer Facts & Figures 2019. https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2019.html.
  2. American Diabetes Association. (2018). Standards of Medical Care in Diabetes-2018 Abridged for Primary Care Providers. Clin Diabetes, 36(1), 14–37. doi: 10.2337/cd17-0119 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aspinwall LG, Taber JM, Kohlmann W, Leaf SL, & Leachman SA (2014). Perceived risk following melanoma genetic testing: a 2-year prospective study distinguishing subjective estimates from recall. J Genet Couns, 23(3), 421–437. doi: 10.1007/s10897-013-9676-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baser RE, Li Y, Brennessel D, Kemeny MM, & Hay JL (2017). Measurement invariance of intuitive cancer risk perceptions across diverse populations: The Cognitive Causation and Negative Affect in Risk scales. J Health Psychol, 1359105317693910. doi: 10.1177/1359105317693910 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, . . . Viswanathan M. (2011). Health literacy interventions and outcomes: an updated systematic review. Evid Rep Technol Assess (Full Rep)(199), 1–941 [PMC free article] [PubMed] [Google Scholar]
  6. Cacioppo JT, Petty RE, & Kao CF (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306–307 [DOI] [PubMed] [Google Scholar]
  7. Center for Disease Control and Prevention (CDC). (2017a). What are the risk factors for colorectal cancer? Retrieved from www.cdc.gov/cancer/colorectal/basic_info/risk_factors.htm [Google Scholar]
  8. Center for Disease Control and Prevention (CDC). (2017b). Who is at risk? Retrieved from https://www.cdc.gov/diabetes/basics/risk-factors.html [Google Scholar]
  9. Conner M, & Norman P. (2005). Predicting Health Behaviour: Research and Practice with Social Cognition Models. Berkshire, England: Open University Press. [Google Scholar]
  10. Curran PG (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66(Supplement C), 4–19. doi: 10.1016/j.jesp.2015.07.006 [DOI] [Google Scholar]
  11. Doak CC, Doak LG, & Root JH (1996). Teaching Patients with Low Literacy Skills (2nd ed.). Philadelphia, PA, USA: J. B. Lippincott Company. [Google Scholar]
  12. Ferrer RA, & Cohen GL (2018). Reconceptualizing Self-Affirmation With the Trigger and Channel Framework: Lessons From the Health Domain. Pers Soc Psychol Rev, 1088868318797036. doi: 10.1177/1088868318797036 [DOI] [PubMed] [Google Scholar]
  13. Ferrer RA, Klein WM, & Graff KA (2017). Self-affirmation increases defensiveness toward health risk information among those experiencing negative emotions: Results from two national samples. Health Psychol, 36(4), 380–391. doi: 10.1037/hea0000460 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ferrer RA, Klein WM, Persoskie A, Avishai-Yitshak A, & Sheeran P. (2016). The Tripartite Model of Risk Perception (TRIRISK): Distinguishing Deliberative, Affective, and Experiential Components of Perceived Risk. Ann Behav Med, 50(5), 653–663. doi: 10.1007/s12160-016-9790-z [DOI] [PubMed] [Google Scholar]
  15. Gregg EW, Zhuo X, Cheng YJ, Albright AL, Narayan KV, & Thompson TJ (2014). Trends in lifetime risk and years of life lost due to diabetes in the USA, 1985–2011: a modelling study. The lancet Diabetes & endocrinology, 2(11), 867–874 [DOI] [PubMed] [Google Scholar]
  16. Hauser DJ, & Schwarz N. (2015). It’s a Trap! Instructional Manipulation Checks Prompt Systematic Thinking on “Tricky” Tasks, 5(2), 2158244015584617. doi: 10.1177/2158244015584617 [DOI] [Google Scholar]
  17. Hay JL, Baser R, Weinstein ND, Li Y, Primavera L, & Kemeny MM (2014). Examining intuitive risk perceptions for cancer in diverse populations. Health, Risk & Society, 16(3), 227–242. doi: 10.1080/13698575.2014.911822 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hay JL, Orom H, Kiviniemi MT, & Waters EA (2015). “I don’t know” my cancer risk: exploring deficits in cancer knowledge and information-seeking skills to explain an often-overlooked participant response. Med Decis Making, 35(4), 436–445. doi: 10.1177/0272989X15572827 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hay JL, Ramos M, Li Y, Holland S, Brennessel D, & Kemeny MM (2016). Deliberative and intuitive risk perceptions as predictors of colorectal cancer screening over time. J Behav Med, 39(1), 65–74. doi: 10.1007/s10865-015-9667-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Howell JL, & Shepperd JA (2012). Reducing information avoidance through affirmation. Psychological Science, 23(2), 141–145. doi: 10.1177/0956797611424164 [DOI] [PubMed] [Google Scholar]
  21. Howell JL, & Shepperd JA (2016). Establishing an information avoidance scale. Psychological Assessment, 28(12), 1695–1708. doi: 10.1037/pas0000315 [DOI] [PubMed] [Google Scholar]
  22. Janssen E, Waters EA, van Osch L, Lechner L, & de Vries H. (2014). The importance of affectively-laden beliefs about health risks: the case of tobacco use and sun protection. J Behav Med, 37(1), 11–21. doi: 10.1007/s10865-012-9462-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Jessop DC, Simmonds LV, & Sparks P. (2009). Motivational and behavioural consequences of self-affirmation interventions: A study of sunscreen use among women. Psychology & Health, 24(5), 529–544. doi: 10.1080/08870440801930320 [DOI] [PubMed] [Google Scholar]
  24. Kaphingst KA, Kreuter MW, Casey C, Leme L, Thompson T, Cheng MR, . . . Lapka C. (2012). Health Literacy INDEX: development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. J Health Commun, 17 Suppl 3, 203–221. doi: 10.1080/10810730.2012.712612 [DOI] [PubMed] [Google Scholar]
  25. Kiviniemi MT, Ellis EM, Orom H, Waters EA, & Hay JL (2020). ‘Don’t know’ responding and estimates of perceived risk: failing to provide a ‘don’t know’ response systematically biases laypeople’s perceived risk estimates. Health, Risk & Society, 1–17. doi: 10.1080/13698575.2020.1714557 [DOI] [Google Scholar]
  26. Krosnich JA (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cog Psychol, 5, 213–236 [Google Scholar]
  27. Krosnick JA, Holbrook AL, Berent MK, Carson RT, Hanemann WM, Kopp RJ, . . . Conaway M. (2002). The impact of “no opinion” response options on data quality - Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66(3), 371–403 [Google Scholar]
  28. McQueen A, & Klein WMP (2006). Experimental manipulations of self-affirmation: A systematic review. Self and Identity, 5(4), 289–354. doi: 10.1080/15298860600805325 [DOI] [Google Scholar]
  29. Miles A, Voorwinden S, Chapman S, & Wardle J. (2008). Psychologic predictors of cancer information avoidance among older adults: the role of cancer fear and fatalism. Cancer Epidemiol Biomarkers Prev, 17(8), 1872–1879. doi: 10.1158/1055-9965.epi-08-0074 [DOI] [PubMed] [Google Scholar]
  30. Napper L, Harris PR, & Epton T. (2008). Developing and Testing a Self-affirmation Manipulation. Self and Identity, 8(1), 45–62. doi: 10.1080/15298860802079786 [DOI] [Google Scholar]
  31. Napper L, Harris PR, & Epton T. (2009). Developing and Testing a Self-affirmation Manipulation. Self and Identity, 8(1), 45–62. doi: 10.1080/15298860802079786 [DOI] [Google Scholar]
  32. National Cancer Institute. (2019a). Colon Cancer Treatment (PDQ®)–Health Professional Version. Retrieved from https://www.cancer.gov/types/colorectal/hp/colon-treatment-pdq#section/_1
  33. National Cancer Institute. (2019b). Colon Cancer Treatment (PDQ®)–Patient Version. Retrieved from https://www.cancer.gov/types/colorectal/patient/colon-treatment-pdq#section/_93
  34. National Institute of Diabetes and Digestive and Kidney Diseases. (2016). Symptoms & Causes of Diabetes. Retrieved from https://www.niddk.nih.gov/health-information/diabetes/overview/symptoms-causes
  35. Nelson DE, Kreps GL, Hesse BW, Croyle RT, Willis G, Arora NK, . . . Alden S. (2004). The Health Information National Trends Survey (HINTS): development, design, and dissemination. J Health Commun, 9(5), 443–460; discussion 481–444. doi: 10.1080/10810730490504233 [DOI] [PubMed] [Google Scholar]
  36. Orom H, Biddle C, Waters EA, Kiviniemi MT, Sosnowski AN, & Hay JL (2017). Causes and consequences of uncertainty about illness risk perceptions. J Health Psychol, 1359105317745966. doi: 10.1177/1359105317745966 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Orom H, Schofield E, Kiviniemi MT, Waters EA, Biddle C, Chen X, . . . Hay JL (2018). Low Health Literacy and Health Information Avoidance but Not Satisficing Help Explain “Don’t Know” Responses to Questions Assessing Perceived Risk. Med Decis Making, 38(8), 1006–1017. doi: 10.1177/0272989X18799999 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Prevent Cancer Foundation. (2013). Colorectal Cancer Fact Sheet. Retrieved from https://preventcancer.org/wp-content/uploads/2015/06/Colorectal-Cancer-Fact-Sheet-2013.pdf
  39. Riley KE, Hay JL, Waters EA, Biddle C, Schofield E, Li Y, . . . Kiviniemi MT (2019). Lay beliefs about risk: relation to risk behaviors and to probabilistic risk perceptions. J Behav Med. doi: 10.1007/s10865-019-00036-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sheeran P, Harris PR, & Epton T. (2014). Does heightening risk appraisals change people’s intentions and behavior? A meta-analysis of experimental studies. Psychol Bull, 140(2), 511–543. doi: 10.1037/a0033065 [DOI] [PubMed] [Google Scholar]
  41. Sherman DK, Cohen GL, Nelson LD, Nussbaum AD, Bunyan DP, & Garcia J. (2009). Affirmed yet unaware: Exploring the role of awareness in the process of self-affirmation. Journal of Personality and Social Psychology, 97(5), 745–764. doi: 10.1037/a0015451 [DOI] [PubMed] [Google Scholar]
  42. Steele CM (1988). The Psychology of Self-Affirmation: Sustaining the Integrity of the Self. In Berkowitz L (Ed.), Advances in Experimental Social Psychology (Vol. 21, pp. 261–302): Academic Press. [Google Scholar]
  43. Taber JM, McQueen A, Simonovic N, & Waters EA (2019). Adapting a self-affirmation intervention for use in a mobile application for smokers. J Behav Med. doi: 10.1007/s10865-019-00028-1 [DOI] [PubMed] [Google Scholar]
  44. Tuomilehto J, Lindström J, Eriksson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, . . . Rastas M. (2001). Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. New England Journal of Medicine, 344(18), 1343–1350 [DOI] [PubMed] [Google Scholar]
  45. van Koningsbruggen GM, & Das E. (2009). Don’t derogate this message! Self-affirmation promotes online type 2 diabetes risk test taking. Psychology & Health, 24(6), 635–649. doi: 10.1080/08870440802340156 [DOI] [PubMed] [Google Scholar]
  46. Waters EA, Hay JL, Orom H, Kiviniemi MT, & Drake BF (2013). “Don’t know” responses to risk perception measures: implications for underserved populations. Med Decis Making, 33(2), 271–281. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Waters EA, Kiviniemi MT, Orom H, & Hay JL (2016). “I don’t know” My Cancer Risk: Implications for Health Behavior Engagement. Ann Behav Med, 50(5), 784–788. doi: 10.1007/s12160-016-9789-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Waters EA, McQueen A, & Cameron L. (2014). Perceived risk and health risk communication The Routledge Handbook of Language and Health Communication: Routledge. [Google Scholar]
  49. Weinstein ND, & Klein WM (1995). Resistance of personal risk perceptions to debiasing interventions. Health Psychol, 14(2), 132–140 [DOI] [PubMed] [Google Scholar]
  50. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, . . . Hale FA (2005). Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med, 3(6), 514–522. doi: 10.1370/afm.405 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Whitlock EP, Lin JS, Liles E, Beil TL, & Fu R. (2008). Screening for colorectal cancer: a targeted, updated systematic review for the U.S. Preventive Services Task Force. Ann Intern Med, 149(9), 638–658 [DOI] [PubMed] [Google Scholar]
  52. Windschitl PD (2002). Judging the accuracy of a likelihood judgment: The case of smoking risk. Journal of Behavioral Decision Making, 15(1), 19–35. doi: 10.1002/bdm.401| [DOI] [Google Scholar]

RESOURCES