Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 25.
Published in final edited form as: J Health Psychol. 2017 Feb 1;24(9):1221–1232. doi: 10.1177/1359105317693910

Measurement invariance of intuitive cancer risk perceptions across diverse populations: The Cognitive Causation and Negative Affect in Risk scales

Raymond E Baser 1, Yuelin Li 2, Debra Brennessel 3, M Margaret Kemeny 4, Jennifer L Hay 5
PMCID: PMC7906483  NIHMSID: NIHMS1671143  PMID: 28810422

Abstract

Intuitive cancer risk perceptions may inform strategies to motivate cancer prevention behaviors. This study evaluated factor structure and measurement invariance of two new measures of intuitive cancer risk, the Cognitive Causation and Negative Affect in Risk scales. Single- and multiple-group confirmatory factor analysis models were fit to responses from three diverse samples. The confirmatory factor analysis models fit the data well, with all comparative fit indices (CFI) ≥ 0.94. Items flagged by chi-square difference tests as potentially non-invariant were largely invariant between samples according to practical fit indices (e.g. ΔCFI). These novel scales may be particularly relevant in diverse, underserved populations.

Introduction

Beliefs about cancer risk are key components of decisions regarding cancer screening and prevention behaviors (Slovic et al., 2005). This is consistent with individual-level theories of health behavior change (Conner and Norman, 2005), and is supported by extensive empirical research (Watts et al., 2003; Woloshin et al., 2000). In fact, illness risk perceptions are often better predictors of health actions than objective risk status (Aiken et al., 1995; Lipkus et al., 2000). Assessment of cancer risk perception has traditionally relied on quantitative probability measures, requiring respondents to indicate their risk in either numeric (e.g., from 0–100%) or verbal (e.g., from very low to very high) likelihood formats. These measures assume that people evaluate cancer risk in a rational, rule-based manner, and focus on a narrow domain of the broader cancer risk perception construct.

Recent theories and empirical findings have moved beyond exclusive focus on rational deliberation in risk determination. Cancer is associated with greater, more elaborated fear and worry than other diseases (Vrinten et al., 2016; Borland et al., 1994), supporting the need to assess non-rational domains of cancer risk perception. Theoretically, the Self-Regulation Model (Cameron and Leventhal, 2003), the affect heuristic (Finucane et al., 2000; Peters et al., 2006; Slovic et al., 2002), and the Risk-as-Feelings hypothesis (Loewenstein et al., 2001) all articulate the importance of emotion in the rapid, automatic formulation of cancer risk judgments. Empirically, a feelings-of-risk measure performed better than probability magnitude measures in predicting subsequent influenza vaccination (Weinstein et al., 2007). A similar measure predicted behavioral intentions for colon cancer screening better than both absolute and comparative probability measures (Dillard et al., 2012). Measurement of intuitive elements of the risk appraisal process—including thoughts as well as feelings—may improve interventions seeking to raise risk awareness, and suggest new strategies to motivate cancer screening and prevention behavior.

Health risk perceptions differ across cultures (Francois et al., 2009; Huerta and Macario, 1999; Joseph et al., 2009; Lee, 2010; Pasick et al., 2009). Similarly, psychometric properties of instruments can vary along cultural and demographic dimensions due to systematic subgroup differences in item relevance, interpretation, and reading level. For example, Shepperd et al. (2015) found that Cronbach’s alpha reliability coefficients for three commonly used psychological instruments were significantly smaller, and unacceptably poor, among respondents with low compared to high education. This underscores the importance of developing and validating measures of risk perception in diverse populations, and of psychometric analysis to establish measurement invariance across important subgroups.

The Cognitive Causation and Negative Affect in Risk Scales

We recently completed initial validation of two such measures, Cognitive Causation and Negative Affect in Risk (Hay, Baser, et al., 2014). Our initial item pool resulted from focus-groups conducted in an ethnically diverse, inner-city primary care clinic (Hay et al., 2005). Exploratory factor analysis (EFA) of university student responses to these 47 items yielded five factors, the first two of which had promising psychometrics and measured novel cancer risk perception constructs (Hay, Baser, et al., 2014). Based on the content of the items, the two scales were named Cognitive Causation (CC) and Negative Affect in Risk (NAR), highlighting the role of both thoughts and emotions in intuitive processing of risk perceptions.

CC taps the perspective that thoughts about cancer risk may encourage development of cancer, and that minimizing such thoughts might reduce cancer risk. Beliefs that certain thoughts or behaviors ward off, or encourage, negative events are common across cultures (Wiseman and Watt, 2004). The 10 items received relatively high levels of endorsement in three different samples (Hay, Baser, et al., 2014). Notably, urban primary care patients endorsed each of the CC items at roughly twice the frequency of college students and older community men. The CC scale had Cronbach’s alpha ≥ 0.90 in all samples, and had small-to-moderate correlations (r = 0.20–0.33) with commonly used risk perception measures (e.g., likelihood and feelings-of-risk), indicating minimal overlap with existing assessments of risk.

We propose that NAR measures anticipatory affect, emotions generated during risk information processing. NAR items had considerably higher levels of endorsement than CC items within all three samples, and Cronbach’s alpha coefficients were all ≥ 0.89. NAR was correlated with cancer worry (r = 0.33), but at a magnitude modest enough to indicate discrimination between these constructs.

The Current Study

The ability to measure intuitive aspects of cancer risk appraisal processes will aid efforts to raise awareness of cancer risk and prevention strategies. CC and NAR, specifically, may represent modifiable, and previously unmeasured, psychological barriers to cancer screening and engagement in other risk reduction behaviors. Prior work (Hay, Baser, et al., 2014) provided strong initial evidence of the validity and reliability of the CC and NAR scales. However, their factor structure was derived from EFA in a convenience sample of university students. Although both scales had excellent internal consistency reliability within each of the three samples, it is unclear whether the factor structure of the CC and NAR items is robust under a more rigorous, restrictive measurement model as well as stable across diverse samples.

The goal of the current study was to continue the initial validation of the CC and NAR scales by evaluating the robustness and generalizability of their factor structure across three racially, culturally, and socioeconomically diverse samples: University Students (primarily young, female, Caucasian, and with 12+ years of education), older Community Men (all male, age 50+, predominantly Caucasian and college-educated), and Urban Primary Care patients (predominantly non-Caucasian, lower-educated, lower-income, and foreign-born). Specifically, we had three objectives: (a) assess the fit of the two-factor structure of CC and NAR items by fitting separate single-group confirmatory factor analysis (CFA) models to the responses of the three samples; (b) evaluate the generalizability and measurement invariance of the scales by fitting multiple-group CFA models; and (c) outline recommendations for future use of the CC and NAR scales, including scoring instructions and refinements suggested by the current analysis.

Method

Participants

Three distinct samples of participants were utilized in this study and are briefly described below. The samples are described in more detail in Hay, Baser, et al. (2014). The study received Institutional Review Board approval at all participating institutions.

University Students.

We surveyed 568 undergraduate psychology students at a private university in the northeast United States. Students were age 17–55 years (median 19), 78% female, and 69% Caucasian, with the remainder Hispanic (11%), African-American (10%), Asian (5%), and other (5%).

Community Men.

Men age ≥ 50 years were sampled from the general population and 182 completed the items as part of a larger study. Participants were age 50–86 years (median 60), primarily Caucasian (82%), relatively well-educated (71%, college degree or higher), and had relatively high incomes (60%, >$70K/yr).

Urban Primary Care.

We utilized a sample of 127 ethnically diverse (32% African-American, 31% Caribbean Black, 13% Asian/Pacific Islander, 10% Hispanic, 3% non-Hispanic Caucasian, and 11% other), primary care participants from a New York City Health and Hospitals facility in Queens, NY. Only 28% were born in the United States. Most (63%) were female, had not attended college (68%), and ranged in age 18–88 years (median 46). Among the 67% who reported household income, 75% made < $30K/yr.

Measures

Participants completed the 16 items in Table 1 as part of a larger pool of 47 items. Three CC items were added after the Community Men survey was finalized. Specifically, CC8, CC9, and CC10 were not administered to the Community Men.

Table 1:

Cognitive Causation and Negative Affect in Risk Items

Item Number Item
Cognitive Causation
CC1 If I think too hard about the possibility of getting cancer, I could get it.
CC2 If I don’t believe I will get cancer, I won’t.
CC3 Negative thoughts about getting cancer might make me get it.
CC4 Considering that I could get cancer might bring on bad luck.
CC5 Too much thought about cancer risk could encourage the disease.
CC6a Being hopeful about my cancer risk might protect me from getting it.
CC7 Thinking that I am likely to get cancer may give me cancer.
CC8 In general, if a person thinks about the possibility of getting cancer, they are more likely to get it.
CC9a In general, people who don’t think too much about getting cancer tend to avoid it.
CC10a For those who already have cancer, limiting their thoughts about cancer risk helps them get better.
Negative Affect in Risk
NAR1 I get frightened when I think I could get cancer.
NAR2 Thinking about getting cancer makes me afraid.
NAR3 I get a bad feeling just thinking about the possibility of getting cancer.
NAR4 Thinking about my chances of getting cancer makes me uncomfortable.
NAR5 I dread getting cancer.
NAR6 I can’t think about getting cancer without feeling afraid.
Open in a new tab

Note. The four response options to each item are: Strongly Disagree, Disagree, Agree, and Strongly Agree.

a

Based on the results presented in this manuscript, items CC6, CC9, and CC10 should be omitted from the scoring of the Cognitive Causation Scale.

Statistical Analyses

All CFA models were fit using Mplus software, v6.11 (Muthen & Muthen, 2011), using theta parameterization and weighted least-squares means-and-variance adjusted estimation (WLSMV) to account for the ordinal nature of the items. First, we fit separate single-group CFA models for the three samples. Items were specified to load on a single factor and to have independent errors. CC and NAR factors were allowed to correlate. Goodness-of-fit was evaluated with three indices: the comparative fit index (CFI) and Tucker-Lewis Index (TLI), for which values ≥ 0.90 indicate reasonably good fit (Bentler, 1990); and root mean squared error of approximation (RMSEA), for which values ≤ 0.05 suggest good model fit, and values between 0.05 and 0.08 suggest reasonable fit (Browne and Cudeck, 1993).

Second, we evaluated measurement invariance of CC and NAR items using multiple-group CFA models. Specifically, we were interested in identifying items that might have been interpreted differently by the highly diverse Urban Primary Care group compared to the other two samples. Using established procedures for examining measurement invariance for items with ordinal responses (Muthén and Asparouhov, 2002), we tested for measurement invariance by fitting two series of multiple-group CFA models: one series with Urban Primary Care compared to Community Men, and the second series with Urban Primary Care compared to the University Students.

The first model in each series was a baseline configural model in which all item characteristics (i.e., loadings and thresholds) were freely estimated and allowed to vary between the sample pairs. The second model in each series, the full invariance model, forced each item’s loadings and thresholds to be identical and invariant between the samples. The fit of the restricted, full invariance model was statistically compared to the baseline configural model using a chi-square difference test (Δχ2) for WLSMV-estimated nested models (DIFFTEST in Mplus v6.11).

When Δχ2indicated the full invariance model had significantly worse fit than the baseline configural model (p < 0.05), we inspected Mplus modification indices to identify the item most responsible for the poorer fit of the restricted model. A third, partial invariance model was then specified as identical to the full invariance model, except the loadings and thresholds of the item highlighted by the modification indices were allowed to vary between the samples. Following the procedure recommended by Muthén and Asparouhov (2002), each item’s loading and thresholds were both restricted to be equal across groups, or were both free to vary across groups. Again, Δχ2 (Mplus DIFFTEST) compared this partial invariance model to the baseline model and, if significant, modification indices were inspected to identify another item most responsible for the poor fit. This process was repeated until Δχ2was no longer significant.

The Δχ2 plus modification indices, used alone, are subject to suggest spurious model modifications, capitalizing on chance variations in sample characteristics (MacCallum et al., 1992). Rather than modify the CC and NAR scales based solely on this procedure, we interpreted findings of potential non-invariance in light of both statistical and substantive considerations.

From a statistical perspective, we inspected the changes in other fit indices (i.e., ΔCFI, ΔTLI, and ΔRMSEA) between the baseline configural model and the other models fit in a series (Chen, 2007; Cheung and Rensvold, 2002; Little et al., 2007; Sass et al., 2014). Values of ΔCLI < 0.01 support measurement invariance, suggesting that differences in fit between models are negligible, even when Δχ2 is significant (Cheung and Rensvold, 2002; Chen, 2007). TLI, however, is a parsimony-adjusted fit index, and it is possible for more restricted, parsimonious models to have higher TLIs than less parsimonious ones due to gains in parsimony outweighing any resulting losses in fit. Given this, measurement invariance is supported if the invariance model has a TLI as high or higher than the baseline configural model (i.e., if ΔTLI ≤ 0.00) (Marsh et al., 2010). Meade et al. (2008) suggest that ΔRMSEA < 0.007 supports measurement invariance, but recommended ΔRMSEA not be used to assess measurement invariance due to variability in its effectiveness across study conditions.

From a substantive perspective, we interpreted findings of non-invariance by carefully examining the content of items identified as potentially non-invariant and by attempting to rationally deduce probable causes of non-invariance. We also consulted a recent study verifying Haitian-Creole and Spanish translations of the items (Hay, Brennessel, et al., 2014) for information that might elucidate or otherwise corroborate any finding of non-invariance in the current study.

Results

Single-Group CFA Models

The single-group CFA models all fit the data reasonably well (see top of Table 2). Model fit was excellent for the University and Community Men samples, with TLI and CLI values above 0.95 and RMSEA values of approximately 0.05. Fit was also good among Urban Primary Care respondents, with CFI = 0.95, TLI = 0.94, and RMSEA = 0.085 for the 13-item model and 0.078 for the 16-item model. In each sample, all item loadings were statistically different from zero (p < 0.001). Figure 1 depicts standardized item loadings and factor correlations for each sample. The correlation between CC and NAR factors was similar in the University and Community Men samples (0.15 and 0.18 respectively), but notably larger (r = 0.36) among Urban Primary Care patients.

Table 2.

Single and Multiple Group Confirmatory Factor Analysis Model Fit and Measurement Invariance Tests

CFA Models by Sample N x2 x2 df CFI ΔCFI TLI ΔTLI RMSEA RMSEA 90% CI ΔRMSEA Δx2 Δx2 df Δx2 p-value
 University Students (16 items) 568 301.3 103 0.981 -- 0.978 -- 0.058 (0.051 – 0.066) -- -- -- --
 Community Men (13 items)a 182 85.9 64 0.992 -- 0.99 -- 0.043 (0.012 – 0.066) -- -- -- --
 Urban Primary Care (16 items) 127 182.9 103 0.948 -- 0.939 -- 0.078 (0.059 – 0.096) -- -- -- --
 Urban Primary Care (13 items)b 127 122.1 64 0.95 -- 0.939 -- 0.085 (0.061 – 0.107) -- -- -- --
CFA Measurement Invariance Models
Urban Primary Care and Community Men (13 items)
 Baseline Model: Loadings and thresholds vary 309 208.6 128 0.979 -- 0.974 -- 0.064 (0.048 – 0.079) -- -- -- --
 Full MI: Loadings and thresholds equal 309 260.3 163 0.974 0.005 0.976 −0.002 0.062 (0.048 – 0.076) 0.002 58.5 35 0.008
 Partial MI: CC4 Free 309 255.5 160 0.975 0.004 0.976 −0.002 0.062 (0.048 – 0.076) 0.002 53.0 32 0.011
 Partial MI: CC4 + CC6 Free 309 244.2 157 0.977 0.002 0.977 −0.003 0.060 (0.045 – 0.074) 0.004 39.7 29 0.089
 Partial MI: CC4 + CC6 + NAR1 Free 309 237.8 154 0.978 0.001 0.978 −0.004 0.059 (0.044 – 0.074) 0.005 32.5 26 0.176
Urban Primary Care and University Students (16 items)
 Baseline Model: Loadings and thresholds vary 695 474.7 206 0.977 -- 0.973 -- 0.061 (0.054 – 0.069) -- -- -- --
 Full MI: Loadings and thresholds equal 695 532.9 250 0.976 0.001 0.977 −0.004 0.057 (0.050 – 0.064) 0.004 78.4 44 0.001
 Partial MI: CC10 Free 695 526.3 247 0.976 0.001 0.977 −0.004 0.057 (0.050 – 0.064) 0.004 68.5 41 0.005
 Partial MI: CC10 + CC9 Free 695 519.4 244 0.976 0.001 0.977 −0.004 0.057 (0.050 – 0.064) 0.004 59.6 38 0.014
 Partial MI: CC10 + CC9 + CC6 Free 695 506.8 241 0.977 0 0.977 −0.004 0.056 (0.049 – 0.063) 0.005 42.7 35 0.174
Open in a new tab

Note. CFI = Comparative Fit Index, TLI= Tucker-Lewis Index, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Interval, MI = Measurement Invariance, CC = Cognitive Causation, NAR = Negative Affect in Risk, Δx2 = Chi-Square difference test, df = degrees of freedom.

a

Three items (CC8-CC10) were not administered to Community Men

b

This 13-item model in the Urban Primary Care sample was used for comparison to the Community Men sample.

Figure 1.

Figure 1.

Open in a new tab

Standardized factor loadings from single-group confirmatory factor analysis (CFA) models for cognitive causation and negative affect in risk across three samples (University Students, Community Men, Urban Primary Care, presented from left to right, respectively). The double-headed arrow represents the correlations between the two latent factors (Cognitive Causation, Negative Affect in risk). Items CC8-CC10 were not administered to the Community Men and are depicted in the figure as “--”.

Measurement Invariance

The full invariance multiple-group CFA model for Urban Primary Care vs. Community Men had significantly worse fit compared to the baseline configural model according to Δχ2 (Δχ2(35) = 58.5, p = 0.008), despite having almost identical values for the other fit indices (Table 2). Modification indices suggested allowing the loading and thresholds for item CC4 to vary between samples. Δχ2 still suggested this partial invariance model had worse fit compared to the baseline model (Δχ2(32) = 53.0, p = 0.011). Modification indices suggested additionally freeing parameters for CC6, which resulted in a non-significant Δχ2 and the final partial invariance model between Urban Primary Care and Community Men (Δχ2(29) = 39.7, p = 0.089).

A similar process evaluated measurement invariance of the 16-item model between Urban Primary Care and University samples. After freeing the loadings and thresholds for items CC10, CC9, and CC6 to vary across the samples, no further changes were indicated by either Δχ2 (p = 0.174) or modification indices (see bottom of Table 2).

Overall, the full invariance models had good fit, with CFI, TLI, and RMSEA values nearly identical to the corresponding values from the baseline configural model. Despite the significant Δχ2, the ΔCFI, ΔTLI, and ΔRMSEA values all supported full invariance.

Discussion

This study examined the factor structure and measurement invariance of two novel measures of intuitive cancer risk belief, the CC and NAR scales, across three racially, culturally, and socioeconomically diverse samples. The factor structure was supported in all samples, including the largely foreign-born, minority, and socioeconomically disadvantaged Urban Primary Care sample. Although Δχ2 tests indicated minor departures from full measurement invariance between samples, all other change-in-fit indices supported full measurement invariance of the CC and NAR scales, providing justification for cross-sample comparisons of their scores.

Further research is needed to examine relationships of NAR and CC with behavioral outcomes such as cancer screening behaviors. Atkinson and colleagues (2015) found associations between risk perceptions and colorectal cancer screening. Another recent meta-analysis found that heightened health risk appraisals, including negative anticipatory emotions, were associated with subsequent risk-reducing intentions and behaviors, such as tobacco cessation, safe-sex behavior, dietary behavior, sun protection, and safe driving (Sheeran et al., 2014). Notably, these associations were stronger when multiple components of risk appraisal were heightened (e.g., anticipatory emotion, likelihood risk perception, and perceived severity), suggesting that future interventions adopt a multi-pronged strategy to amplify uptake of cancer risk-reducing behaviors.

Valid, reliable measures of intuitive risk perception will improve upon risk awareness-raising approaches to public health messaging, such as public service announcements, that focus narrowly on probability magnitude estimates. It is vital these measures be psychometrically sound across diverse populations to maximize the impact of these efforts. This may be particularly important in, but not limited to, populations with lower acculturation levels, health literacy, and education. Indeed, recent research has shown that the robust psychometrics of psychological instruments validated in college students and other convenience samples cannot be assumed to generalize to more diverse populations, highlighting continued need for this work (Shepperd et al., 2015).

A limitation of this data is that, due to intra-sample homogeneity but inter-sample heterogeneity, it is unclear which characteristic(s) of the samples were responsible for findings of potential non-invariance. Instead, we interpreted results based on broader differences between sample features overall. We did not have access to response rates for our study samples. Further research is needed to examine generalizability of these findings in larger, diverse samples.

Examination of Potentially Non-Invariant Items

Items CC4, CC6, CC9, and CC10 were flagged as potentially non-invariant between samples. CC4 asks participants to ponder “luck” (“Considering that I could get cancer might bring on bad luck”), which may prompt distinct cultural and/or societal connotations. The concept of luck may be more salient for the ethnically diverse, mixed-gender Urban Primary Care respondents compared to the relatively homogeneous Community Men. Indeed, this was the least endorsed CC item among Community Men, with only 3.3% responding “agree” or “strongly agree”. By contrast, 21.5% of Urban Primary Care respondents endorsed CC4.

Item CC6 (“Being hopeful about my cancer risk may protect me from getting it”) showed evidence of non-invariance in both series of models. Similar to “luck”, it is possible that the term “hopeful” may carry different emotional or cultural import across samples. However, a recent study verifying Haitian Creole and Spanish translations of these items (Hay, Brennessel, et al., 2014) revealed participants had trouble understanding “hopeful” as applied to cancer risk. It seems likely even in the current study – English language only – that some respondents struggled with the juxtaposition of “hopeful” and “cancer risk”.

Items CC9 (“In general, people who don’t think too much about getting cancer tend to avoid it”) and CC10 (“For those who already have cancer, limiting their thoughts about cancer risk helps them get better”) were identified as non-invariant between University and Urban Primary Care samples. CC9 was written to measure the belief that if other people think too much about getting cancer, they increase their risk of developing the disease. However, the actual wording of CC9 is the logical inverse of that statement. The logical double negation in CC9 might obscure its intended meaning among respondents with low English reading levels, such as non-native English speakers and/or those with low education. Both characteristics were considerably more prevalent among Urban Primary Care compared to University respondents, and may have resulted in more variable interpretations of CC9 among this group. With respect to CC10, participants in the translation study (Hay, Brennessel, et al., 2014) reported difficulty conceptualizing cancer risk in the context of already having cancer. CC10 was likely similarly confusing to respondents in the current study.

Recommended Future Use and Scoring of the CC and NAR Scales

Based on the results of this study and the examination of item content above, we recommend dropping items CC6, CC9, and CC10 from the CC scale. Although CC4 was also flagged as potentially non-invariant, likely due to differences in the meaning of “luck” between the samples, we can find no compelling substantive reason to drop the item at this time. On the contrary, despite potential variability in interpreting “luck”, this item is the most explicit, face valid connection between the CC scale and superstitious thinking, a construct that conceptually guided writing of the CC items. Measurement invariance of the remaining CC and NAR items, including CC4, should be further evaluated in future studies.

We calculated Cronbach’s alpha coefficients from the items’ polychoric correlation matrix to account for their ordinal response format (Gadermann et al., 2012). After omitting items CC6, CC9, and CC10, the reliability of the CC scale did not change among Urban Primary Care patients (α = 0.90) and only slightly decreased among University and Community Men respondents, from 0.92 to 0.91. The NAR scale reliability was 0.92, 0.89, and 0.91 in the University, Community Men, and Urban Primary Care samples, respectively.

The scoring method we have chosen for the CC and NAR scales converts the mean item scores to the percent of maximum possible for each scale, yielding scale scores with range 0–100 (Cohen et al., 1999). This scoring method places scores on a uniform range commonly found in other settings, making it easy to directly and quickly compare relative levels of different scales scored in this fashion. This method can also be used when respondents have missing or invalid item answers; however, we recommend scoring a scale only when at least half of items on the scale have valid answers.

We have written scoring routines for the CC and NAR scales for use with the free, open-source R statistical software (R Core Team, 2016). The CC and NAR scoring functions can be accessed by installing the PROscorer package (Baser, 2017) in R. For the sake of error-avoidance and scientific reproducibility, we encourage users of the CC and NAR items to use these scoring functions rather than write their own scoring syntax in R or other statistical software.

Conclusion

Our results provide additional support for the factor structure of the CC and NAR scales, and offer evidence that they have invariant measurement properties across diverse samples and are suitable for group comparisons. These constructs may be operative when personal cancer risk assessment is prompted during cancer control and prevention interventions. Indeed, CC and NAR have relevant behavioral correlates: CC and NAR with colorectal cancer screening adherence (Hay, Ramos, et al., 2016) and patient activation (Hay, Zabor, et al., 2016), and NAR with intention to undergo colonoscopy (Boonyasiriwat et al., 2014). Risk perception in diverse populations is an under-researched topic (Huerta and Macario, 1999). Our commitment to conducting this research in a primary care population with rich demographic diversity will enhance the generalizability of the scales and allow us to document differences in cancer risk beliefs across different population subgroups. Superstitious thinking about future events may be embedded in cultural belief systems, and may be more common in Latino or Asian populations (Darke and Freedman, 1997; Subbotsky and Quinteros, 2002), warranting further work in a wide range of underserved populations, as well as comparison across racial/ethnic groups, education levels, and immigration status. Research on the diverse range of beliefs about cancer held by the public – and studies to develop psychometrically-robust assessments of them – should continue to engage diverse populations from the outset, and will ultimately result in effective intervention strategies to reduce cancer burdens and disparities across all population subgroups.

Acknowledgments

This project was supported by NIH grant support R03 CA93182, K07 CA098106, and R21 CA133376 to Jennifer Hay, as well as by a Cancer Center Support Grant from the National Cancer Institute to MSKCC (award number P30 CA008748).

Contributor Information

Raymond E Baser, Memorial Sloan Kettering Cancer Center.

Yuelin Li, Memorial Sloan Kettering Cancer Center.

Debra Brennessel, Mount Sinai School of Medicine.

M Margaret Kemeny, Mount Sinai School of Medicine.

Jennifer L Hay, Memorial Sloan Kettering Cancer Center.

References

  1. Aiken LS, Fenaughty AM, West SG, et al. (1995) Perceived determinants of risk for breast cancer and the relations among objective risk, perceived risk, and screening behavior over time. Womens Health 1: 27–50. [PubMed] [Google Scholar]
  2. Atkinson TM, Salz T, Touza KK, et al. (2015). Does colorectal cancer risk perception predict screening behavior? A systematic review and meta-analysis. Journal of Behavioral Medicine, 38(6), 837–850. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baser RE. (2017). PROscorer: Scoring Functions for Patient-Reported Outcome (PRO) Measures and Other Psychometric Instruments. R Package Version 0.0.1. Retrieved from http://CRAN.R-project.org/package=PROscorer [Google Scholar]
  4. Bentler PM (1990) Comparative fit indexes in structural models. Psychological bulletin 107(2): 238. [DOI] [PubMed] [Google Scholar]
  5. Boonyasiriwat W, Hung M, Hon SD, et al. (2014) Intention to Undergo Colonoscopy Screening Among Relatives of Colorectal Cancer Cases: a Theory-Based Model. Annals of Behavioral Medicine 47(3): 280–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Borland R, Donaghue N, & Hill D (1994). Illnesses that Australians most feared in 1986 and 1993. Australian Journal of Public Health, 18(4), 366–369. [DOI] [PubMed] [Google Scholar]
  7. Browne M and Cudeck R (1993) Alternative ways of assessing model fit. In: Bollen KA and Long JS (eds), Testing structural equation models, Newbury Park, CA: Sage, pp. 136–162. [Google Scholar]
  8. Cameron LD and Leventhal H (eds) (2003) The self-regulation of health and illness behaviour. London and New York: Routledge. [Google Scholar]
  9. Chen FF (2007) Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal 14(3): 464–504. [Google Scholar]
  10. Cheung GW and Rensvold RB (2002) Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal 9(2): 233–255. [Google Scholar]
  11. Cohen P, Cohen J, Aiken LS, et al. (1999) The problem of units and the circumstance for POMP. Multivariate behavioral research 34(3): 315–346. [Google Scholar]
  12. Conner M and Norman P (eds) (2005) Predicting health behaviour: Research and practice with social cognition models, 2nd edition. Buckingham: Open University Press. [Google Scholar]
  13. Darke PR and Freedman JL (1997) The belief in good luck scale. Journal of Research In Personality 31(4): 486–511. [Google Scholar]
  14. Dillard AJ, Ferrer RA, Ubel PA, et al. (2012) Risk perception measures’ associations with behavior intentions, affect, and cognition following colon cancer screening messages. Health Psychology 31(1): 106–13. [DOI] [PubMed] [Google Scholar]
  15. Finucane ML, Alhakami A, Slovic P, et al. (2000) The affect heuristic in judgments of risks and benefits. Journal Of Behavioral Decision Making 13(1): 1–17. [Google Scholar]
  16. Francois F, Elysee G, Shah S, et al. (2009) Colon cancer knowledge and attitudes in an immigrant Haitian community. Journal of Immigrant and Minority Health 11(4): 319–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gadermann AM, Guhn M and Zumbo BD (2012) Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation 17(3): 1–13. [Google Scholar]
  18. Hay JL, Baser R, Weinstein ND, Li Y, Primavera L, & Kemeny MM. (2014). Examining intuitive risk perceptions for cancer in diverse populations. Health, Risk & Society, 16(3), 227–242. 10.1080/13698575.2014.911822 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hay JL, Brennessel D, Kemeny MM, & Lubetkin EI. (2014). Examining Intuitive Cancer Risk Perceptions in Haitian-Creole and Spanish-Speaking Populations. Journal of Transcultural Nursing, 16(3), 227–242. 10.1080/13698575.2014.911822 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hay JL, Ramos M, Li Y, Holland S, Brennessel D, & Kemeny MM. (2016). Deliberative and intuitive risk perceptions as predictors of colorectal cancer screening over time. Journal of Behavioral Medicine, 39(1), 65–74. 10.1007/s10865-015-9667-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hay JL, Shuk E, Cruz G, & Ostroff J (2005). Thinking through cancer risk: characterizing smokers’ process of risk determination. Qualitative Health Research, 15(8), 1074–1085. [DOI] [PubMed] [Google Scholar]
  22. Hay JL, Zabor EC, Kumar J, Brennessel D, Kemeny MM, & Lubetkin EI. (2016). Cancer beliefs and patient activation in a diverse, multilingual primary care sample. Psycho-Oncology, 25(9), 1071–1078. 10.1002/pon.4196 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Huerta EE and Macario E (1999) Communicating health risk to ethnic groups: Reaching Hispanics as a case study. Journal of the National Cancer Institute. Monographs (25): 23–26. [DOI] [PubMed] [Google Scholar]
  24. Joseph G, Burke NJ, Tuason N, et al. (2009) Perceived susceptibility to illness and perceived benefits of preventive care: an exploration of behavioral theory constructs in a transcultural context. Health Educ Behav 36(5 Suppl): 71S–90S. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lee SJ (2010) Uncertain Futures: Individual Risk and Social Context in Decision-Making in Cancer Screening. Health Risk Soc 12(2): 101–117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lipkus IM, Kuchibhatla M, McBride CM, et al. (2000) Relationships among breast cancer perceived absolute risk, comparative risk, and worries. Cancer epidemiology, biomarkers and prevention 9(9): 973–5. [PubMed] [Google Scholar]
  27. Little TD, Card NA, Slegers DW, et al. (2007) Representing contextual effects in multiple-group MACS models. Little TD, Bovaird JA, and Card NA (eds), Modeling contextual effects in longitudinal studies: 121–147. [Google Scholar]
  28. Loewenstein GF, Weber EU, Hsee CK, et al. (2001) Risk as feelings. Psychological Bulletin 127(2): 267–86. [DOI] [PubMed] [Google Scholar]
  29. MacCallum RC, Roznowski M and Necowitz LB (1992) Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin 111(3): 490–504. [DOI] [PubMed] [Google Scholar]
  30. Marsh HW, Lüdtke O, Muthén B, et al. (2010) A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment 22(3): 471. [DOI] [PubMed] [Google Scholar]
  31. Meade AW, Johnson EC and Braddy PW (2008) Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology 93(3): 568–592. [DOI] [PubMed] [Google Scholar]
  32. Muthén B and Asparouhov T (2002) Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. Mplus Web Notes 4(5): 1–22. [Google Scholar]
  33. Muthen LK and Muthen BO (2011) Mplus User’s Guide,. 6th ed. Los Angeles, CA: Muthen & Muthen. [Google Scholar]
  34. Pasick RJ, Burke NJ, Barker JC, et al. (2009) Behavioral theory in a diverse society: Like a compass on Mars. Health Educ Behav 36(5 Suppl): 11S–35S. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Peters E, McCaul KD, Stefanek M, et al. (2006) A heuristics approach to understanding cancer risk perception: Contributions from judgment and decision-making research. Annals of Behavioral Medicine 31(1): 45–52. [DOI] [PubMed] [Google Scholar]
  36. R Core Team (2015) R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. [Google Scholar]
  37. Sass DA, Schmitt TA and Marsh HW (2014) Evaluating Model Fit With Ordered Categorical Data Within a Measurement Invariance Framework: A Comparison of Estimators. Structural Equation Modeling: A Multidisciplinary Journal 21(2): 167–180. [Google Scholar]
  38. Sheeran P, Harris PR and Epton T (2014) Does heightening risk appraisals change people’s intentions and behavior? A meta-analysis of experimental studies. Psychological Bulletin 140(2): 511–543. [DOI] [PubMed] [Google Scholar]
  39. Shepperd JA, Emanuel AS, Dodd VJ, et al. (2015) The reliability of psychological instruments in community samples: A cautionary note. Journal of Health Psychology (Epub before print): 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sherman SJ, Cialdini RB, Schwartzman DF, et al. (1985) Imagining can heighten or lower the perceived likelihood of contracting a disease - the mediating effect of ease of imagery. Personality and Social Psychology Bulletin 11(1): 118–127. [Google Scholar]
  41. Slovic P, Peters E, Finucane ML, et al. (eds) (2002) The affect heuristic. Intuitive judgment: Heuristics and biases., Cambridge, England: Cambridge University Press. [Google Scholar]
  42. Slovic P, Peters E, Finucane ML, et al. (2005) Affect, risk, and decision making. Health psychology 24(4 Suppl): S35–40. [DOI] [PubMed] [Google Scholar]
  43. Subbotsky E and Quinteros G (2002) Do cultural factors affect causal beliefs? Rational and magical thinking in Britain and Mexico. British Journal Of Psychology 93: 519–543. [DOI] [PubMed] [Google Scholar]
  44. Vrinten C, McGregor LM, Heinrich, et al. (2016). What do people fear about cancer? A systematic review and meta-synthesis of cancer fears in the general population. Psycho-Oncology, doi: 10.1002/pon.4287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Watts BG, Vernon SW, Myers RE, et al. (2003) Intention to be screened over time for colorectal cancer in male automotive workers. Cancer epidemiology, biomarkers & prevention 12(4): 339–49. [PubMed] [Google Scholar]
  46. Weinstein ND, Kwitel A, McCaul KD, et al. (2007) Risk perceptions: Assessment and relationship to influenza vaccination. Health Psychology 26(2): 146–51. [DOI] [PubMed] [Google Scholar]
  47. Wiseman R and Watt C (2004) Measuring superstitious belief: Why lucky charms matter. Journal of behavioral decision making 37(8): 1533–1541. [Google Scholar]
  48. Woloshin S, Schwartz LM, Byram S, et al. (2000) A new scale for assessing perceptions of chance: A validation study. Medical decision making 20(3): 298–307. [DOI] [PubMed] [Google Scholar]

RESOURCES