Is the Accuracy of Self-Reported Colorectal Cancer Screening Associated with Social Desirability?

Abstract

Background

Self-reported cancer screening behaviors are often overreported and may lead to biased estimates of prevalence and of subgroup differences in screening. We examined whether the tendency to give socially desirable responses was associated with concordance between self-reported colorectal cancer (CRC) screening behaviors and medical records.

Methods

Primary care patients (n = 857) age 50 to 74 years completed a mail, face-to-face, or telephone survey that assessed CRC screening and social desirability measured by a short version of the Marlowe–Crowne scale. We used medical records to verify self-reports of fecal occult blood testing (FOBT), sigmoidoscopy, colonoscopy, and barium enema.

Results

Social desirability scores were lower for whites versus African Americans, college graduates, and patients reporting no prior screening tests; they were higher for telephone versus mail or face-to-face survey respondents. In univariable logistic regression analysis, social desirability scores were not associated with concordance for FOBT (OR = 1.03, 95% CI = 0.94–1.13), sigmoidoscopy (OR = 0.95, 95% CI = 0.86–1.04), or colonoscopy (OR = 0.99, 95% CI = 0.88–1.11); however, lower social desirability scores were associated with increased concordance for barium enema (OR = 0.87, 95% CI = 0.77–0.99). In multivariable analyses, no associations were statistically significant.

Conclusion

Social desirability as measured by the Marlowe–Crowne scale was not associated with accuracy of self-reported CRC tests in our sample, suggesting that other explanations for overreporting need to be explored.

Impact

By understanding sources of response bias, we can improve the accuracy of self-report measures.

Introduction

Population-based screening for breast, cervical, and colorectal (CRC) cancers has been shown to reduce morbidity and mortality (1). To assess whether efforts to promote cancer screening are successful, we need to measure adherence to screening guidelines. Two national surveys (2, 3) monitor screening rates at the population level, and both rely on self-reports. Likewise, many health promotion intervention trials, particularly those that are community or population based, use self-report to measure intervention outcomes. Thus, accurate self-report measures are needed to monitor progress toward increasing adherence to cancer screening guidelines.

Studies of the accuracy of self-report compared with medical record or administrative databases have found that cancer screening behaviors, including mammography, Pap tests, fecal occult blood tests (FOBT), and endoscopy, are overreported (4–8). Overreporting not only leads to inflated estimates of screening prevalence but also may differ systematically by population subgroups in ways that cause subgroup differences to be over- or underestimated (6). There also is some evidence that overreporting may differ by intervention group status such that those receiving an intervention are more likely to overreport mammography (9) and CRC screening behaviors (8) compared with a control group, thus inflating the estimate of intervention effectiveness.

A potential source of bias that has been hypothesized as a cause of overreporting in studies of cancer screening is social desirability (6, 10–12). Social desirability has been defined as the tendency to respond to questions in socially or culturally sanctioned ways (13). Measures of social desirability have been shown to be associated with under-reporting total caloric intake in women (14) and overreporting measures of physical activity (15). The construct of social desirability has been less studied in relation to self-reported cancer screening behaviors, and the few studies that did examine its effects did not directly measure social desirability (10, 12). We sought to address this gap in the literature by answering the following questions: (i) Does social desirability vary by sociodemographic characteristics (age, gender, ethnicity, education, and marital status) or other study variables (number of prior CRC screening tests, family history of CRC, and survey mode)? (ii) Is social desirability associated with concordance between medical records and self-reported CRC screening with FOBT, sigmoidoscopy, colonoscopy, or barium enema? (iii) Does adjusting for covariates change the association between social desirability and concordance?

Methods

Background

We conducted a secondary analysis of data from a randomized controlled trial designed to assess the reliability and validity of a self-report questionnaire of CRC screening behaviors developed by a National Cancer Institute (NCI) workgroup (16). The trial was funded by the Centers for Disease Control and Prevention to assess whether mode of survey administration (mail, telephone, or face-to-face) affected reliability or validity (7).

Trial participants were men and women 51 to 74 years of age who were primary care patients for at least 5 years at a large multispecialty group practice in Houston, Texas. Persons with a prior history of CRC were excluded. Of the 1,004 enrolled patients who were randomized to mail, telephone, or face-to-face mode of survey administration, 857 completed a baseline survey in 2005–2006. These 857 patients constituted the sample for this study. Self-reported CRC screening with FOBT, sigmoidoscopy, and colonoscopy was found to meet acceptable standards of test–retest reliability, concordance, sensitivity, and specificity for all survey modes; however, all were overreported as measured by the report-to-records ratio. Barium enema was underreported and showed only fair sensitivity. Additional details about the trial are reported elsewhere (7).

Measures

For this article, our primary outcome measure was concordance defined as agreement between self-report and the medical record, considered to be the “gold” standard. As described in Vernon and colleagues (7), respondents were classified as adherent or nonadherent for each test according to American Cancer Society screening guidelines in effect at the time of the study (17): FOBT within the past year or sigmoidoscopy, colonoscopy, or barium enema within the past 5 years. We used a 5-year interval for colonoscopy, rather than 10 as recommended by guidelines, because of the difficulty in identifying a sufficient number of patients who had received care at the clinic for 10 or more years. Patients could report multiple tests that occurred during the study period.

Social desirability was the primary independent variable of interest and was measured with a 10-item version of the Marlowe–Crowne Social Desirability Scale (18). Items are rated true or false and are summed to obtain a score ranging from zero to 10 (Appendix). Higher scores reflect a greater tendency to give socially desirable responses. Validated 10-item short versions of the Marlowe–Crowne instrument have shown internal consistency reliability scores ranging from 0.49 to 0.70 in samples of undergraduate students (18, 19). In our sample, coefficient alpha was 0.60.

Covariates included age in years (continuous), gender (male/female), race/ethnicity (white/African American/other), education (≥ college/some college/≤ high school), marital status (married/single), family history of CRC (yes/no), number of CRC tests during the 5-year study period (0/1/2+), and mode of survey administration (face-to-face/mail/telephone).

Statistical analysis

One-way ANOVA was used to examine mean differences in social desirability scores by covariates (question i). Covariates that were statistically significant (P < 0.05) in univariable analysis were included in multivariable analyses using logistic regression to examine the association between social desirability and concordance, before and after adjusting for covariates (questions ii and iii). Patients with multiple tests within guidelines were included in the analyses for each test they had, that is, FOBT, colonoscopy, sigmoidoscopy, and barium enema. STATA 11 (StataCorp LP) was used to conduct the analyses.

Results

Description of the sample

The mean age of the sample was 59.2 years old (SD = 5.9). Approximately 59% were White and 26% were African American. Sixty-six percent were female, 74% were married, more than 50% had at least a college degree, and 11% had a CRC family history. Nineteen percent reported having 2 or more tests during the study period. According to medical records, 16% were screened with FOBT, 27% with colonoscopy, 26% with sigmoidoscopy, and 12% with barium enema. Concordance between medical record and self-reported CRC screening was 85% for FOBT, 91% for colonoscopy, 85% for sigmoidoscopy, and 92% for barium enema.

Mean differences in social desirability scores by covariates

Pairwise comparisons showed that whites had statistically significantly lower social desirability scores than African Americans and those categorized as “other” race/ethnicity (Table 1). Respondents with 4 or more years of college education had lower social desirability scores than those with less than 4 years of college. Patients who reported that they had no prior CRC screening tests had lower social desirability scores compared with those who reported 1 or more screening tests. Telephone respondents had statistically significantly higher social desirability scores than either mail or face-to-face respondents; there was no difference between mail and face-to-face respondents.

Table 1.

Mean differences in social desirability scores by study variables

	N	Mean	SD	F	DF	P
Age	857	6.81	2.01	1.18		0.254
Gender				2.51	1	0.114
Male	293	6.66	2.08
Female	564	6.88	1.98
Race/ethnicity				22.30	2	<0.001
White	509	6.44	1.95
African American	226	7.45	2.02
Other	122	7.13	1.92
Education				4.78	2	0.009
≤HS/GED	115	7.09	2.07
<College	267	7.03	1.86
≥College	468	6.62	2.08
Married				0.93	1	0.335
Yes	636	6.77	1.99
No	215	6.93	2.09
Family history of CRC				0.39	1	0.535
Yes	97	6.67	2.10
No	736	6.81	2.01
Number of prior CRC tests				3.70	2	0.025
0	353	6.58	2.00
1	344	6.97	2.04
≥2	160	6.94	1.96
Survey mode				3.95	2	0.020
Face-to-face	280	6.64	1.95
Mail	291	6.70	2.11
Phone	286	7.08	1.96

NOTE: Higher scores on the 10-item Marlowe–Crowne (18) scale indicate more socially desirable responses.

Association between social desirability scores and concordance

In univariable logistic regression analysis, social desirability was not associated with concordance for FOBT (OR = 1.03, 95% CI = 0.94–1.13), sigmoidoscopy (OR = 0.95, 95% CI = 0.86–1.04), or colonoscopy (OR = 0.99, 95% CI = 0.88–1.11). Lower social desirability scores were associated with increased concordance for barium enema in univariable analysis (OR = 0.87, 95% CI = 0.77–0.99). The association between social desirability and concordance was not statistically significant for any CRC screening test in multivariable analyses. We also modeled the false positive rate; univariable results were generally similar to those for concordance. ORs were 0.98 (95% CI = 0.88–1.09) for FOBT, 1.01 (95% CI =0.90–1.14) for sigmoidoscopy, 1.01 (95% CI = 0.89–1.15) for colonoscopy, and 1.10 (95% CI = 0.89–1.34) for barium enema. Adjustment for covariates did not change these associations.

Discussion

Validity estimates such as concordance, sensitivity, and specificity provide useful information on the agreement between self-report and medical records, but they tell us little about factors that influence accuracy. We found that although several factors were associated with social desirability scores, social desirability was not associated with concordance for any of the CRC screening tests. Our findings were similar to those of Matthews and colleagues (11) who found that although social desirability was associated with race ethnicity, it was not associated with the accuracy of self-reported CRC screening. These findings are consistent with the view that overreporting of CRC screening may not be attributable to a tendency among survey respondents to present themselves in a favorable manner relative to perceived social norms.

Although social desirability was not measured directly, 2 studies evaluated strategies to reduce the influence of social desirability on self-reported cancer screening behaviors (10, 12). Although not statistically significant, Johnson and colleagues (12) found self-reports of mammography and Pap testing were more accurate (i.e., in agreement with medical records) when an intention question preceded questions about screening. Beebe and colleagues (10) also examined the effect of social desirability on self-reported CRC screening by asking a question about intention to get screened before or after asking about past screening behavior. They found that asking about intention before asking about screening resulted in lower reports of screening; however, self-reports were not validated against medical records. These findings indirectly support the view that social desirability influences responses to questions about cancer screening behaviors.

Although social desirability was not associated with the accuracy of self-reported CRC screening in our study, it is notable that higher social desirability scores were observed for some subgroups. Telephone survey respondents compared with mail or face-to-face respondents reported higher social desirability scores, suggesting that different modes of data collection may create different demand characteristics among respondents, a finding that deserves further investigation in future studies, particularly as new communication technologies, such as the Internet and smart phones, are used to collect survey data.

Limitations of our study are that the results may not generalize to other populations because study participants were a self-selected sample of relatively educated patients from one medical practice in a large urban area. Given our findings of subgroup differences in social desirability scores by race and education in this relatively homogeneously population, future studies should examine the effect of social desirability on the accuracy of self-reported CRC and other screening behaviors in more diverse populations. Despite general agreement that social desirability may influence the accuracy of self-reports, there is no consensus about how to measure it. Future studies should explore alternative ways to measure this construct. Nevertheless, social desirability, as measured by the Marlowe–Crowne scale, was not associated with accuracy of self-reported CRC tests in our sample, suggesting that other explanations for overreporting need to be explored. For example, telescoping, a cognitive memory error that occurs when an event is recalled as occurring more recently or more distally than it did in fact occur, also may lead to overreporting (20).

Our findings extend prior research by measuring social desirability using a validated scale and by assessing its association with sociodemographic and other variables and with the accuracy of self-reported CRC screening behaviors. Our findings also provide support for the use of survey measures, such as the one used in this study, to monitor the prevalence of screening and to evaluate intervention effects when medical records are unavailable.

Appendix

Social desirability scale (18)

1. I never hesitate to go out of my way to help someone in trouble. (True)

2. I have never intensely disliked anyone. (True)

3. When I don’t know something I don’t at all mind admitting it. (True)

4. I am always courteous, even to people who are disagreeable. (True)

5. I would never think of letting someone else be punished for my wrong doings. (True)

6. I sometimes feel resentful when I don’t get my way. (False)

7. There have been times when I felt like rebelling against people in authority even though I knew they were right. (False)

8. I can remember “playing sick” to get out of something. (False)

9. There have been times when I was quite jealous of the good fortune of others. (False)

10. I am sometimes irritated by people who ask favors of me. (False)