Lack of validity of self-reported mammography data

Abstract

This qualitative literature review aimed to describe the totality of peer-reviewed scientific evidence from 1990 to 2017 concerning validity of self-reported mammography. This review included articles about mammography containing the words accuracy, validity, specificity, sensitivity, reliability or reproducibility; titles containing self-report, recall or patient reports, and breast or ‘mammo’; and references of identified citations focusing on evaluation of 2-year self-reports. Of 45 publications meeting the eligibility criteria, 2 conducted in 1993 and 1995 at health maintenance organisations in Western USA which primarily served highly educated whites provided support for self-reports of mammography over 2 years. Methodological concerns about validity of self-reports included (1) telescoping, (2) biased overestimates particularly among black women, (3) failure to distinguish screening and diagnostic mammography, and (4) failure to address episodic versus consistent mammography use. The current totality of evidence supports the need for research to reconsider the validity of self-reported mammography data as well as the feasibility of alternative surveillance data sources to achieve the goals of the Healthy People Initiative.

Introduction

The Healthy People Initiative, administrated by the Office of Disease Prevention and Health Promotion of the US Centers for Disease Control and Prevention (CDC), provides science-based, 10-year national objectives which constitute a national prescription for improving the health of all Americans.¹ The programme establishes benchmarks and monitors progress over time, partly to measure the impact of prevention activities.¹ The Initiative also identifies specific data sources to be used for each objective. For breast cancer prevention, Objective C-17 for Healthy People 2020 aims to ‘Increase the proportion of women who receive a breast cancer screening based on the most recent guidelines’.² The target population includes women ages 50–74 years. The data source designated for surveillance of progress towards this objective is the National Health Interview Survey (NHIS), also administered by the CDC.² The NHIS is a nationwide, cross-sectional, inperson, household interview survey based on cluster sampling of households and non-institutional group quarters (eg, college dormitories).³ The following are specific NHIS questions used for monitoring: (1) Have you ever had a mammogram? and (2) When did you have your most recent mammogram?² Mammograms themselves are described as ‘An x-ray of each breast to look for breast cancer’.⁴ Monitoring estimates track the percentage of women aged 50–74 years who have had a mammogram in the past 2 years. Data used for monitoring are therefore based on self-report, which has been criticised for its tendencies towards over-reporting, particularly among minority populations.⁵ Moreover, these NHIS questions do not distinguish between screening mammograms and mammograms which are used for follow-up after a diagnosis of breast cancer has been made, thereby adding to the probability of overestimation.⁵

Possible reasons for overestimation among blacks and African–Americans include the less detailed wording of the NHIS questions pertaining to mammography. In part, this possibility became apparent in the data from the US Behavioral Risk Factor Surveillance System (BRFSS). BRFSS is a long-standing state and local telephone survey of non-institutionalised residents regarding health-related risk behaviours, chronic health conditions and use of preventive services.⁶ More than 400 000 adult interviews are conducted each year.⁶ The BRFSS questionnaire wording reveals that more specific descriptions of mammography (ie, ‘A mammogram is an X-ray of the breast and involves pressing the breast between two plastic plates’) resulted in lower estimates of mammography use, particularly among African–Americans.⁵ A possible reason is that the more graphic description resulted in increased specificity in responses.⁵ It is also proposed that women with poor health who may be seeking care for numerous conditions requiring frequent contact with the medical system may make the specifics of mammography less distinct and more difficult to recall.⁷

At present, plans are under way for Healthy People 2030,⁸ so it seems important and timely to conduct a comprehensive qualitative review of peer-reviewed scientific publications pertaining to the validity of self-reported mammography.

Methods

We used Medline search strategies previously reported in meta-analyses of the validity of self-reported mammography.^{9 10} These strategies included using article titles containing the words accuracy, validity, specificity, sensitivity, reliability or reproducibility, and titles containing self-report, recall or patient reports, and breast or ‘mammo’. We also searched the references of identified citations to locate additional studies of interest. We described the resulting publications in terms of time, place, age, race and ethnicity, source of the study population, type of healthcare facility, whether there was information on annual and/or biennial frequency of mammography, and whether 2-year self-reports were specifically addressed. The enquiry focused on 2-year self-report. This is particularly pertinent to Healthy People since women with mammography screening within 2 years are considered up to date. In addition, since Medicare provides insurance benefits for mammography to all women 65 years and older, we also explored specific information about this population.

Results

Forty-five publications were identified (4, 9–52),^{4 9–52} and these are summarised in table 1. In all, 9 articles were published from 1990 to 1994,^11–19 13 from 1995 to 1999,^20–32 9 from 2000 to 2004,^33–41 8 from 2005 to 2009,9 10 42–47 5 from 2010 to 2014,^{4 48–51} and 1 from 2015 to January 2018.⁵² Aside from the USA, countries of origin included Canada,⁴⁹ Israel⁴⁵ and the Netherlands.⁴⁸ The lower age limit for inclusion for all but three studies was 40 years. Two of the three studies accepting women younger than 40 years were concerned with validity of self-reports among persons with known genetic risk for breast cancer.^{48 49} Participants included a variety of racial (black, white, Native American, Asian) and ethnic/religious (Arab, French Canadian, Hispanic, Orthodox Jewish) groups. Studies included persons from across the socioeconomic spectrum, although several studies (reviewed in refs ¹⁰) focused on the socioeconomically disadvantaged. One study⁵⁰ concerned persons with intellectual developmental disabilities. Settings (specifically identified in table 1) for the 42 non-meta-analysis studies included health maintenance organisations (HMOs) (n=12), non-HMO clinical services (n=13), populations (n=13) and participants in research investigations (n=4). Of the 45 articles, 27 addressed 2-year recall or recall in the elderly. Of these, only two studies supported the validity of self-reported, 2-year recall among the elderly (65+ years of age). Each was done in HMO settings in 1993 and 1995 and reported in 2003.^{36 37} While finding the accuracy of self-reports acceptable in the study settings, the authors nonetheless cautioned against projecting their findings to the general population: ‘Caution is necessary concerning the generalizability of our findings to the entire US population and other diverse populations, because of the characteristics of our study sample and setting’.³⁶ In the second study of Caplan et al,³⁷ they noted: ‘It is important to keep in mind that this study used a relatively homogenous insured managed care population composed of mainly white women, aged 40–75 years, with at least a high school education, who were either currently employed or retired. Although the results cannot be generalized to the United States population, they provide credible insight regarding the utility of the BRFSS in an important segment of the population…Our study results suggest that self-reported data ascertained using the BRFSS provide an accurate estimate of the prevalence of screening for breast…cancers in KPC [(Kaiser Permanente Colorado)] and possibly other similar managed care populations with similar enrollees’.

Table 1.

Description of scientific, peer-reviewed research about the validity of self-reported mammography: Australia, Canada, Israel, the Netherlands and the USA, 1990–2017

Reference	First author and year of publication	Country of study		Age range (or meta- analysis)		Clinic (C), population-based (P) or other (O)			N	Survey administration method			Elderly subpopulation specifically addressed		Test of 2-year self-report
11	King, 1990	USA		50–74		C (HMO)			200	Phone interview			No		No
12	Loftus, 1990	USA		40+		C (HMO)			119	Mail			No		No
13	Brown, 1992	USA		Overall 17–79; mammography group not stated		C (HMO)			162	Phone interview			No		No
14	Degnan, 1992	USA		50–74		C (HMO)			487 (1988), 486 (1990)	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘Surveys that ask ‘when was your last mammogram’ will overestimate usage. Such surveys can, however, accurately estimate change in usage’.
15	Fulton-Kehoe, 1993	USA		50–75		C (HMO)			78	Mail			No		No
16	Gordon, 1993	USA		40–74		C (HMO)			386	Phone interview and mail			No		Yes
	Conclusions about 2-year self-report: ‘Self-reported data may overestimate the percentage of the population that has been screened and underestimate the interval since the last cancer detection procedures’.
17	Whitman, 1993	USA		50+		C (public health department)			924	Phone interview			No		No
18	Etzi, 1994	USA		50–74		C (public health department van)			237	Phone interview			No		No
19	Sudman, 1994	USA		50+		C (HMO)			32	Focus groups, face-to-face interviews			No		Yes
	Conclusions about 2-year self-report: The ‘gross accuracy of mammogram self-report’ was 76.3%. ‘It was not possible to recommend any changes in the questions currently used in the NHIS to obtain information about…mammograms’.
20	Hiatt, 1995	USA		40–74		C (HMO)			687	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘…reliance on self-report data to assess a population’s screening status relative to some goal should be questioned. Such assessment will substantially overestimate progress toward goals that have been set’.
21	Johnson, 1995	USA		35–65		P (Native American)			201	Face-to-face interview			No		No
22	Kottke, 1995	USA		40–89		P			1019	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘We noted a discrepancy between the self-reported and verified rates of testing. This difference in rates suggests that the true rates may be significantly less than the self-reported rates’.
23	Montaño, 1995	USA		50+ (for mammography)		C			3281 patient charts	Mail			No		No
24	Suarez, 1995	USA		40+		P			450	Face-to-face interview			No		Yes
	Conclusions about 2-year self-report: Self-reports ‘Will greatly overestimate the prevalence of screening’.
25	Crane 1996	USA		50+		C (county health department)			576	Phone interview and face-to-face (<4%) interview			No		No
26	Paskett, 1996	USA		40+		P			441	Face-to-face interview			No		No
27	Zapka, 1996	USA		50–74		C (Western Massachusetts)			392	Mail or phone interview			No		No
28	Warnecke, 1997	USA		50+		C (HMO)			178	Not stated			No		No
29	Champion, 1998	USA		45–64		O (study participants)			268	Face-to-face interview			No		Yes
	Conclusions about 2-year self-report: ‘Self-report alone may not provide accurate rates of mammography compliance. Further research is necessary with ethnic and low-income women’.
30	McGovern, 1998	USA		40–92		C (county medical centre)			477	Face-to-face interview			No		No
32	Lawrence, 1999	USA		50–70		C (military)			232	Phone interview			No		No
33	Thompson, 1999	USA		50–69		C (public hospital)			361	Mail and telephone			No		No
31	Barratt, 2000	Australia		30–69		P (well women)			124	Phone interview			No		No
34	Martin, 2000	USA		40+		C (HMO)			194	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘…self-reports are reasonably accurate compared with medical records…’.
35	McPhee, 2002	USA		40–74		P			846	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘Population estimates of breast…cancer screening rates based upon patient self-reports need to be adjusted downward, by as much as one-quarter to one-third, for low-income, ethnic women’.
36	Caplan, 2003	USA		50–80		C (HMO)			949	Phone interview			Yes		Yes
	Conclusions about elderly and 2-year self-report: ‘The accuracy of self-reports was not related to age race/ethnicity, years since last preventive checkup, smoking status, perceived health status, or perceived risk of developing breast cancer, after controlling for all of the other variables in the model…We found that self-reported data on mammographic screening, having the most recent mammogram within a defined interval (2 years), could be used in clinical decision-making and surveillance. However, it would certainly be preferable to use medical records if they were available at a cost and level of effort that was manageable’. ‘Caution is necessary concerning the generalizability of our findings to the entire US population and other diverse populations, because of the characteristics of our study sample and setting’.
37	Caplan, 2003	USA		40–74		C (HMO)			480	Phone interview			Yes		Yes
	Conclusions about 2-year self-report: ‘The results suggest that self-reported BRFSS [(Behavioral Risk Factor Surveillance System)] data are highly sensitive for assessing the prevalence of breast…cancer screening in this managed care population but not very specific. However, it is important to keep in mind that this study used a relatively homogenous insured managed care population composed of mainly white women, aged 40–75 years, with at least a high school education, who were either currently employed or retired. Although the results cannot be generalized to the United States population, they provide credible insight regarding the utility of the BRFSS in an important segment of the population…Our study results suggest that self-reported data ascertained using the BRFSS provide an accurate estimate of the prevalence of screening for breast…cancers in KPC [(Kaiser Permanente Colorado)] and possibly other similar managed care populations with similar enrollees. Therefore, it would seem reasonable for the BRFSS to continue to use self-reporting as the means of obtaining its data. In addition, it is reasonable and appropriate for the BRFSS to continue to use its current wording to obtain its data regarding women’s usage of mammography…’.
38	Norman, 2003	USA		40–64		Cases (incident breast cancer)—C Controls—P			2495 cases; 615 controls	Phone interview			No		Yes
	Conclusions about 2-year self-report: ‘In an interview-based case-control study of the efficacy of screening mammography, 1) estimated true prevalence of recent screening mammography adjusted for sensitivity and specificity will be slightly lower than self-reported prevalence, and 2) differential misclassification of exposure status is slight. Therefore, odds ratios will likely be biased toward the null, underestimating screening efficacy’.
39	Armstrong, 2004	USA		50–75		C (Philadelphia, Pennsylvania Medicaid managed care organisation)			399	Phone interview and mail			Yes		No
	Conclusions about elderly (and race): Adherence rates were not affected by age 65+ years. African–American adherence was significantly greater when measured by self-report than by administrative claims or the medical record.
40	Fiscella, 2004	USA		40+		P (1996 Medical Expenditure Panel Survey)			3090	Face-to-face interview			No		No
	Pertinent information: ‘In the annual Household Survey conducted in 1996, women were asked ‘How long has it been since you had a mammogram’? Possible responses included ‘Within past year’, ‘Within past 2 years’, ‘Within past 5 years’, ‘More than 5 years’, and ‘Never’. Women who reported receiving one within the past year were coded as having received a mammogram. In the Medical Events Survey, women were asked to recall any medical services, events, or procedures that they received during the prior 4 months between 1996 and 1997: ‘Looking at this card, which of these services, if any, did you have during the visit’?’ Racial disparities were found for the Medical Events Survey, but not the Household Survey. Conclusion: ‘Estimates of racial, but not ethnic, disparities in mammography seem to depend on how the question is asked. These results caution against exclusive reliance on annual self-reports for monitoring disparities in preventive care’.
41	Tumiel-Berhalter, 2004	USA		40+		C			314	Face-to-face interview			No		Yes
	Conclusions about 2-year self-report: ‘…self-report of recommended screening was consistently higher than medical record documentation [(% last mammogram within guidelines = 91.1% by self-report and 57.2% by chart review)]…Self-report of receipt of mammography…was consistently higher than medical record documentation of screening. Self-report misrepresented actual screening practices as identified by high sensitivity rates and low specificity rates. However, high negative predictive values suggested that asking women about their recent mammography use may be an inexpensive, easy intervention to increase screening among women currently not being screened by encouraging dialog between patient and provider about reasons for not being screened and/or other means of obtaining screens’.
42	Johnson, 2005	USA		50–94		P			587	Phone and face-to-face interviews			No		Yes
	Conclusions about 2-year self-report: ‘The most interesting of our findings were those related to the intentions manipulation, which are supportive of the proposition that first asking about future intentions reduces the social desirability demands of reporting positive past behavior, thereby increasing data quality…’.
43	Fiscella, 2006	USA		65+		P (Medicare Current Beneficiary Survey, 1999–2002)			49 645	Personal records and face-to-face interview			Yes		No
	Conclusions about elderly: ‘This study shows that estimates of racial/ethnic disparities, across a variety of preventive care procedures, vary depending on whether self-report or claims are used to assess them. Whether these differences reflect biases in participant report or in billing claims is unclear. These competing explanations have profoundly different policy implications, and thus warrant careful study. Future monitoring of disparities in screening will require more careful distinction of screening from diagnostic uses of preventive procedures’. Note: Results were criticised by Craig et al (please see 46 below) who stated that an error in the way results were reported ‘Inherently reduced their validity estimates, and artificially increased discordance between self-report and claims-based measures’.
44	Holt, 2006	USA		65+		P (Medicare Current Beneficiary Survey)			5461	Personal records and face-to-face interview			Yes		No
	Conclusions about elderly: ‘Our findings show that self-report of mammography compared with self-report verified by claims data provide conflicting evidence of disparities in mammography, particularly among Black women. The results suggest caution in over-reliance on self-report data for estimating disparities in the receipt of preventive services. On the basis of these findings, we believe it is premature to conclude that disparities in mammography have been eliminated. Further exploration of the reasons for differences between self-report and claims information is warranted’.
9	Rauscher, 2008	USA		Meta-analysis		We calculated summary random-effects estimates for sensitivity and specificity, separately for mammography, clinical breast exam, Pap smear, prostate-specific antigen testing, digital rectal exam, fecal occult blood testing and colorectal endoscopy.
	Observations and conclusions: ‘Black-White and Hispanic-White disparities in mammogram and Pap smear prevalence estimates seemed to be considerably larger than those based on the observed estimates alone…The Healthy People 2010 goals call for increasing the percentage of women adhering to national cancer screening guidelines. Healthy People 2010 calls for an increase, by 2010, in Pap smear utilization in the preceding 3 years from 92% to 97%, mammography in the preceding 2 years from 67% to 70%, annual fecal occult blood test from 35% to 50%, and colorectal endoscopy in the preceding 5 years from 37% to 50%. Results from this meta-analysis indicate that we are probably further from these goals than survey data suggest. Another broad goal of Healthy People 2010 is the reduction of disparities in health and health care utilization. Again, according to this meta-analysis, disparities in cancer screening by race/ethnicity are likely to be larger than they seem to be in national survey data. These inaccuracies need to be taken into account when interpreting progress toward the Healthy People 2010 goals of increasing utilization and reducing disparities. Because the NHIS is the major source of data on cancer screening used for tracking prevalence in the U.S. population, validation studies should be undertaken for a sample of respondents within the NHIS, and designed with enough power to detect meaningful differences in sensitivity and specificity for different racial/ethnic and socioeconomic groups’. Summary: ‘When estimates of self-report accuracy from this meta-analysis were applied to cancer-screening prevalence estimates from the National Health Interview Survey, results suggested that prevalence estimates are artificially increased and disparities in prevalence are artificially decreased by inaccurate self-reports…National survey data are overestimating cancer-screening utilization for several common procedures and may be masking disparities in screening due to racial/ethnic differences in reporting accuracy’.
45	Baron-Epel, 2008	Israel		52–74		C (Maccabi Health Services)			1536	Phone interview			No		Yes
	Conclusions about 2-year self-report: Agreement between self-reported mammography and claims records depends on cultural and socioeconomic factors.
46	Craig, 2009	USA		65+		P (Medicare Current Beneficiary Survey, 1991–2006)			15 537	Personal records and face-to-face interview			Yes		No
	Conclusions about elderly: ‘In this study, the likelihood that a screened woman reports screening decreases by 1.8% per month of recall and by an additional 8.7% if the screening event occurred in a different calendar year. The combined evidence suggests that over a quarter of older women failed to report mammography use a year after screening. In their analysis of Medicare Current Beneficiary Survey responses, Fiscella, Holt and colleagues stated that the mammography question’s referent period was ‘since last year,’ which is inaccurate…’.
47	Cronin, 2009	USA		40–79		P			Not stated	Telephone interview			Yes		Yes
	Conclusions (and observations) about elderly and 2-year self-report: ‘Self-report estimates of mammography use in the prior two years from the Vermont BRFSS are 14–27 percentage points higher than actual screening rates across age groups. The differences in NHIS screening estimates from models are similar for women 40–49 and 50–59 years and greater than for those 60–69, or 70–79 (27 and 26 percentage points vs. 14, and 14, respectively). Over reporting is highest among African American women (24.4 percentage points) and lowest among Hispanic women (17.9) with white women in between (19.3). Values of sensitivity and specificity consistent with our results are similar to previous validation studies of mammography. Conclusion: Over-estimation of self-reported mammography usage from national surveys varies by age and race/ethnicity. A more nuanced approach that accounts for demographic differences is needed when adjusting for over-estimation or assessing disparities between populations’.
10	Howard, 2009	USA		Meta-analysis		‘Objectives To conduct a systematic review and meta-analysis of the accuracy of self-reported Pap smear and mammography screening compared to medical record…About 37 articles were reviewed and accuracy indices of self-report were calculated. Meta-analysis with random effects was used. Study heterogeneity was investigated and meta-regressions were done including in the models those factors that were hypothesized, a priori, to potentially explain heterogeneity. Results Pooled sensitivity and specificity…for mammography [recall] were 94.9% (95% CI; 93.4%–96.4%) and 61.8% (95% CI; 54.1%–69.5%)…There was significant heterogeneity for all indices. Stratifying by the study population source (population versus clinic-based), population characteristics (minority or low socio-economic status versus not), length of recall (within past 12 months versus longer), and expected completeness of the medical record (authors searched radiology or pathology reports of all likely facilities women may have attended, versus studies that did not) did not eliminate heterogeneity’.
	Conclusions about 2-year self-report: ‘Women tend to over-report their participation in…mammography screening in a given timeframe. The pooled estimates should be interpreted with caution due to unexplained heterogeneity’.
5	Njai, 2011	USA		Meta-analysis		‘We adjusted BRFSS mammography use data for age by using 2000 census estimates and for misclassification by using the following formula: (estimated prevalence − 1 + specificity) / (sensitivity + specificity − 1). We used values reported in the literature for the formula (sensitivity = 0.97 for both black and white women, specificity = 0.49 and 0.62, respectively, for black and white women)… After adjustment for misclassification, the percentage of women aged 40 years or older in 1995 who reported receiving a mammogram during the previous 2 years was 54% among white women and 41% among black women, compared with 70% among both white and black women after adjustment for age only. In 2006, the percentage after adjustment for misclassification was 65% among white women and 59% among black women compared with 77% among white women and 78% among black women after adjustment for age only’.
Conclusions about 2-year self-report: ‘Self-reported data overestimate mammography use — more so for black women than for white women. After adjustment for respondent misclassification, neither white women nor black women had attained the Healthy People 2010 objective (≥70%) by 2006, and a disparity between white and black women emerged’.
48	Pijpe, 2011	The Netherlands		<30 to 50+		O (The Netherlands Collaborative Group on Hereditary Breast Cancer)			177	Mail			No		Partial (last 5 years)
	Conclusions (and observations) about 2-year self-report: ‘Although cases more often tended to underestimate their exact age at first mammogram, whereas unaffected carriers tended to overestimate, this difference in the direction of inaccuracy was not statistically significant…Accuracy of age at last mammogram was moderate and improved to excellent for agreement within 1 year. Carriers tended to underreport the time since last mammogram (‘telescoping’) and over-reported the number of mammograms. Conclusion: Accuracy of self-reported lifetime mammography history in carriers highly varied, depending on the measure under investigation. However, the extent of the observed misclassification was small and mostly non-differential’.
49	Larouche, 2012	Canada		21–81		O (participants in the INHERIT* Study			307	Self-administered			Yes		No
	Conclusions about elderly: ‘Overall, the agreement between self-reports and administrative data was 88% (j = 0.74). [although accuracy decreased significantly with increasing age]…Self-report overestimates the use of mammography, mainly because women tend to minimize the elapsed time since their last mammography. Self-reports should be used cautiously to assess adherence to mammographic screening following BRCA1/2 testing’.
50	Son, 2013	USA		40+		O (participants in the ‘Women Be Healthy’ programme)			155	Face-to-face and computer-assisted interview			Yes		No
	Conclusions about the elderly: There was no association between age and accuracy of self-report. ‘Clinicians and researchers are cautioned to corroborate self-reported data with other sources for patients and research participants with intellectual disability’.
51	Allgood, 2014	USA		40 to 65+		P (venue-based sampling in two low-income communities on the west side of Chicago)			1221	Comparison of self-reported survey responses to medical records			Yes		Yes
	Conclusions: Across all categories of all sociodemographic variables examined, mammography use estimates based on self-reports were considerably larger than the corresponding estimates based on medical record documentation. Overall impact: ‘Relying on known faulty self-reported mammography data as a measure of mammography use provides an overly optimistic picture of utilization, a problem that may be exacerbated in vulnerable minority communities’ (p2).
52	Nandy, 2016		USA		40–74		O (Korean-American women recruited from religious organisations)	97			Self-report on written survey versus medical record	No		No

*Interdisciplinary Health Research International Team on Breast Cancer Susceptibility.

BRFSS, Behavioral Risk Factor Surveillance System; HMO, health maintenance organisation; NHIS, National Health Interview Survey.

Holt et al ⁴⁴ conducted a particularly relevant study in which they compared the responses of 5461 participants in the Medicare Current Beneficiary Survey with claims data. Each participant, in effect, served as her own control. The authors concluded that ‘On the basis of these findings, we believe it is premature to conclude that disparities in mammography have been eliminated. Further exploration of the reasons for differences between self-report and claims information is warranted’.

Two meta-analyses focused on current self-reporting methods used for the NHIS⁹ and BRFSS.⁴ Each of these reports concluded that these methods overestimate mammography utilisation and underestimate racial disparities or inequalities. Specifically, Rauscher et al ⁹ concluded that

When estimates of self-report accuracy from this meta-analysis were applied to cancer-screening prevalence estimates from the National Health Interview Survey, results suggested that prevalence estimates are artificially increased and disparities in prevalence are artificially decreased by inaccurate self-reports…National survey data are overestimating cancer-screening utilization for several common procedures and may be masking disparities in screening due to racial/ethnic differences in reporting accuracy

Rauscher et al ⁹ specifically cautioned against reliance on the NHIS, stating that

Because the NHIS is the major source of data on cancer screening used for tracking prevalence in the U.S. population, validation studies should be undertaken for a sample of respondents within the NHIS, and designed with enough power to detect meaningful differences in sensitivity and specificity for different racial/ethnic and socioeconomic groups

Njai et al ⁵ concluded that ‘Self-reported data overestimate mammography use — more so for black women than for white women. After adjustment for respondent misclassification, neither white women nor black women had attained the Healthy People 2010 objective (≥70%) by 2006, and a disparity between white and black women emerged’. With reference to 2-year self-report, they concluded that ‘Women tend to over-report their participation in…mammography screening in a given timeframe. The pooled estimates should be interpreted with caution due to unexplained heterogeneity’.⁴

Discussion

The present qualitative review of the totality of published evidence suggests a lack of validity of self-reports of mammography. This review also documents the historical development of scientific evidence about the quality of self-reported information provided in response to health survey questions about mammography screening. It demonstrates a remarkably consistent set of challenges to surveillance practices of the Healthy People programme, even as methods of analysis have grown increasingly complex. The narrative approach was also chosen, in part, because extensive, well-done meta-analyses confirming previous concerns about self-report have already been published^{4 9 10} to little or no apparent effect.⁵³ Perhaps, by presenting more than quarter-century of research as it has evolved, the depth of scientific objections will become clearer.

In part, persistence of the present self-reported information protocols for mammography may reflect assertions that self-report is the only feasible, cost-effective way to obtain such information.⁵² Nonetheless, the aforementioned NHIS questions (ie, Have you ever had a mammogram? and When did you have your most recent mammogram?)⁴ are subject to several cogent concerns about bias, including (1) telescoping, whereby people recall distant events as occurring more recently than they actually happened⁵⁴; (2) greater likelihood of producing inconsistent/overestimates from black women⁷; (3) failure to distinguish between screening and diagnostic mammography⁴; and (4) failure to address the issue of whether mammography screening is consistently used (as opposed to being ‘up to date’). This is so, even though additional questions already included in the NHIS survey were used as resources for tracking the progress of the Healthy People programme.⁵⁵

Biased overestimates of mammography screening use may have serious adverse clinical and public health consequences. For example, Dr Harold Freeman, a past president of the American Cancer Society, wrote in the New York Times:

…for many years, the dominant cause of higher mortality has been late-stage disease at the time of initial treatment, in part as a result of black women being less likely to undergo mammography. However, this gap has been closed. The CDC reports that the rate of mammography is now the same in black and white women….⁵⁶

Similarly, the Susan G Komen Foundation, a leading organisation which focuses exclusively on breast cancer, quotes data to the effect that ‘Black women now have slightly higher rates of mammography use than other women’.⁵⁷ Based on the present data, neither the Freeman nor the Komen statements are likely to be accurate.

Aside from making more comprehensive use of existing NHIS information, additional surveillance alternatives include greater use of administrative claims⁵⁸ and HEDIS (Healthcare Effectiveness Data and Information Set),⁵⁹ as well as expansion of mammography registries.⁶⁰ Specifically, Smith-Bindman et al ⁵⁸ noted that 94% of women who had at least one mammogram within a 2-year reference period were accurately classified by administrative claims data as having undergone a mammogram during that period. Also, while Medicare claims are not available from HMOs, these organisations and others do provide information on mammography utilisation to the HEDIS.⁵⁹ Finally, the National Cancer Institute’s Breast Cancer Surveillance Consortium⁶⁰ might serve as a national mammography registry model, but at present it only operates in the states of New Hampshire, North Carolina; Vermont; Washington; San Francisco, California; and Chicago, Illinois.⁶⁰

In conclusion, the current totality of evidence supports the need for research to reconsider the validity of self-reported mammography data as well as the feasibility of alternative surveillance data sources to achieve the goals of the Healthy People Initiative.