Sensitivity of Self-report Mammography Use in Older Women

Abstract

Background

Recent survey evidence indicates a decline in mammography use among older women. The objective of this study was to detect sensitivity variation in self-reported mammography use and pose evidence-based suggestions to increase survey accuracy.

Methods

Using 1991-2006 Medicare Current Beneficiary Survey (MCBS), 15,357 women, age 65 or older, were selected based on use of mammography services. The women were interviewed in the community setting at random periods after screening and asked, “Have you had a mammogram or breast x-ray since [today's date] one year ago?” Statistical analyses were conducted between March 11 and April 28 of 2008. This study tested whether sensitivity (i.e., probability of an affirmative response) was dependent on length of the recall period and on respondent demographic and socioeconomic characteristics.

Results

Overall, 90.4% of the older women self-reported use; however, sensitivity decreased as the recall period lengthened (90% at 6 months, 80% at 12 months). This time effect was significantly higher among older, economically disadvantaged women. Sensitivity also decreased an additional 13.8% if the event occurred in the previous calendar year, and 3.5% if conducted in a non-English language or by proxy.

Conclusion

Greatest sensitivity use occurred during the 6-month period after service without straddling calendar years. These findings may aid the tailoring of future surveys for older adults, improving the recall of preventive services.

Introduction

After decades of increasing mammography rates in the United States, a comparison of data from the 2000 and 2005 National Health Interview Surveys (NHIS) suggested a decline in self-reported mammography use^{1, 2}. Due to the importance of cancer screening, a growing literature has examined the accuracy of mammography measures^3-24.

Sensitivity of self-reported use is the probability that a woman who had a mammogram reports receiving this service. Sensitivity analysis requires a cohort of women who used mammography services, and are subsequently surveyed. A cohort of women with mammography claims is a reasonable alternative to a medical chart-based cohort, because claims have been shown to be highly specific^{6, 20, 24}. Using a claims-based cohort from the Medicare Current Beneficiary Survey, 1991-2006, this study tests whether the probability of reporting (sensitivity) decreases with length of recall, survey attributes, and respondent demographic, socioeconomic, and health characteristics.

Methods

Medicare Current Beneficiary Survey (MCBS)

Medicare, a United States public health insurance plan for qualified persons age 65 and older and persons with certain disabilities, began covering biennial screening mammography in January 1, 1991 and increased coverage to annual screenings on January 1, 1998. The MCBS is a federally sponsored continuous panel survey of Medicare beneficiaries conducted annually since 1991. Using Part B claims, mammography events were identified throughout the calendar year using previously published criteria (HCPC codes 76090, 76091 and 76092 or BETOS code I1C) to create a cohort of screened women¹⁸. The analytical sample of 15,357 respondents provided 29,533 event-interview records representing women, age 65 or older, who had a mammography event and were asked in the fall subsequent to this event about their use of mammography services between the years 1991 and 2006. Each respondent could have multiple mammography events prior to their interview; however, only the most recent mammography date was included in the analysis.

As part of the interview survey, respondents were asked, “Have you had a mammogram or breast X-ray since [today's date or previous supplement round interview date] a year ago?” Although self-reported use is measured between January and August by the MCBS, mammography events are uniformly distributed across the previous year; therefore, the time between actual use and self-reported use (i.e., recall period) is uniformly distributed. This naturally occurring randomness in the recall period allowed for the identification of the causal relationship between its length and sensitivity. Furthermore, recall periods of the same length may or may not include New Year's Eve (12/31), due to randomness in the interview and mammography event dates. This independent randomness allowed for the identification of whether including multiple calendar years in the referent period had an effect on sensitivity, in addition to recall effects. Heterogeneity was assessed by incorporating interview characteristics and demographic, socioeconomic, and health characteristics into the estimation.

Statistical Analyses: Model of Sensitivity

A modified complementary log-log regression model of sensitivity allows rate estimation using a binary outcome and accommodates static and time dependent effects:

$$Pr(\mathit{\text{yes}})=\frac{e^{\lambda }}{e^{\lambda }+1}{e}^{-rT}$$ (1)

where r is the rate of loss in sensitivity over the recall period, T, and λ is the logit transformation of the sensitivity of self-reported use conditional on the null recall period (i.e., T = 0). By estimating λ, referred to as logit sensitivity, the specification is analogous to a logit regression model, where the probability is proportional to time, T.

To estimate patterns in sensitivity, two regression equations were incorporated into equation 1, specifying a generalized form:

$$Pr(\mathit{\text{yes}})=\frac{e^{\lambda +X_1^{\text{'}}{\beta }_1}}{e^{\lambda +X_1^{\text{'}}{\beta }_1}+1}{e}^{-(r+X_2^{\text{'}}{\beta }_2)T}$$ (2)

where X₁ is a vector of control variables that may amplify or attenuate logit sensitivity, λ; and X₂ contains variables that influence the rate of loss to recall, r (See Table 1).

Table 1. Sample Description.

		Self-reported Use
	All	Yes	No	p-value
Number of Respondents	29,533	26,838	2,695
Demographics
Age in Years	75.22	75.09	76.55	0.00
Race
White	0.90	0.91	0.87	0.00
Black	0.07	0.07	0.09	0.00
Other	0.03	0.03	0.04	0.00
Hispanic	0.03	0.03	0.03	0.33
Socioeconomics
Annual Income < $25,000	0.60	0.59	0.68	0.00
Education
Less the High School	0.12	0.12	0.17	0.00
High School Graduate	0.55	0.55	0.55	0.60
Some Higher Education	0.33	0.33	0.27	0.00
Survey Attributes
Length of Referent Period in Months	12.43	12.41	12.55	0.00
Length of Recall Period in Months	5.66	5.31	9.17	0.00
New Year's Days since Mammogram	0.16	0.12	0.53	0.00
Proxy Response	0.04	0.03	0.05	0.00
Non-English Interview	0.01	0.01	0.02	0.11

Statistical Analyses began on March 11, 2008 and were completed by April 28, 2008. Database management was conducted in SAS 9.1.3²⁵. The descriptive statistics (Table 1), week-specific means (Figure 1), and maximum likelihood results were estimated using STATA MP 9.0²⁶. No sampling weights were applied in this estimation, because mammography event weights were developed for the analysis of older women. The study was reviewed and approved by the University of South Florida Institutional Review Board, which considered it exempt due to its use of publicly available limited data sets (45 CFR 46.101(b)(4)).

Figure 1. Sensitivity of Self-reported Mammography by Length of Recall Period in Weeks.

The points represent means of self-reported mammography use stratified by the number of weeks between the survey and the imaging dates. All recall periods past 52 weeks (2.86%) were excluded from this graph.

Results

Among the 29,533 mammography events, 26,838 were self-reported (i.e., sensitivity of 90.9%). Comparing respondent demographic and socioeconomic characteristics (See Table 1), women who reported a negative response (9.1%) were a year and half older and more likely to be a racial/ethnic minority than women who accurately responded. Women with negative responses also gained less income. They were more likely to have dropped out of high school and forgone higher education.

In addition to respondent differences, the evidence reflected underlying relationships between survey attributes and responses (Table 1). A negative response appeared to be related to the length of the referent period, particularly the amount of time since the mammography event. This relationship between recall and sensitivity is shown in Figure 1, a scatter plot of mean responses stratified by the length of the recall period in weeks. For those interviewed within six months of their mammogram, sensitivity was above 90%, but after one year had passed, almost 40% of the women failed to report their mammography use. This failure to report may have been attributable to aging/memory loss, recall-related bias, or the phrasing of the question, particularly in regard to the time frame of the question.

The percentage change in sensitivity attributable to length of the recall period was 1.8% for each month since mammography. The null sensitivity and rate of loss varied by survey attributes. If the mammography event occurred in a prior calendar year, the null sensitivity dropped by 8.7%, in addition to the losses related to the length of the recall period. Null sensitivity of self-reported use increased at a rate of 0.1% points per calendar year, but decreased by 2.6% or 1.9% points in cases where the interview was conducted by proxy or in a language other than English, respectively. The rate of loss was around 0.7% over the first six months, but increased to 3.6% for each month beyond the first six.

Discussion

In this study, the likelihood that a screened woman reports screening decreases by 1.8% per month of recall and by an additional 8.7% if the screening event occurred in a different calendar year. The combined evidence suggests that over a quarter of older women failed to report mammography use a year after screening.

The overall estimate (90.9%) was similar to findings from smaller, managed care studies of younger populations. Armstrong and colleagues estimated sensitivity of self-report to be 93% among the members of the Philadelphia Medicaid Managed Care Organization (N=276)²⁴. Among the members of Kaiser Permanente Colorado (N=480), sensitivity of self-report was 88.4%. Among Hispanic and non-Hispanic members of Kaiser Permanente members in Northern California (N=1,354), sensitivity was 81% and 91%, respectively¹¹. The survey question in the Northern California study asked about mammography use in the last two years, as opposed to the one year period in the Colorado and Maryland surveys.

In their analysis of MCBS responses, Fiscella, Holt and colleagues stated that the mammography question's referent period was “since last year,” which is inaccurate¹⁰. The referent period was “since [today's date] a year ago” only in the first year of the survey and these respondents (27.2% of the sample) were dropped in their study, because of insufficient claims data for the last 12 months. For the remaining respondents (73.7%), the question's referent period was “since [previous interview date] a year ago,” which ranged between 9 and 15 months. By construction, the error inherently reduced their validity estimates, and artificially increased discordance between self-report and claims-based measures. This error further demonstrates the need for more cautious review of survey questions prior to analysis and interpretation.

All respondents were asked the same question at nearly the same time with only slight variation in the length of the referent period, limiting generalizability. By interviewing only in the fall, this study may have underestimated the effect of straddling multiple calendar years (8.7% points). Had the interviews been conducted in the spring, the effect might have been larger. Lastly, this study did not examine whether breast disease was related to the most recent mammography event or whether multiple mammography events occurred during the reference period, which may increase the likelihood of recall.

Table 2. Relationship between Survey Attributes and the Sensitivity of Self-reported Mammography Use in Older Adults.

	Without Interview		With Interview
	Attributes*		Attributes**
	Estimate	p-value	Estimate	p-value
Null Sensitivity, α	0.899	0.000	0.968	0.000
Change in Null Sensitivity
New Year's Days since
Mammogram			-0.087	0.000
Calendar Year of Survey – 2006			0.001	0.000
Proxy Response			-0.026	0.000
Non-English Interview			-0.019	0.036
Monthly Rate of Loss to Recall, r	0.018	0.000	0.007	0.006
Change in Rate by Recall Period
Length
Less than 3 months			-0.001	0.530
3 to 6 months			Ref.	Ref.
6 to 9 months			0.029	0.000
9 to 12 months			0.028	0.000
More than 12 months			0.050	0.000

^*Null case for the first model represents the sensitivity of self-reported mammography use for an event that occurred six months prior to the interview.

^**Null case is further adjusted in the second model to represent the sensitivity of self-reported mammography use during a 2006 English language non-proxy interview for an event that occurred six months prior to the interview, but within the calendar year.

Table 3. Relationship between Respondent Characteristics and the Sensitivity of Self-reported Mammography Use in Older Adults^*.

	Estimate	p-value
Null Sensitivity, α	0.931	0.000
Change in Null Sensitivity
Decades Above 65 Years of Age	-0.022	0.000
Black	-0.025	0.000
Other	-0.031	0.001
Hispanic	0.002	0.807
Annual Income < $25k	-0.007	0.069
Less than High School	-0.016	0.000
Some Higher Education	0.010	0.007
Monthly Rate of Loss to Recall, r	0.011	0.000
Change in Rate of Loss to Recall
Decades Above 65 Years of Age	0.005	0.000
Black	0.005	0.003
Other	0.007	0.039
Hispanic	-0.001	0.769
Annual Income < $25k	0.003	0.012
Less than High School	0.002	0.133
Some Higher Education	-0.003	0.001

^*The null respondent characteristics represent a 65 year old white non-Hispanic woman who graduated from high school and has an annual income greater than $25,000. The inclusion of interview attributes into this model did not change the sign or statistical significance of these estimates.