Abstract
Summary
In the Study of Osteoporotic Fractures (SOF), 18.5 % of incident hip fractures identified in Medicare Fee-for-Service claims data were not reported to or confirmed by the cohort. Cognitive impairment was a modest risk factor for false-negative hip fracture ascertainment via self-report.
Introduction
Prospective cohort studies of fractures that rely on participant self-report to be the initial signal of an incident fracture could be prone to bias if a significant proportion of fractures are not self-reported.
Methods
We used data from the SOF merged with Medicare Fee-for-Service claims data to estimate the proportion of participants who had an incident hip fracture identified in Medicare claims that was either not self-reported or confirmed (by review of radiographic reports) in SOF.
Results
Between 1/1/1991 and 12/31/2007, 647 SOF participants had a hip fracture identified in Medicare claims, but 120 (18.5 %) were either not reported to or confirmed by the cohort. False-negative hip fracture ascertainment was associated with a reduced modified Mini-Mental State Exam (MMSE) score (odds ratio 1.31 per SD decrease, 95 % C.I. 1.06–1.63). Point estimates of associations of predictors of incident hip fracture were changed minimally when the misclassification of incident hip fracture status was corrected with use of claims data.
Conclusions
A substantial minority of incident hip fractures were not reported to or confirmed in the SOF. Cognitive impairment was modestly associated with false-negative hip fracture ascertainment. While there was no evidence to suggest that misclassification of incident hip fracture status resulted in biased associations of potential predictors with hip fracture in this study, false-negative incident fracture ascertainment in smaller cohort studies with limited power may increase the risk of type 2 error (not finding significant associations of predictors with incident fractures).
Keywords: Cohort studies, False-negative ascertainment of fractures, Hip fracture, Medicare claims data, Misclassification of fractures
Introduction
Many large cohort studies with fracture outcomes including the Framingham Study, the Iowa Women’s Health Study, the Study of Osteoporotic Fractures (SOF), the Women’s Health Initiative, and the Osteoporotic Fractures in Men (MrOS) study rely at least, in part, on participant self-report to be the initial signal that an incident fracture may have occurred. While confirmation of self-reported fracture events via radiology reports can verify true fracture events and identify false-positive self-reports, this process does not account for incident fractures that are not reported to the cohort. In addition, confirmation of self-reported fracture events is not possible without access to medical records, and this may be another cause of false-negative ascertainment of true fracture events. False-negative ascertainment of fracture events and the degree to which participant characteristics are associated with both fractures and false-negative ascertainment of those fractures conceivably could introduce bias into cohort studies.
While some previous studies have found self-report to be accurate, these studies have focused on false-positive hip fracture events (a positive self-report of hip fracture with no fracture documentation in medical records) and have not estimated underascertainment of hip fracture [1, 2], included a younger study population [3], sampled only a small random subset of those not reporting fractures to assess underreporting [3, 4], or have had only very small numbers of hip fracture cases and thus, were limited with respect to statistical power [5, 6].
Two studies comparing hip fracture self-reports against national fracture registries in Denmark [7] and Iceland [8] found, respectively, that 41 % and 61 % of hip fractures in the registry were not self-reported by participants in survey studies who were specifically queried about their fracture history. Women aged 65 years or older participating in the Iowa Women’s Health Study were queried about incident hip fractures every 2 years, and hip fracture incidence by self-report was 2.61 per 1,000 patient years, whereas the incidence of hip fracture identified through inpatient Medicare claims for the same participants was 4.2 per 1,000 patient years [9]. Medicare claims to verify and/or identify incident hip fractures in cohort studies is an attractive alternative to medical record review, since incident hip fractures can be identified with high accuracy through inpatient Medicare claims. The positive predictive value of a discharge diagnosis of hip fracture (ICD-9 code 820.0×) from an inpatient stay is reported to be 98 % [1, 10].
The SOF is unique in that participants are queried about incident fractures via postcard once every 4 months (with a response rate of 95 % through 2007). Hence, we hypothesized that the proportion of incident hip fractures that were either not reported to or confirmed by the cohort study would be very low. Using data from SOF linked to the available Medicare claims of SOF participants, our first aim was to estimate the proportion of hip fractures identified in Medicare administrative claims data that were also identified through the cohort study (e.g., self-report with confirmation by review of radiographic reports). The second aim was to examine whether the associations of known predictor variables with incident hip fractures identified and confirmed by the cohort study were altered when those participants with false-negative hip fracture ascertainment were reclassified as having had an incident hip fracture once their claims history was known.
Methods
From 1986 to 1988, 9,704 women at least 65 years old participated in the first examination of the prospective SOF. Women were recruited from population-based listings at four metropolitan centers in the United States (Baltimore, Minneapolis, Pittsburgh, and Portland, Oregon). The methods of recruitment have been described previously [11]. After the baseline examination, subsequent examinations were conducted approximately every 2 years.
Merging of SOF data with Medicare claims and defining the study cohort
Since outpatient Medicare claims are not available prior to 1991, we tried to match inpatient and outpatient Medicare claims for the period January 1, 1991 to December 31, 2007 for all SOF participants actively enrolled during all or part of this time period. Because >80 % of Portland participants were recruited from Kaiser Permanente Northwest and were enrolled in Medicare Advantage (MA) (and therefore did not have detailed claims data in Centers for Medicare and Medicaid Services [CMS] files), we excluded all Portland participants (n=2,421) from our analyses. We additionally excluded those who died (n=272) or terminated (n=45) from SOF prior to 1/1/1991. Among the 6,965 actively enrolled, surviving women from the other three SOF sites as of 1/1/1991, enrollment in MA during the period 1/1/91 through 12/31/07 varied from 4 to 14 % of SOF participants at Baltimore, 40–50 % of participants at Minneapolis, and 0 % in 1991 rising to 30 % of participants at Pittsburgh as of 2007 (Fig. 1).
Fig. 1.
Analytical cohort flow diagram
A finder file with one record for each with a social security number (SSN) or Medicare ID (Health Insurance Claim [HIC] number) was submitted to and used by CMS to match SOF participants in their data files. We used an algorithm similar to those previously described [12, 13] to confirm that social security or HIC number and at least two of three components of data of birth (month, day, and year) and gender agreed between SOF data and CMS claims data. By these criteria, 6,420 (92.2 %) were successfully matched to their Medicare claims, and of these, 6,003 were enrolled in FFS Medicare, both parts A and B, for at least 1 month during the period 1/1/1991 through 12/31/2007. Among these women, 5,008 women, who attended the second SOF visit in 1989–1990 had femoral neck BMD measured and had not reported a hip fracture prior to 1991 to the SOF cohort, comprised the overall study population for these analyses.
Ascertainment of incident hip fracture
In SOF, incident fractures were identified by follow-up contact with participants or proxies every 4 months by mailed postcards or phone, with a 95 % response rate to these queries through 2007. Incident hip fractures were confirmed by physician adjudication of radiographic reports. Incident hip fractures were identified in the Medicare claims MedPAR file by a primary or secondary discharge ICD-9 diagnosis code of 820.0× or 733.14. Because 733.14 can include hip fractures due to pathology other than osteoporosis (such as malignancy), we did secondary analyses excluding those cases with a primary diagnosis of 733.14 (n=13) and those cases with only a secondary diagnosis of hip fracture (n=12).
Ascertainment of predictors of hip fracture
At the SOF baseline examination, participants were queried about self-rated health status, height at age 25, and history of stomach surgery, diabetes, and Parkinson’s disease. Cognition was assessed by a modified Mini-Mental State Examination (MMSE) [14, 15], and depth perception was assessed. As described in previous publications [16, 17], at the SOF visit 2 examination, height, weight, contrast sensitivity for vision, and femoral neck bone mineral density (BMD) using dual energy x-ray absorptiometry (DXA) were measured. Participants were queried about clinical fractures since age 50, history of chronic obstructive pulmonary disease, current smoking status, current use of thyroid medications, and social support (living alone). The ability to stand at least five times from a chair without using one’s arms was evaluated by direct observation. Participants’ usual walking speed over a 6-m distance was measured in meters per second. The change of weight (kilograms) from age 25 to the time of each examination was calculated from self-reported body weight at age 25 and measured weight at visit 2. Body mass index (BMI) was calculated as the ratio of current weight (kilograms) to height at age 25 (in meters) squared. Participants also completed assessments of executive functioning using the Trails B test, Independent Activities of Daily Living (IADL’s), and depressive symptoms using the Geriatric Depression Scale.
Statistical analysis
A cross tabulation was performed of the proportions of SOF participants who had incident hip fractures reported to and confirmed by the cohort versus those who had incident hip fractures identified in Medicare claims. True-positive hip fracture ascertainments were defined as hip fractures identified in Medicare claims that were also self-reported to SOF and confirmed by review of reports of x-rays taken within 31 days of the first hip fracture admission date in claims. False-negative hip fracture ascertainments were a first hip fracture identified in claims with no self-reported and confirmed hip, pelvis, proximal femur, or distal femur fractures on follow-up postcards in SOF within 31 days of the fracture date in claims. In this analysis, we restricted the study population to those 702 women who had a claims-identified fracture and who were concurrently enrolled in SOF at the time of their hip fracture, so that there would be full potential for the fracture to be reported to and confirmed in SOF.
We estimated the associations of false-negative hip fracture ascertainment compared to true-positive hip fracture ascertainment with the variables that had been associated with incident hip fracture risk in prior published studies from SOF [16, 18] because the greatest potential for bias would be for those specific variables associated with both incident hip fracture status and false-negative ascertainment of these fractures. Candidate variables were age at the second SOF visit, 6-m walk time, chair stand speed, MMSE score, living with others, BMI, height at age 25, prior clinical fracture since age 50, self-reported health status, IADL’s, smoking status, femoral neck BMD, study site, tertiles of follow-up time period (<7 years, 7–11 years, ≥12 years), and an interaction term between study site and follow-up time.
The multivariable-adjusted associations of predictor variables with false-negative hip fracture ascertainment were estimated with logistic regression. Age, study site, follow-up time period, an interaction term between study site and follow-up time period were forced into the models and then other candidate predictors with an unadjusted association with false-negative hip fracture ascertainment at a p value <0.2 were simultaneously added to the full multivariable model.
To estimate the bias that may be present in the estimated age- and femoral neck BMD-adjusted associations of predictors with incident hip fractures, we compared the point estimates and confidence intervals of these associations from four sets of proportional hazards models, adjusted for study site, follow-up time period, and the interaction between study site and follow-up time period. For the first set (Aim 2a analyses), the dependent variable was incident hip fracture identified and confirmed in SOF between January 1, 1991 and December 31, 2007. Individuals were censored at death, termination from the cohort (typically at participant or family request or after repeated failed attempts to contact them), incident hip fracture, or end of study period (12/31/2007). For the second set of models, the dependent variable was incident hip fracture identified in either SOF or Medicare claims from the Baltimore, Minneapolis, or Pittsburgh sites. Individuals were censored only at incident hip fracture, death, or end of study period (12/31/2007), since hip fractures might be identified in Medicare claims after termination of participation from SOF. We used the comparisons between these two sets of regressions to examine practically how the estimated associations between predictors and incident hip fracture may be altered when a cohort study such as SOF is able to augment cohort data with FFS Medicare claims data. We included predictors that had been previously reported to be associated with or nearly associated with incident hip fracture [16, 18].
However, because a significant proportion of the incident hip fractures were reported to SOF during time periods when participants at the three sites (Minneapolis, Baltimore, or Pittsburgh) were enrolled in MA, the first two regressions still both misclassify those enrolled in MA who have a hip fracture and do not self-report it or do not have it confirmed by the cohort. The comparison of those two models does not fully address the issue of how large the bias may truly be from false-negative ascertainment hip fracture self-reports. Therefore, the two regressions described above were repeated including only those 4,248 active SOF participants who were alive and enrolled in FFS Medicare as of 1/1/1991, such that hip fractures identified in Medicare claims would be a better “gold standard” against which we could compare hip fractures identified through self-report and confirmed by the cohort study (Aim 2b analyses). Individuals were censored at death, incident hip fracture, the end of the study period, or enrollment into MA.
Results
Of the 5,008 women in the study population for these analyses, there were 655 women with a SOF-adjudicated hip fracture and 704 women with a claims-identified hip fracture during follow-up, who did not have a fracture detected in SOF prior to 1/1/1991. Ninety-one women with a SOF-adjudicated hip fracture did not have a Medicare claim for that fracture largely because they were enrolled in MA at the time of their fracture and are excluded from these analyses. One hundred sixty-six women had a Medicare claim for hip fracture that were not reported to and/or confirmed in SOF.
Of the 704 hip fracture claims, 28 occurred after termination from the SOF study (Fig. 2a). From the remainder of 676, we excluded 29 women for which review medical record radiographic data collected in SOF revealed that the presence of a proximal femur fracture was uncertain or that the fracture location was the pelvis or distal femur.
Fig. 2.
a Flow diagram of hip fractures for Aim 1 analysis. b Flow diagram of hip fractures for Aim 2 analyses. *Fracture location uncertain, or in pelvis or distal femur
Of the remaining 647 hip fractures identified in Medicare claims, 527 (81.5 %) were reported and confirmed in the cohort study (e.g., true positives), yielding a false-negative hip fracture ascertainment rate of 18.5 % (Table 1). Of the 120 false-negative hip fracture ascertainments among participants actively enrolled in SOF at the time of the hip fracture claim, 105 were not self-reported to the cohort and 15 were due to the inability of the cohort to obtain confirmatory radiographs or to data recording error.
Table 1.
Characteristics of hip fracture patients’ false-negative hip fracture ascertainment compared to those with true positive ascertainmentc
| Predictor | True positives (N=527) | False negatives (N=120) | P value |
|---|---|---|---|
| 6-m walk speed tertiles, (m/s, %) | |||
| <0.79 | 31.6 | 31.4 | 0.751 |
| 0.79–0.96 | 35.3 | 32.2 | |
| ≥0.97 | 33.1 | 36.4 | |
| MMSE scorea, 0–26, mean (SD) | 24.5 (1.7) | 24.3 (1.7) | 0.190 |
| Trails B score, 0–180 (s), mean (SD) | 144.1 (73.7) | 142.0 (72.9) | 0.780 |
| Geriatric Depression Scale scoreb, mean (SD) | 1.73 (2.19) | 1.70 (2.08) | 0.968 |
| Lives with others vs. lives alone (%) | 48.1 | 60.2 | 0.021 |
| Body mass index (kg/cm2), mean (SD) | 25.1 (4.2) | 25.6 (4.4) | 0.217 |
| Height at age 25a (cm) mean (SD) | 163.1 (5.9) | 162.6 (5.8) | 0.386 |
| Clinical fractures since age 50 (%) | 49.7 | 45.0 | 0.351 |
| Self-reported health status (very poor/poor/fair vs. good/excellent)a (%) | 14.4 | 18.3 | 0.281 |
| Any IADL impairment (%) | 42.5 | 39.5 | 0.554 |
| Femoral neck BMD (g/cm2) mean (SD) | 0.591 (0.090) | 0.611 (0.093) | 0.028 |
| Depth perception (lowest quartile vs. top three quartiles)a (%) | 23.8 | 27.4 | 0.417 |
All measurements were obtained at SOF exam 2, unless otherwise noted
MMSE Mini-Mental Status Exam, IADL Instrumental Activities of Daily Living, BMD bone mineral density
Recorded at SOF exam 1
Skewed normal distribution, Kruskall–Wallis test performed to test differences
For the following variables, there was no difference in the frequency distributions between those with true-positive hip fracture ascertainment and those with false-negative hip fracture ascertainment; age group (<80 years, 80–84 years, ≥85 years), p value=0.208; current smoking status, p value=0.329; maternal history of hip fracture, p value=0.606; or proportion able to stand without use of the arms, p value=0.824. Values are not reported in the table due to some cell sizes being less than 11
Predictors of false-negative hip fracture self-reports (Aim 1 analysis)
Compared to the 527 women with true-positive hip fracture ascertainment, the 120 women with false-negative hip fracture ascertainment had higher femoral neck BMD, slightly lower MMSE scores, and were marginally more likely to be living with others (Table 1). After multivariable adjustment (including adjustment for age, study site, follow-up time period, and the interaction of site and follow-up time (Table 2), only lower MMSE score (odds ratio 1.31, 95 % CI 1.06–1.63 for each standard deviation decrease in score) was independently associated with false-negative hip fracture ascertainment. The c-statistic of this regression was 0.72, indicating that only a modest proportion of the phenomenon of false-negative hip fracture ascertainment was explained by the predictors in these models. A sensitivity analysis excluding 13 participants with a hip fracture diagnosis of 733.14 and 12 participants with only a secondary diagnosis of hip fracture showed virtually identical results (data not shown).
Table 2.
Multivariable-adjusted associations of variables with false-negative vs. true-positive hip fracture ascertainment
| Predictor | Odds of false-negative vs. true-positive hip fracture self-report (95 % CI) |
|---|---|
| Age category | |
| <80 years (Reference) | 1.00 |
| 80–84 years | 0.97 (0.49–1.92) |
| ≥85 years | 0.68 (0.23–1.96) |
| Femoral neck BMD (per 1 SD decrease) | 0.97 (0.78–1.21) |
| MMSE score (per 1 SD decrease) | 1.31 (1.06–1.63) |
| Living with others (vs. living alone) | 1.51 (0.96–2.36) |
Adjusted for SOF study site, follow-up time period, and site * time interaction. Associations significant at P<0.05 are italicized BMD bone mineral density, SD standard deviation, MMSE Mini-Mental Status Examination
Associations of predictors with hip fractures self-reported to the cohort and with hip fractures in Medicare claims (Aim 2 analyses)
Between January 1, 1991 and December 31, 2007, 655 of 5,008 women in our analytic cohort self-reported a first hip fracture and had that fracture confirmed by SOF (Fig. 2b). First hip fractures were identified for an additional 166 women in claims that were not reported to or confirmed in SOF (some of whom experienced their hip fracture after terminating from the study). In general, the age-, and femoral neck BMD-adjusted point estimates and confidence intervals for associations of predictors with incident hip fractures reported to SOF were not substantially altered when the 166 individuals with a Medicare claim for hip fracture that had not been reported to and confirmed by SOF were reclassified as having had an incident hip fracture (Table 3). The age-adjusted rate of incident hip fracture was 0.00876/patient-year using confirmed self-reports of hip fracture only and was 0.01128/patient-year when the additional 166 women with a Medicare claim for hip fracture were reclassified as having an incident hip fracture.
Table 3.
Age-, site-, and FN BMD-adjusted hazard ratios (95 % CI) of predictors with incident hip fractures identified through SOF only vs. SOF or claims
| Predictor | Hazard ratio (95 % CI)
|
|
|---|---|---|
| Incident hip fracture in SOF (since 1/1/1991) (N=655 fractures) | Incident hip fracture in SOF or claims (since 1/1/1991) (N=821 fractures) | |
| 6-m walk time: tertiles | ||
| 1st tertile | 1.54 (1.25–1.89) | 1.64 (1.37–1.97) |
| 2nd tertile | 1.24 (1.02–1.52) | 1.22 (1.02–1.45) |
| 3rd tertile | 1.0 (ref) | 1.0 (ref) |
| Used arms to stand up from a chair (yes/no) | 1.25 (0.76–2.06) | 1.37 (0.87–2.14) |
| GDS depression score (+2.2) (higher = worse) | 1.09 (1.01–1.18) | 1.10 (1.03–1.18) |
| GDS ≥6, depressed (yes/no) | 0.87 (0.55–1.35) | 0.91 (0.62–1.35) |
| MMSE score, (−1.6 U) | 1.08 (1.01–1.16) | 1.13 (1.06–1.21) |
| Trails B score (+64 s) (using trimmed trails B variable) | 1.23 (1.14–1.32) | 1.25 (1.17–1.33) |
| Lives with others (yes/no) | 1.24 (1.05–1.47) | 1.17 (1.01–1.36) |
| Femoral neck BMD (−0.11 g/cm3) | 2.14 (1.94–2.35) | 2.16 (1.97–2.35) |
| Body mass index (−4.6 kg/m2) | 1.06 (0.97–1.17) | 1.03 (0.95–1.12) |
| Change in weight since age 25 (−19.5 %) | 1.09 (0.99–1.20) | 1.05 (0.97–1.15) |
| Prior clinical fractures (yes/no) | 1.23 (1.05–1.44) | 1.20 (1.05–1.38) |
| Fair/poor/very poor vs. good/excellent health status | 1.08 (0.86–1.34) | 1.17 (0.96–1.42) |
| IADL score (+1) | 1.11 (1.03–1.18) | 1.11 (1.04–1.18) |
| Any IADL impairment (yes/no) | 1.30 (1.10–1.53) | 1.31 (1.14–1.52) |
| Current smoker (yes/no) | 1.30 (0.98–1.72) | 1.17 (0.90–1.52) |
| Medical conditions (yes/no) | ||
| History of stomach surgery | 1.61 (0.97–2.70) | 1.71 (1.09–2.70) |
| Use of thyroid hormone pills | 1.25 (1.00–1.56) | 1.26 (1.03–1.55) |
| Type II diabetes mellitus | 1.58 (1.15–2.17) | 1.72 (1.30–2.27) |
| COPD | 1.26 (0.98–1.63) | 1.22 (0.96–1.54) |
| Parkinson’s disease | 1.86 (0.92–3.75) | 1.77 (0.91–3.42) |
| Lowest quartile distant depth perception | 1.32 (1.11–1.58) | 1.32 (1.13–1.55) |
| Contrast sensitivity (per 1 SD decrease) | 1.10 (0.82–1.48) | 1.13 (0.86–1.48) |
N for analyses is 5,008 women alive, enrolled in SOF, and who had femoral neck BMD measured at the second SOF visit. All analyses were adjusted for age, femoral neck BMD, and clinic site. Associations significant at P<0.05 level are italicized
When the analysis was restricted to those 4,248 women continuously enrolled in FFS Medicare from January 1, 1991 onward (such that all hip fractures could be detected in claims), 472 of 602 hip fractures identified in claims were self-reported to and confirmed by the cohort and 130 additional hip fractures were identified in claims (Fig. 2b). In general, the age- and femoral neck BMD-adjusted point estimates and confidence intervals of associations of predictors with incident hip fractures reported to SOF again were minimally changed when the additional 130 women with a Medicare claim for hip fracture were reclassified as having had an incident hip fracture (Table 4). Sensitivity analyses excluding 13 participants with a hip fracture diagnosis of 733.14 and 12 participants with only a secondary diagnosis of hip fracture showed virtually identical results (data not shown).
Table 4.
Age-, site-, and FN BMD-adjusted hazard ratios (95 % CI) of predictors with incident hip fractures identified through SOF only vs. SOF or claims among FFS enrollees
| Predictor | Hazard ratio (95 % CI)
|
|
|---|---|---|
| Incident hip fracture in SOF (N=472 fractures) | Incident hip fracture in SOF or claims (N=602 fractures) | |
| 6-m walk time: tertiles | ||
| 1st tertile | 1.58 (1.23–2.02) | 1.52 (1.23–1.87) |
| 2nd tertile | 1.30 (1.02–1.66) | 1.18 (0.95–1.46) |
| 3rd tertile | 1.0 (ref) | 1.0 (ref) |
| Used arms to stand up from a chair (yes/no) | 1.26 (0.73–2.16) | 1.45 (0.90–2.33) |
| GDS depression score (+2.2) (higher= worse) | 1.10 (1.00–1.20) | 1.09 (1.01–1.18) |
| GDS ≥6, depressed (yes/no) | 0.82 (0.49–1.38) | 0.89 (0.58–1.39) |
| MMSE score, (−1.6 U) | 1.07 (0.99–1.17) | 1.15 (1.07–1.24) |
| Trails B score (+64 s) (using trimmed trails B variable) | 1.18 (1.08–1.29) | 1.19 (1.10–1.28) |
| Lives with others (yes/no) | 1.32 (1.08–1.60) | 1.22 (1.03–1.45) |
| Femoral neck BMD (−0.11 g/cm3) | 2.16 (1.92–2.43) | 2.23 (2.01–2.48) |
| Body mass index (−4.6 kg/m2) | 1.03 (0.93–1.15) | 1.01 (0.91–1.11) |
| Change in weight since age 25 (−19.5 %) | 1.10 (0.98–1.23) | 1.05 (0.96–1.16) |
| Prior clinical fractures (yes/no) | 1.22 (1.01–1.47) | 1.18 (1.00–1.39) |
| Fair/poor/very poor vs. good/excellent health status | 1.03 (0.80–1.32) | 1.06 (0.85–1.32) |
| IADL score (+1) | 1.12 (1.04–1.21) | 1.09 (1.02–1.17) |
| Any IADL impairment (yes/no) | 1.34 (1.10–1.61) | 1.31 (1.11–1.55) |
| Current smoker (yes/no) | 1.32 (0.96–1.81) | 1.07 (0.79–1.43) |
| Medical conditions (yes/no) | ||
| History of stomach surgery | 1.32 (0.68–2.56) | 1.37 (0.77–2.44) |
| Use of thyroid hormone pills | 1.39 (1.07–1.80) | 1.41 (1.11–1.77) |
| Type II diabetes mellitus | 1.70 (1.21–2.39) | 1.74 (1.29–2.34) |
| COPD | 1.13 (0.84–1.52) | 1.09 (0.84–1.43) |
| Parkinson’s disease | 1.98 (0.88–4.46) | 2.11 (0.99–4.49) |
| Lowest quartile distant depth perception | 1.23 (1.00–1.50) | 1.22 (1.02–1.46) |
| Contrast sensitivity (per 1 SD decrease) | 1.10 (0.76–1.59) | 1.12 (0.80–1.56) |
N for analyses is 4,248 alive, enrolled in SOF, who had femoral neck BMD measured at the second SOF visit, and enrolled in Medicare Fee for Service on 1/1/1991. All analyses were adjusted for age, femoral neck BMD, and clinic site. Associations significant at P<0.05 level were italicized
Discussion
In the absence of administrative claims or study design with a case-finding mechanism that allows for capturing comprehensive medical record data on all participants, prospective cohort studies generally rely on self-report as the initial signal that a participant may have experienced an incident-adverse health event such as a fracture. In spite of intense, frequent follow-up in the SOF prospective cohort study to solicit self-reports of incident fractures, 18.5 % of incident hip fractures documented in Medicare claims were either not reported by cohort members or not confirmed by the study. A few of the hip fracture events identified only in Medicare claims are instances of miscoding of distal femur and pelvis fractures. However, we found that only 19 of 702 women (2.7 %) with first hip fractures identified in claims data had the fracture confirmed to be at a different skeletal site based on review of x-ray reports collected as part of the fracture adjudication process in SOF.
Impaired cognitive status, specifically a lower MMSE score, was associated with a higher likelihood of false-negative hip fracture ascertainment. Other variables that we postulated would be associated with false-negative hip fracture ascertainment, such as depression, inability to rise from a chair without use of the arms, and slower walk speed, were not significantly associated with false-negative hip fracture ascertainment. Models that include a large number of variables that together constitute a rich phenotypic description of the SOF population explained only a modest proportion of the phenomenon of false-negative ascertainment of hip fracture.
With misclassification (predominantly underascertainment) of a substantial proportion of outcome events, there is the potential for loss of statistical power in the estimation of associations between candidate risk factors and incident fractures. Additionally, there is potential for bias for variables that are associated with both the outcome itself and false-negative ascertainment of that outcome. Nonetheless, when available claims data were used to reclassify those with a Medicare claim for hip fracture undetected in the cohort data as truly having had a hip fracture, there was only a slight narrowing of the 95 % confidence intervals for estimated associations of predictors with incident hip fracture, and point estimates of the associations between predictors and incident hip fracture were not substantially altered. The only modest exception was MMSE, the only multivariable-adjusted significant predictor of false-negative hip fracture ascertainment that we could find. The point estimate of the association of MMSE with SOF-identified incident hip fractures was only 1.09 and insignificant, but the estimated excess risk nearly doubled (HR 1.17) and became significant after hip fracture status reclassification using claims data.
These results show that the vast majority of estimated associations of predictors with incident hip fractures from the SOF cohort have been reported accurately in prior publications. Moreover, the process of reviewing radiographic reports for those who do self-report a fracture in SOF identifies false-positive self reports, ensuring a specificity of 100 % for incident fracture identification in SOF. However, large numbers of women in the SOF cohort experienced an incident hip fracture and the potential for missing significant associations between predictors and hip fracture would be greater in smaller cohort studies with a lower number of outcome events.
The use of claims data in the Medicare population is not a complete solution to this problem due to the costs of obtaining, processing, and analyzing claims and the fact that reliable claims for those enrolled in MA plans are not generally available for research purposes from CMS. There is, of course, the possibility that claims data could be made available through collaboration with various health care organizations that offer MA health plans.
There are other important limitations to our study. SOF enrolled only Caucasian women until the late 1990s, and our results may not be applicable to men or non-Caucasian women. Our data is not applicable to other outcomes events such as cardiovascular or neoplastic diseases that were not rigorously ascertained by SOF. The study did not systematically record if a family member or other proxy filled out the 4-month follow-up questionnaires, and hence, we cannot ascertain if proxy reports were associated with false-negative hip fracture self-reports. Finally, we were not able to obtain medical records to confirm the hip fractures identified only in Medicare claims data that were not self-reported; however, the false-positive rate of the claims data was very small (approximately 3 %) for those hip fractures in claims data for which SOF did have medical records available.
In conclusion, false-negative ascertainment occurred in 18.5 % of women with incident hip fractures followed over a 17-year period even when self-report was frequently and vigorously pursued. Cognitive impairment is modestly associated with false-negative hip fracture ascertainment via self-report. In a cohort study with a large sample size and a high rate of completion of follow-up contacts, there may not be any significant bias from this incident hip fracture misclassification, but in smaller less robustly powered studies, there may be a risk of type 2 error (missing associations of predictors with incident hip fracture) if self-report is used to identify incident fractures. In these instances, supplementing self-report with another source of fracture event identification such claims data should be considered. Further studies of the proportion and predictors of false-negative ascertainment of other fracture and nonfracture health events are needed in order to better understand the causes of and impact of this phenomenon in epidemiologic cohort studies.
Acknowledgments
Source of funding The Study of Osteoporotic Fractures (SOF) is supported by the National Institutes of Health funding. The National Institute on Aging (NIA) provides support under the following grant numbers: R01 AG005407, R01 AR35582, R01 AR35583, R01 AR35584, R01 AG005394, R01 AG027574, and R01 AG027576. Dr. Curtis receives support from the NIH (AR053351) and the Agency for Healthcare Research and Quality (R01HS018517). The purchase of the linked CMS administrative claims data was supported, in part, by a grant to Dr. Curtis from the Arthritis Foundation.
Footnotes
Disclaimer The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.
Conflicts of interest None.
Contributor Information
J. T. Schousboe, Email: schouj@parknicollet.com, Park Nicollet Institute, Minneapolis, MN, USA. Division of Health Policy and Management, University of Minnesota, Minneapolis, MN, USA
M. L. Paudel, Division of Epidemiology, University of Minnesota, Minneapolis, MN, USA
B. C. Taylor, Division of Epidemiology, University of Minnesota, Minneapolis, MN, USA. Center for Chronic Diseases Outcomes Research, Department of Veterans Affairs Health Care System, Minneapolis, MN, USA. Department of Medicine, University of Minnesota, Minneapolis, MN, USA
B. A. Virnig, Division of Health Policy and Management, University of Minnesota, Minneapolis, MN, USA
J. A. Cauley, Department of Epidemiology, University of Pittsburgh, Pittsburgh, PA, USA
J. R. Curtis, Division of Clinical Immunology and Rheumatology, University of Alabama at Birmingham, Birmingham, AL, USA
K. E. Ensrud, Division of Epidemiology, University of Minnesota, Minneapolis, MN, USA. Center for Chronic Diseases Outcomes Research, Department of Veterans Affairs Health Care System, Minneapolis, MN, USA. Department of Medicine, University of Minnesota, Minneapolis, MN, USA
References
- 1.Bergmann MM, Byers T, Freedman DS, et al. Validity of self-reported diagnoses leading to hospitalization: a comparison of self-reports with hospital records in a prospective study of American adults. Am J Epidemiol. 1998;147(10):969–977. doi: 10.1093/oxfordjournals.aje.a009387. [DOI] [PubMed] [Google Scholar]
- 2.Chen Z, Kooperberg C, Pettinger MB, et al. Validity of self-report for fractures among a multiethnic cohort of postmenopausal women: results from the Women’s Health Initiative observational study and clinical trials. Menopause. 2004;11(3):264–274. doi: 10.1097/01.gme.0000094210.15096.fd. [DOI] [PubMed] [Google Scholar]
- 3.Honkanen K, Honkanen R, Heikkinen L, et al. Validity of self-reports of fractures in perimenopausal women. Am J Epidemiol. 1999;150(5):511–516. doi: 10.1093/oxfordjournals.aje.a010040. [DOI] [PubMed] [Google Scholar]
- 4.Ismail AA, O’Neill TW, Cockerill W, et al. Validity of self-report of fractures: results from a prospective study in men and women across Europe. EPOS Study Group. European Prospective Osteoporosis Study Group. Osteoporos Int. 2000;11(3):248–254. doi: 10.1007/s001980050288. [DOI] [PubMed] [Google Scholar]
- 5.Beard CM, Melton LJ, 3rd, Cedel SL, et al. Ascertainment of risk factors for osteoporosis: comparison of interview data with medical record review. J Bone Miner Res. 1990;5(7):691–699. doi: 10.1002/jbmr.5650050705. [DOI] [PubMed] [Google Scholar]
- 6.Ivers RQ, Cumming RG, Mitchell P, et al. The accuracy of self-reported fractures in older people. J Clin Epidemiol. 2002;55 (5):452–457. doi: 10.1016/s0895-4356(01)00518-2. [DOI] [PubMed] [Google Scholar]
- 7.Hundrup YA, Hoidrup S, Obel EB, et al. The validity of self-reported fractures among Danish female nurses: comparison with fractures registered in the Danish National Hospital Register. Scand J Public Health. 2004;32(2):136–143. doi: 10.1080/14034940310017490. [DOI] [PubMed] [Google Scholar]
- 8.Siggeirsdottir K, Aspelund T, Sigurdsson G, et al. Inaccuracy in self-report of fractures may underestimate association with health outcomes when compared with medical record based fracture registry. Eur J Epidemiol. 2007;22(9):631–639. doi: 10.1007/s10654-007-9163-9. [DOI] [PubMed] [Google Scholar]
- 9.Virnig B, Durham SB, Folsom AR, et al. Linking the Iowa Women’s Health Study cohort to Medicare data: linkage results and application to hip fracture. Am J Epidemiol. 2010;172 (3):327–333. doi: 10.1093/aje/kwq111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ray WA, Griffin MR, Fought RL, et al. Identification of fractures from computerized Medicare files. J Clin Epidemiol. 1992;45(7):703–714. doi: 10.1016/0895-4356(92)90047-q. [DOI] [PubMed] [Google Scholar]
- 11.Cummings SR, Black DM, Nevitt MC, et al. Appendicular bone density and age predict hip fracture in women. The Study of Osteoporotic Fractures Research Group. Jama. 1990;263(5):665–668. [PubMed] [Google Scholar]
- 12.Kushi LH, Kaye SA, Folsom AR, et al. Accuracy and reliability of self-measurement of body girths. Am J Epidemiol. 1988;128(4):740–748. doi: 10.1093/oxfordjournals.aje.a115027. [DOI] [PubMed] [Google Scholar]
- 13.Potosky AL, Riley GF, Lubitz JD, et al. Potential for cancer related health services research using a linked Medicare-tumor registry database. Med Care. 1993;31(8):732–748. [PubMed] [Google Scholar]
- 14.Magaziner J, Bassett SS, Hebel JR. Predicting performance on the Mini-Mental State Examination. Use of age- and education-specific equations. J Am Geriatr Soc. 1987;35(11):996–1000. doi: 10.1111/j.1532-5415.1987.tb04002.x. [DOI] [PubMed] [Google Scholar]
- 15.Teng EL, Chui HC. The Modified Mini-Mental State (3MS) examination. J Clin Psychiatry. 1987;48(8):314–318. [PubMed] [Google Scholar]
- 16.Taylor BC, Schreiner PJ, Stone KL, et al. Long-term prediction of incident hip fracture risk in elderly white women: study of osteo-porotic fractures. J Am Geriatr Soc. 2004;52(9):1479–1486. doi: 10.1111/j.1532-5415.2004.52410.x. [DOI] [PubMed] [Google Scholar]
- 17.Schousboe JT, Fink HA, Taylor BC, et al. Association between self-reported prior wrist fractures and risk of subsequent hip and radiographic vertebral fractures in older women: a prospective study. J Bone Miner Res. 2005;20(1):100–106. doi: 10.1359/JBMR.041025. [DOI] [PubMed] [Google Scholar]
- 18.Cummings SR, Nevitt MC, Browner WS, et al. Risk factors for hip fracture in white women. Study of Osteoporotic Fractures Research Group. N Engl J Med. 1995;332(12):767–773. doi: 10.1056/NEJM199503233321202. [DOI] [PubMed] [Google Scholar]