Abstract
We evaluated the performance of the Fracture Risk Calculator (FRC) in 5893 men who participated in the baseline visit (March 2000 – April 2002) of the Osteoporotic Fractures in Men Study (MrOS). FRC estimates for 10-year hip and major osteoporotic (hip, clinical spine, forearm, and shoulder) fractures were calculated and compared to observed 10-year fracture probabilities. Possible enhancement of the tool’s performance when bone mineral density (BMD) was included was evaluated by comparing areas under receiver operating characteristic (AUC) curves and by Net Reclassification Improvement (NRI). 5,893 men were followed for an average of 8.4 years. For most quintiles of predicted fracture risk, the ratios of observed to predicted probabilities were close to unity. AUC improved when BMD was included (P < .001; 0.79 vs. 0.71 for hip fracture and 0.70 vs. 0.66 for major osteoporotic fracture, respectively). Using National Osteoporosis Foundation clinical treatment thresholds, BMD inclusion increased NRI significantly, 8.5% (P < .01) for hip and 4.0% (P =.01) for major osteoporotic fracture. We conclude that the FRC risk calculator calibrates well with hip and major osteoporotic fractures observed among older men. Further, addition of BMD to the FRC calculation improves the tool’s performance.
Keywords: fracture, prediction, men, risk assessment
INTRODUCTION
On-line fracture risk assessment tools such as Fracture Risk Calculator (FRC) (1) and FRAX, (2) use clinical risk factors and bone mineral density (BMD) to estimate future fracture risk. (3,4) FRAX results have been incorporated into evidence-based guidelines for osteoporosis management and treatment in the US (5) and UK. (6)
Few have examined performance of fracture risk assessment tools in men, (7,8) highlighting need for additional validation in this population that is under-diagnosed and under–treated for osteoporosis. (9) We sought to validate the FRC tool in 5893 men, aged 65 years and older, who participated in the Osteoporotic Fractures in Men Study (MrOS). (10,11) We evaluated performance of FRC by comparing its predicted 10-year fracture probabilities with those observed up to 10 years in this cohort. In addition, we examined the contribution of BMD to the tool’s performance.
METHODS
PARTICIPANTS
The Osteoporotic Fractures in Men (MrOS) Study is a prospective cohort study designed to identify predictors of fracture risk is older men. Design and recruitment have been previously described. (10,11) From March 2000 to April 2002, 5,994 community-dwelling men 65 years or older were enrolled at 6 clinical centers in the United States: Birmingham, Alabama; Minneapolis, Minnesota; Palo Alto, California; the Monongahela Valley near Pittsburgh, Pennsylvania; Portland, Oregon; and San Diego, California. Those 101 men who had used a bisphosphonate within 30 days prior to the baseline visit were not included in the current analyses.
The institutional review board at each clinical site approved the study protocol and written informed consent was obtained for all participants.
FRACTURE RISK CALCULATOR (FRC) AND 10-YEAR FRACTURE RISK ESTIMATES
The Fracture Risk Calculator (FRC) is a modification of a fracture risk model originally developed in 2000-2001at Division of Research, Kaiser Permanente Medical Care Program, Northern California Region.(12) The choice of risk factors was originally based on the NOF’s 1998 choice of key, independent risk factors (13) but later expanded to include additional risk factors. (1) Since 2005, it has been available for public use on the Foundation for Osteoporosis Research and Education (FORE) website. (4) Subsequently, in 2007, the FRC tool was further modified by incorporating updated US fracture rates, (14) by changes in some input variables (changing to either parent with hip fracture rather than mother and/or sister; adding heavy alcohol use), and by being made accessible at no charge in batch mode for large datasets.
The FRC tool uses 10-year fracture probabilities for age and gender derived from the 2006 U.S. National Inpatient Survey. (14) Specific patient characteristics (body mass index (BMI), history of fracture, parental history of hip fracture, smoking and alcohol consumption, use of corticosteroids, prevalence of rheumatoid arthritis, and secondary osteoporosis) are compared to the base population and relative risks are applied to factors that differ between the individual patient and the base population. Race/ethnicity offsets are based on published fracture risk ratios relative to Caucasian. (15) Data on age, gender, race and BMI are required. If data is missing on any of the other clinical characteristics the value for that characteristic is set to null. The tool provides 10-year risk estimates of both hip fracture and major osteoporotic fracture (hip, clinical spine, forearm, shoulder); risk estimates can be calculated with and without femoral neck bone mineral density (BMD) as an input parameter.
BONE MINERAL DENSITY
BMD (g/cm2) of the total hip and hip sub-regions was measured using dual-energy x-ray absorptiometry (DXA) (QDR 4500W, Hologic, Inc., Waltham, MA). (10) T-scores and Z-scores for the femoral neck were based on data for men obtained from the NHANES III. (16)
OTHER RISK FACTORS
All participants completed questionnaires, which included items about demographics, medical history, fracture history, parental fracture history, smoking status, and alcohol consumption. Ethnicity was self-reported. The history of any fracture after age 45 years was self-reported; excluded from these were fractures resulting from a motor vehicle accident or a fall from greater than standing height. Prescription and nonprescription medications used within the preceding 30 days were identified using the Iowa Drug Information Service (IDIS) Drug Vocabulary (College of Pharmacy, University of Iowa, Iowa City, IA). (17) Secondary osteoporosis was defined as the use of insulin or a self-report of a history of hyperthyroidism.
Bisphosphonate use occurring after the baseline visit was gathered at the Interim visit (2.0 ± 0.2 years from baseline), the Sleep visit (3.4 ± 0.5 years from baseline), Visit 2 (4.6 ± 0.4 years from baseline), Visit 3 (6.9 ± 0.4 years from baseline) and Interim Visit 2 (8.9 ± 0.4 years from baseline). Use was defined as any bisphosphonate reported to be taken within 30 days prior to any visit.
INCIDENT FRACTURE
After the baseline exam, active, surviving participants completed a questionnaire about incident fractures every four months; response rates exceeded 99%. Incident fractures were confirmed by radiology reports or radiographic images when reports were not available.
Follow-up time for the fracture analyses was truncated to 10 years to correspond with the 10-year fracture risk estimates from FRC. Thus, follow-up time ended at the time of the first fracture of that type, time of death or termination, at last contact, or after 10 years of follow-up. For those 418 (7.1%) men who started use of a bisphosphonate after the baseline visit the follow-up time was truncated to the midpoint between the last visit when they were nonusers and the first visit noting bisphosphonate use.
The fracture outcomes of interest in this analysis were hip fracture and major osteoporotic fracture (hip, wrist, shoulder, or clinical spine), excluding fractures caused by excessive trauma. (18) Those fractures with missing information on trauma status were considered low trauma, because it was assumed that if a participant had a fracture caused by excessive trauma he would have remembered it. (19)
STATISTICAL ANALYSIS
Risk factors of participants were compared across incident fracture status category using chi-square tests for categorical variables, t-tests for normally distributed continuous variables, and Wilcoxon rank-sum tests for continuous variables with skewed distributions.
The rates of men meeting or exceeding the NOF’s cost-effectiveness 10-year fracture risk thresholds of 3% for hip fracture and 20% for any one of four major osteoporotic fractures (15) were compared across age categories, both with and without BMD as an input parameter in the risk calculator. Tests for trend across age categories were performed using the Cochran-Armitage test.
Kaplan Meier product-limit estimates (PLE) were used to calculate 10-year fracture probabilities for observed events over time. These PLEs were compared to the median predicted 10-year fracture risk estimates across quintiles of predicted risk from the FRC, both with and without BMD used as an input parameter.
Logistic regression models with fracture as the outcome and the corresponding FRC 10-year fracture risk estimates as the predictor were performed. From these models, receiver operating characteristic (ROC) curves were created and the area under the curves (AUC, c-statistic) were calculated. The ROC curves for FRC fracture risk estimates with and without BMD used in the risk calculation were compared to determine if addition of BMD to the risk calculator improved the AUC.
Discrimination analyses can use the full range of data or, if available, can use clinical risk categories. We chose clinical cut points based on the NOF cost-effective treatment analyses (15) and also examined discrimination performance using a no-category approach. (20) For both approaches, risk stratification tables were created to evaluate the incremental value of adding BMD to the risk prediction calculator. (21) Reclassification after the addition of BMD to the risk calculator was assessed using Net Reclassification Improvement (NRI). (20,22,23) Comparing one model to another, the categorical NRI sums the proportions of individuals who are appropriately moved between risk categories and subtracts those moving inappropriately; these calculations are first performed separately by incident fracture status and then combined. The range of the categorical NRI is -100% to 100%, positive values indicating a net reclassification in the appropriate direction, negative values indicating inappropriate reclassification. When estimating NRI without categories, one again performs calculations separately among those fracturing and those not, evaluating the proportions whose risk score moves inappropriately and subtracting these from the proportions whose risk score moves appropriately; the overall NRI for no-category analyses can range from −200% to +200% with higher values indicating better performance.
Sensitivity analyses were performed removing those men from the analyses missing data on the any of the characteristics used by the risk calculator.
All significance levels reported were two-sided and all analyses were conducted using SAS version 9.2 (SAS Institute Inc, Cary, NC).
RESULTS
Of the 5,893 men included in this analysis, 156 (2.6%) had an incident hip fracture and 335 (5.7%) had an incident major osteoporotic fracture. The men were followed for an average (±SD) of 8.4 (2.3) years. Of those 4673 men with less than 10 years of follow-up, 1650 died and 221 terminated. Those who did not reach 10 years of follow-up had and average (±SD) follow-up time of 9.2 (0.4) years, ranging from 7.3-9.99 years. The men were predominately white (89.4%) with an average (±SD) age at enrollment of 73.6 (5.9) years. The overall BMD Z-score was +0.2, indicating a higher than expected BMD for age than the NHANES III reference range used for Hologic densitometers. (20)
Data was complete for all risk factors passed to the risk calculator for 4,297 (72.9%) men. The factors most commonly missing data were information on parental history of hip fracture (23.9%) and information on medications used for the definition of corticosteroid use and secondary osteoporosis (4.1% missing).
Table 1 shows baseline characteristics of the cohort by incident fracture status. Compared to those without an incident fracture, those fracturing were older, had lower BMD for age, and had about 50% greater likelihood of a prior fracture history (P < .05). Figures 1A and 1B show the proportions of men meeting or exceeding the NOF thresholds for cost-effective treatment (19) by age, with and without BMD being used in the FRC calculation; the proportions increase substantially with age (P trend < .001) but addition of BMD changed the proportions minimally. At all ages, many more men qualified for treatment under the 3% hip criterion than under the 20% major osteoporotic fracture criterion.
Table 1.
Baseline Characteristics by Incident Fracture Status
| Characteristic | All (n = 5893) | Hip | Major Osteoporotic | ||
|---|---|---|---|---|---|
|
| |||||
| No Fracture (n = 5737) | Fracture (n = 156) | No Fracture (n = 5558) | Fracture (n = 335) | ||
| Age, means (SD), y | 73.62 (5.86) | 73.5 (5.81) | 78.12 (6.18)c | 73.42 (5.79) | 76.87 (6.16)c |
| 64 to 69 | 1752 (29.73) | 1738 (30.29) | 14 (8.97)c | 1707 (30.71) | 45 (13.43)c |
| 70 to 74 | 1681 (28.53) | 1654 (28.83) | 27 (17.31) | 1607 (28.91) | 74 (22.09) |
| 75 to 79 | 1421 (24.11) | 1372 (23.91) | 49 (31.41) | 1323 (23.80) | 98 (29.25) |
| ≥80 | 1039 (17.63) | 973 (16.96) | 66 (42.31) | 921 (16.57) | 118 (35.22) |
| Race/ethnicity | |||||
| White | 5269 (89.41) | 5122 (89.28) | 147 (94.23) | 4958 (89.20) | 311 (92.84) |
| Black | 242 (4.11) | 240 (4.18) | 2 (1.28) | 236 (4.25) | 6 (1.79) |
| Asian | 188 (3.19) | 187 (3.26) | 1 (0.64) | 179 (3.22) | 9 (2.69) |
| Hispanic | 124 (2.10) | 121 (2.11) | 3 (1.92) | 118 (2.12) | 6 (1.79) |
| Other/unknown | 70 (1.19) | 67 (1.17) | 3 (1.92) | 67 (1.21) | 3 (0.90) |
| Body mass index, mean (SD), kg/m2a | 27.4 (3.82) | 27.43 (3.81) | 26.53 (3.83)c | 27.44 (3.79) | 26.9 (4.14)c |
| <25 | 1570 (26.64) | 1513 (26.37) | 57 (36.54)c | 1447 (26.03) | 123 (36.72)c |
| 25 to <30 | 3029 (51.40) | 2957 (51.54) | 72 (46.15) | 2886 (51.93) | 143 (42.69) |
| ≥30 | 1294 (21.96) | 1267 (22.08) | 27 (17.31) | 1225 (22.04) | 69 (20.60) |
| Fracture history (>45 years) | 1247 (21.16) | 1198 (20.89) | 49 (31.41)c | 1134 (20.41) | 113 (33.73)c |
| Parental history of hip fracture | 744 (16.60) | 725 (16.59) | 19 (16.81) | 698 (16.45) | 46 (19.09) |
| Current cigarette smoker | 205 (3.48) | 195 (3.40) | 10 (6.41)c | 191 (3.44) | 14 (4.18) |
| Oral corticosteroid use in the past 30 days | 104 (1.84) | 104 (1.89) | 0 | 102 (1.91) | 2 (0.62) |
| History of rheumatoid arthritis | 304 (5.16) | 292 (5.09) | 12 (7.69) | 279 (5.02) | 25 (7.46)c |
| Have secondary osteoporosisb | 172 (3.04) | 166 (3.01) | 6 (4.00) | 161 (3.02) | 11 (3.42) |
| ≥3 alcoholic drinks/day on average | 237 (4.03) | 233 (4.07) | 4 (2.56) | 230 (4.14) | 7 (2.10) |
| Femoral Neck BMD, mean (SD), g/cm2 | 0.79 (0.13) | 0.79 (0.13) | 0.66 (0.11)c | 0.79 (0.13) | 0.71 (0.12)c |
| Femoral Neck BMD T-score, mean (SD) | -1.12 (0.91) | -1.09 (0.90) | -1.97 (0.80)c | -1.09 (0.90) | -1.64 (0.90)c |
| Normal (>-1.0) | 2459 (41.73) | 2444 (42.60) | 15 (9.68)c | 2383 (42.88) | 76 (22.75)c |
| Low bone mass (between -1.0 and -2.5) | 3151 (53.48) | 3055 (53.25) | 96 (61.94) | 2951 (53.09) | 200 (59.88) |
| Osteoporosis (≤-2.5) | 282 (4.79) | 238 (4.15) | 44 (28.39) | 224 (4.03) | 58 (17.37) |
| Femoral Neck BMD Z-score, mean (SD) | 0.21 (0.92) | 0.23 (0.92) | -0.52 (0.82)c | 0.23 (0.92) | -0.23 (0.92)c |
Values shown as No. (%) unless otherwise indicated.P values for continuous variables with normal distributions are from a t-test, with skewed distributions from a Wilcoxon rank-sum test. P values for categorical variables from a chi-square test.
BMI values for 2 participants with missing data were set to the median value, 26.9 kg/m2.
Secondary osteoporosis is defined as having a history of hyperthyroidism or taking insulin.
P value compared to non fracture of that type < .05
Figure 1. Proportion of MrOS Cohort Meeting or Exceeding the NOF Cutpoint for Treatment Based on FRC 10-Year Fracture Probability.
The gray bars represent the FRC risk estimates without BMD in the risk calculator, white bars with BMD included in the calculator.
The median predicted FRC 10-year fracture probabilities (using BMD) for the middle quintiles were 2.4% and 6.0% for hip and major osteoporotic fracture, respectively; while observed fracture probabilities (PLE) for the middle quintiles were 2.1% and 5.5%. The tool’s predictive accuracy is further shown in Figures 2A and 2B. For most quintiles of hip and major osteoporotic fracture risk, the observed probabilities and the FRC median risk predictions were close to the line of agreement, regardless of whether or not BMD was included in the tool’s risk calculations; of the 20 quintiles examined, observed to predicted ratios for 17 were within 20% of the ideal 1.0 ratio.
Figure 2. Observed Versus Predicted Fracture Probability (%), With and Without BMD, by Quintiles of Fracture Risk.
Squares represent the predicted fracture probability without BMD used in the risk calculator, circles with BMD used in the calculator.
| Hip fracture: | Major osteoporotic fracture: |
| Without BMD: Quintile 1: <1.3% | Without BMD: Quintile 1: <4.1% |
| Quintile 2: 1.3 to <2.0% | Quintile 2: 4.1 to <5.6% |
| Quintile 3: 2.0 to <3.2% | Quintile 3: 5.6 to <7.1% |
| Quintile 4: 3.2 to <6.0% | Quintile 4: 7.1 to <11.3% |
| Quintile 5: ≥6.0% | Quintile 5: ≥11.3% |
| With BMD: Quintile 1: <0.9% | With BMD: Quintile 1: <3.5% |
| Quintile 2: 0.9 to <1.8% | Quintile 2: 3.5 to <5.1% |
| Quintile 3: 1.8 to <3.3% | Quintile 3: 5.1 to <7.3% |
| Quintile 4: 3.3 to <6.7% | Quintile 4: 7.3 to <11.3% |
| Quintile 5: ≥6.7% | Quintile 5: ≥11.3% |
Figures 3A and 3B show the ROC curves for hip and major osteoporotic fracture; the c-statistics were statistically significantly higher (P < .001) when BMD was included in the risk calculator (hip fracture, 0.71 versus 0.79; major osteoporotic fracture, 0.66 versus 0.70, for BMD excluded versus included, respectively).
Figure 3. Receiver Operating Characteristic (ROC) Curves for FRC-Predicted Fracture Risk.
The solid line represents the FRC estimates without BMD used in the risk calculator, dashed lines with BMD used in the calculator. The thick solid line is the line of no discrimination. The circles represent the NOF treatment cutpoints.
Without including BMD in the calculations, the NOF cut points correspond to sensitivities and specificities of 0.74/0.57 for hip fracture and 0.10/0.95 for major osteoporotic fracture, respectively. Using the NOF 3% hip threshold to calculate NRI, we found BMD addition improved the tool’s performance for hip fracture (overall NRI 8.5%; P < .01) and using the NOF 20% major osteoporotic fracture threshold BMD also improved the tool’s performance (overall NRI 4.0%; P = .01) (Tables 2A and 2B). The majority of improvement in classification of both hip and major osteoporotic fracture came from moving those destined to fracture from the lower risk to the higher risk category.
Table 2.
A. Assignments to Risk Categories Based on FRC 10-Year Hip Fracture Probability
|
|
|||
|---|---|---|---|
| Fracture Risk With BMD | |||
| Fracture Risk Without BMD | <3% | ≥3% | Total |
| <3% | |||
| Men, n | 2765 | 545 | 3310 |
| Events, n | 24 | 17 | 41 |
| Nonevents, n | 2741 | 528 | 3269 |
| Percentage of men with events | 0.87 | 3.12 | 1.24 |
| ≥3% | |||
| Men, n | 539 | 2043 | 2582 |
| Events, n | 4 | 110 | 114 |
| Nonevents, n | 535 | 1933 | 2468 |
| Proportion of men with events | 0.74 | 5.38 | 4.42 |
| Total | |||
| Men, n | 3304 | 2588 | 5892 |
| Events, n | 28 | 127 | 155 |
| Nonevents, n | 3276 | 2461 | 5737 |
| Percentage of men with events | 0.85 | 4.91 | 2.63 |
|
| |||
| B. Assignments to Risk Categories Based on FRC 10-Year Major Osteoporotic Fracture Probability. | |||
|
|
|||
| Fracture Risk With BMD | |||
| Fracture Risk Without BMD | <20% | ≥20% | Total |
|
| |||
| <20% | |||
| Men, n | 5479 | 123 | 5602 |
| Events, n | 280 | 22 | 302 |
| Nonevents, n | 5199 | 101 | 5300 |
| Proportion of men with events | 5.11 | 17.89 | 5.39 |
| ≥20% | |||
| Men, n | 79 | 211 | 290 |
| Events, n | 7 | 25 | 32 |
| Nonevents, n | 72 | 186 | 258 |
| Proportion of men with events | 8.86 | 11.85 | 11.03 |
| Total | |||
| Men, n | 5558 | 334 | 5892 |
| Events, n | 287 | 47 | 334 |
| Nonevents, n | 5271 | 287 | 5558 |
| Percentage of men with events | 5.16 | 14.07 | 5.67 |
Using the no-category approach, addition of BMD resulted in NRI’s of 62.9% (p<.001) and 38.2% (p<.001) for hip and for major osteoporotic fracture, respectively; both appropriate movement upward among those destined to fracture and appropriate movement downward among those not fracturing contributed to these outcomes. (Table 3)
Table 3.
Appropriate and Inappropriate Movement in Direction in FRC 10-Year Hip and Major Osteoporotic Fracture Probability When Bone Mineral Density Added To Tool (Results of No Category Analysis).
| Adding BMD to Risk Estimate Calculation: | ||||||
|---|---|---|---|---|---|---|
| Increased risk score | Decreased risk score | No change in risk score | Total | NRI | P-value | |
| Hip | ||||||
| Men, n | 2454 | 3287 | 151 | 5892 | 62.89 | <0.001 |
| Events, n | 114 | 41 | 0 | 155 | 47.10 | <0.001 |
| Non-events, n | 2340 | 3246 | 151 | 5737 | 15.79 | <0.001 |
| Major Osteoporotic | ||||||
| Fracture | ||||||
| Men, n | 2447 | 3336 | 109 | 5892 | 38.21 | <0.001 |
| Events, n | 201 | 131 | 2 | 334 | 20.96 | <0.001 |
| Non-events, n | 2246 | 3205 | 107 | 5558 | 17.25 | <0.001 |
Results were similar after sub-setting the analyses to those with complete risk factor data (data not shown).
DISCUSSION
We found that the FRC risk calculator calibrated well with both hip and major osteoporotic fractures observed among older men. Addition of BMD to the FRC calculation improved the tool’s performance assessed both by greater area under ROC curves and by increased proportions of men correctly being reclassified to either appropriate high or low risk categories or being moved in the direction of appropriate risk score.
Assessing the performance of a risk tool involves several assumptions. Tests of calibration involve grouping individuals into risk categories (e.g. quintiles or clinical value categories) and observing the fit between the median of each predicted risk category and the probability of fracture within that category. Conversely, tests of discrimination examine individuals and the correctness of risk categorization based on subsequent fracture versus no fracture status; this approach should use clinically meaningful cut points for categorization, if available--- if not, then a no-category approach can be used. Finally, area under the ROC curve examines the relationships between sensitivity and specificity across all levels of risk.
The ratio of NOF thresholds (i.e. 20% and 3%) is 6.7:1, and reflects the rate ratios of these fracture types observed in US women ages 60-65 years (14). In contrast, the ratio of these fracture categories in the MrOS cohort is 2.2 close to the ratio observed in US women after age 75 years. (14) This marked lowering of the fracture type ratios observed with increasing age is a likely explanation of the discordance we observed between the large proportion of MrOS subjects qualifying under the NOF 3% hip fracture threshold yet relatively few qualifying under the 20% NOF major osteoporotic fracture threshold.
A similar validation study of FRC was performed among women aged 50-80 years who had BMD testing in a large integrated healthcare delivery system. (1) Based on data from 94,489 women of whom 1,579 had a hip fracture, that study found FRC underestimated by 30-40% observed 10-year hip fracture probabilities, possibly due to greater likelihood of incomplete or missing input data being set to null (alcohol intake, BMI, prior fractures, family history). In the current analyses we found observed to expected fracture ratios close to 1.0 and found little difference in results when subsetting to those with complete characteristic data.
Fracture risk models have shown area under ROC curves in the range of 0.54-0.89. (1,24--31) Adding or subtracting BMD from fracture risk predictive models has been shown to change31 or not change (1,25) various model’s performance. We found adding BMD to FRC risk prediction increased the area under the ROC curves. In contrast, Lo et al, (1) using FRC in women, found that area under ROC curve was high whether or not BMD was included in the model (0.85) or when it was not used (0.83). These differences in FRC performance could be gender-related, but are more likely age-related, given that the majority of women in the Lo et al study were aged <65 years.
As shown in Table 4, relative risks used in FRC differ little from those used in FRAX. Both tools provide 10-year probabilities of both hip fracture and of any one of 4 major osteoporotic fractures (clinical spine, wrist, proximal humerus, and hip). FRC differs from FRAX in that FRAX uses more complex calculations that account for interactions of risk variables with age and also calculates fracture probability offset by expected mortality. The FRAX mortality offset becomes increasingly important in the elderly, substantially reducing fracture risk estimates obtained in those beyond age 75 years.
Table 4.
Comparison of Fracture Risk Calculator (FRC) hazard ratios and approximate FRAX hazard ratios (reverse calculated from the FRAX tool) for the average MrOS participant (age 74 years, BMI 27.4, and femoral neck bone mineral density [BMD] T-score -1.1).
| FRC | FRAX | |||
|---|---|---|---|---|
| Fracture estimate type | Hip 1 | Any Major 2 | Hip 1 | Any Major 2 |
| Age range, years | 45-90 | 45-90 | ||
| Race/ethnicity2 | ||||
| Black | 0.43 | 0.43 | ||
| Asian | 0.50 | 0.50 | ||
| Hispanic | 0.53 | 0.53 | ||
| Body Mass Index (BMI) | 1.6 if <21 | (1.7) 4,5 | (1.2) 4,5 | |
| BMD per unit T- or Z-score | 1.6 | 1.6 | 2.2 | 1.5 |
| Smoking (current) | 1.7 | 1.7 | 1.5 | 1.0 |
| Fracture after age 45 yr | 1.8 | 1.8 | 1.5 | 1.5 |
| Parent with hip fracture | 1.8 | 1.8 | 3.8 | 1.8 |
| Alcohol >3 units/day | 1.7 | 1.7 | 1.5 | 1.3 |
| Glucocortioid exposure | 2.3 | 2.3 | 1.7 | 1.5 |
| Rheumatoid arthritis | 1.8 | 1.8 | 1.4 | 1.3 |
| Secondary osteoporosis | (1.8)4 | (1.8)4 | (1.8)4 | (1.4)4 |
Hip fracture
Major osteoporotic fracture
Relative to white- (ref 19)
This factor is not considered in risk calculation when BMD is included
Per 5 unit BMI decrease
The FRC tool has the advantages of transparency and being relatively easy to incorporate into population management programs that have electronic medial records. On the other hand, FRAX has achieved world-wide acceptance and has been incorporated into bone densitometer reporting in the US. Furthermore, as FRAX gains popularity, health administrators may adapt to electronic medical record systems to accept FRAX’s input and output data.
Our study has several strengths. The baseline input data are fairly complete on a large number (nearly 6,000) of men. The follow-up for incident fracture is 99% complete and self-reported fractures were validated by review of radiograph reports; the majority of subjects had length of follow-up close to the 10-year value used by the risk calculator, thus obviating the need to extrapolate.
MrOS is an epidemiological observational study and a healthy volunteer bias could be present. The fact that the group’s mean BMD Z-score was +0.2 suggests that they are at substantially less risk for fracture (assuming RR 2.0 per SD BMD, the +0.2 Z-score translates into 40% lower fracture risk).
Limitations include drug exposure information based only on the 30 days prior to baseline and no information on glucocorticoid dosage. Further limitations in medical history and drug exposure also reduced our ability to accurately determine multiple causes of secondary bone loss. We assumed that absence of degree of fracture trauma was equivalent to low trauma; however, few subjects were missing the degree of trauma (3.2% of hip fractures and 8.1% of major osteoporotic fractures). We excluded nearly 2% of the cohort at baseline because they reported taking bisphosphonates – thus making the remaining cohort less prone to fracture than the original cohort, but those remaining also had lower FRC scores as a result of the winnowing.
As use of the electronic medical record expands, more health care organizations should be capable of inputting risk information into batch risk calculators and using the information obtained for population management. Allowing providers to obtain automated patient fracture risk estimates could help overcome the usual barriers that prevent many from performing these risk calculations. Institutions considering adoption of fracture risk tools should consider the tool’s performance based on its calibration and discrimination in the population being served.
We conclude, based on calibration and discrimination that among older men the FRC fracture risk calculator performs well and that performance of this tool may be enhanced by adding BMD to the calculations.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Hau Liu, Stanford University, Stanford, CA; Valley Medical Center, San Jose, CA.
Terri Blackwell, California Pacific Medical Center Research Institute, San Francisco, CA.
Andrew R. Hoffman, Department of Medicine, Stanford University, Stanford, CA.
Kristine E. Ensrud, Center for Chronic Disease Outcomes Research, VA Medical Center, Minneapolis, MN; Department of Medicine and Division of Epidemiology & Community Health, University of Minnesota, Minneapolis, MN.
Eric S. Orwoll, Bone and Mineral Unit, Oregon Health & Science University, Portland, OR.
References
- 1.Lo JC, Pressman AR, Chandra M, Ettinger B. Fracture risk tool validation in an integrated healthcare delivery system. Am J Manag Care. 2011;17(3):188–194. [PubMed] [Google Scholar]
- 2.Kanis JA, Oden A, Johnell O, et al. The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18(8):1033–1046. doi: 10.1007/s00198-007-0343-y. [DOI] [PubMed] [Google Scholar]
- 3. [December 1,2011]; FRAX. www.shef/ac/uk/FRAX/
- 4.FORE 10-year Fracture Risk Calculator for Health Care Professionals. [December 1, 2011]; http://riskcalculator.fore.org/
- 5.Dawson-Hughes B. A revised clinician’s guide to the prevention and treatment of osteoporosis. J Clin Endocrinol Metab. 2008;93(7):2463–2465. doi: 10.1210/jc.2008-0926. [DOI] [PubMed] [Google Scholar]
- 6.National Osteoporosis Guideline Group (NOGG); [December 1, 2011]. Guideline for the diagnosis and management of osteoporosis in postmenopausal women and men from the age of 50 years in the U.K. www.sheffield.ac.uk/NOGG/downloads.html. [Google Scholar]
- 7.Schwartz AV, Vittinghoff E, Bauer DC, et al. Study of Osteoporotic Fractures (SOF) Research Group; Osteoporotic Fractures in Men (MrOS) Research Group; Health, Aging, and Body Composition (Health ABC) Research Group. Association of BMD and FRAX score with risk of fracture in older adults with type 2 diabetes. JAMA. 2011;305(21):2184–2192. doi: 10.1001/jama.2011.715. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Berry SD, Kiel DP, Donaldson MG, et al. Application of the National Osteoporosis Foundation Guidelines to postmenopausal women and men: the Framingham Osteoporosis Study. Osteoporos Int. 2010;21(1):53–60. doi: 10.1007/s00198-009-1127-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Feldstein AC, Nichols G, Orwoll E, et al. The near absence of osteoporosis treatment in older men with fractures. Osteoporos Int. 2005;16(8):953–962. doi: 10.1007/s00198-005-1950-0. [DOI] [PubMed] [Google Scholar]
- 10.Orwoll E, Blank JB, Barrett-Connor E, et al. Design and baseline characteristics of the osteoporotic fractures in men (MrOS) study--a large observational study of the determinants of fracture in older men. Contemp Clin Trials. 2005;26(5):569–585. doi: 10.1016/j.cct.2005.05.006. [DOI] [PubMed] [Google Scholar]
- 11.Blank JB, Cawthon PM, Carrion-Petersen ML, et al. Overview of recruitment for the osteoporotic fractures in men study (MrOS) Contemp Clin Trials. 2005;26:557–568. doi: 10.1016/j.cct.2005.05.005. [DOI] [PubMed] [Google Scholar]
- 12.Ettinger B, Hillier TA, Pressman A, Che M, Hanley DA. Simple computer model for calculating and reporting 5-year osteoporotic fracture risk in postmenopausal women. J Women’s Health. 2005;14:159–171. doi: 10.1089/jwh.2005.14.159. [DOI] [PubMed] [Google Scholar]
- 13.National Osteoporosis Foundation. Osteoporosis: Review of the evidence for prevention, diagnosis, and treatment and cost-effectiveness analysis: Status Report. Osteoporos Int. 1998;8(suppl 4):1–128. [PubMed] [Google Scholar]
- 14.Ettinger B, Black DM, Dawson-Hughes B, Pressman AR, Melton LJ., 3rd Updated fracture incidence rates for the US version of FRAX. Osteoporos Int. 2010;21(1):25–33. doi: 10.1007/s00198-009-1032-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tosteson AN, Melton LJ, 3rd, Dawson-Hughes B, et al. Cost-effective osteoporosis treatment thresholds: the United States perspective. Osteoporos Int. 2008;19(4):437–447. doi: 10.1007/s00198-007-0550-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Looker AC, Wahner HW, Dunn WL, et al. Updated data on proximal femur bone mineral levels of US adults. Osteoporos Int. 1998;8(5):468–489. doi: 10.1007/s001980050093. [DOI] [PubMed] [Google Scholar]
- 17.Pahor M, Chrischilles EA, Guralnik JM, Brown SL, Wallace RB, Carbonin P. Drug data coding and analysis in epidemiologic studies. Eur J Epidemiol. 1994;10(4):405–411. doi: 10.1007/BF01719664. [DOI] [PubMed] [Google Scholar]
- 18.Cummings SR, Cawthon PM, Ensrud KE, Cauley JA, Fink HA, Orwoll ES Osteoporotic fractures in men (MrOS) Research Groups; Study of Osteoporotic Fractures Research Groups. BMD and risk of hip and nonvertebral fractures in older men: a prospective study and comparison with older women. J Bone Miner Res. 2006;21(10):1550–1556. doi: 10.1359/jbmr.060708. [DOI] [PubMed] [Google Scholar]
- 19.Mackey DC, Lui LY, Cawthon PM, et al. High-trauma fractures and low bone mineral density in older women and men. JAMA. 2007;298(20):2381–2388. doi: 10.1001/jama.298.20.2381. [DOI] [PubMed] [Google Scholar]
- 20.Pencina MJ, D’Agostino RB, Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi: 10.1002/sim.4085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–935. doi: 10.1161/CIRCULATIONAHA.106.672402. [DOI] [PubMed] [Google Scholar]
- 22.Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795–802. doi: 10.7326/0003-4819-150-11-200906020-00007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pencina MJ, D’Agostino RB, Sr, D’Agostino RB, Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–172. doi: 10.1002/sim.2929. [DOI] [PubMed] [Google Scholar]
- 24.Sornay-Rendu E, Munoz F, Delmas PD, Chapurlat RD. The FRAX tool in French women: How well does it describe the real incidence of fracture in the OFELY cohort? J Bone Miner Res. 2010;25(10):2101–2107. doi: 10.1002/jbmr.106. [DOI] [PubMed] [Google Scholar]
- 25.Bolland MJ, Siu AT, Mason BH, et al. Evaluation of the FRAX and Garvan fracture risk calculators in older women. J Bone Miner Res. 2011;26(2):420–427. doi: 10.1002/jbmr.215. [DOI] [PubMed] [Google Scholar]
- 26.Sandhu SK, Nguyen ND, Center JR, Pocock NA, Eisman JA, Nguyen TV. Prognosis of fracture: evaluation of predictive accuracy of the FRAX algorithm and Garvan nomogram. Osteoporos Int. 2010;21(5):863–871. doi: 10.1007/s00198-009-1026-7. [DOI] [PubMed] [Google Scholar]
- 27.Ensrud KE, Lui LY, Taylor BC, et al. A comparison of prediction models for fractures in older women: is more better? Arch Intern Med. 2009;169(22):2087–2094. doi: 10.1001/archinternmed.2009.404. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pressman AR, Lo JC, Chandra M, Ettinger B. Methods for Assessing Fracture Risk Prediction Models: Experience with FRAX in a large integrated healthcare delivery system. J Clin Densitometry. 2011;14(4):407–415. doi: 10.1016/j.jocd.2011.06.006. [DOI] [PubMed] [Google Scholar]
- 29.Donaldson MG, Cawthon PM, Schousboe JT, et al. Novel methods to evaluate fracture risk models. J Bone Miner Res. 2011;26(8):1767–1773. doi: 10.1002/jbmr.371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Hippisley-Cox J, Coupland C. Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractureScores. BMJ. 2009 Nov 19;339:b4229. doi: 10.1136/bmj.b4229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Leslie WD, Lix LM, Johansson H, Oden A, McCloskey E, Kanis JA. Manitoba Bone Density Program. 2010 Independent clinical validation of a Canadian FRAX tool: fracture prediction and model calibration. J Bone Miner Res. 25(11):2350–2358. doi: 10.1002/jbmr.123. [DOI] [PubMed] [Google Scholar]