Systematic review and meta-analysis of the performance of clinical risk assessment instruments for screening for osteoporosis or low bone density

Abstract

Summary

We performed a systematic review and meta-analysis of the performance of clinical risk assessment instruments for screening for DXA-determined osteoporosis or low bone density. Commonly evaluated risk instruments showed high sensitivity approaching or exceeding 90 % at particular thresholds within various populations but low specificity at thresholds required for high sensitivity. Simpler instruments, such as OST, generally performed as well as or better than more complex instruments.

Introduction

The purpose of the study is to systematically review the performance of clinical risk assessment instruments for screening for dual-energy X-ray absorptiometry (DXA)-determined osteoporosis or low bone density.

Methods

Systematic review and meta-analysis were performed. Multiple literature sources were searched, and data extracted and analyzed from included references.

Results

One hundred eight references met inclusion criteria. Studies assessed many instruments in 34 countries, most commonly the Osteoporosis Self-Assessment Tool (OST), the Simple Calculated Osteoporosis Risk Estimation (SCORE) instrument, the Osteoporosis Self-Assessment Tool for Asians (OSTA), the Osteoporosis Risk Assessment Instrument (ORAI), and body weight criteria. Meta-analyses of studies evaluating OST using a cutoff threshold of <1 to identify US postmenopausal women with osteoporosis at the femoral neck provided summary sensitivity and specificity estimates of 89 % (95%CI 82–96 %) and 41 % (95%CI 23–59 %), respectively. Meta-analyses of studies evaluating OST using a cutoff threshold of 3 to identify US men with osteoporosis at the femoral neck, total hip, or lumbar spine provided summary sensitivity and specificity estimates of 88 % (95%CI 79–97 %) and 55 % (95%CI 42–68 %), respectively. Frequently evaluated instruments each had thresholds and populations for which sensitivity for osteoporosis or low bone mass detection approached or exceeded 90 % but always with a trade-off of relatively low specificity.

Conclusions

Commonly evaluated clinical risk assessment instruments each showed high sensitivity approaching or exceeding 90 % for identifying individuals with DXA-determined osteoporosis or low BMD at certain thresholds in different populations but low specificity at thresholds required for high sensitivity. Simpler instruments, such as OST, generally performed as well as or better than more complex instruments.

Introduction

Osteoporosis affects over 200 million people worldwide and is associated with significant costs, morbidity, and mortality secondary to fractures [1–4]. Osteoporosis is underdiagnosed and undertreated, despite availability of effective treatments [5–7]. Osteoporosis screening is recommended by many national clinical practice guidelines [8].

The gold standard test for diagnosing osteoporosis is dual-energy X-ray absorptiometry (DXA), which measures bone mineral density (BMD). Evidence of osteoporosis treatment efficacy to reduce fracture risk has been shown in clinical trials for individuals with osteoporosis by DXA criteria or prior osteoporotic fracture. For this reason, DXA testing is typically done to identify individuals who have not previously experienced an osteoporotic fracture who are most likely to benefit from treatment. In recent years, there has been interest in the use of osteoporosis clinical risk assessment instruments as an initial prescreening tool prior to DXA. These risk instruments/tools assess individuals’ clinical risk factors for osteoporosis to help gauge whether risk is sufficient for further evaluation with DXA. There are several potential advantages to use of clinical risk assessment tools as an initial osteoporosis screening test, including greater accessibility—these instruments can be used anywhere, including outpatient offices, nursing homes, etc., compared with DXA which is typically done at referral centers—and lower costs—the cost associated with risk instruments is primarily the time it takes to administer them, typically several minutes, in comparison to DXA which costs approximately $50 in the USA in 2014 [9].

The purpose of this study was to perform a systematic review and meta-analysis of the performance of clinical risk assessment instruments for identifying individuals with osteoporosis or low BMD by DXA criteria.

Methods

Data sources and search strategies

Literature search strategies were developed and performed in collaboration with a professional research librarian (AAS) to locate studies reporting the performance of clinical risk assessment instruments for identifying individuals with osteoporosis or low bone density (osteopenia) by DXA criteria in addition to studies that assessed instruments for predicting absolute fracture risk. This study reports findings for the systematic review and meta-analysis of clinical risk assessment instruments for identifying individuals with osteoporosis or low bone density; findings for the performance of risk instruments for predicting absolute fracture risk were reported in a prior study [10].

Databases searched included the following: Embase.com Embase (1974–2011), Wiley Cochrane Library (1898–2011), OvidSP MEDLINE (1948–June 2011), OvidSP MEDLINE Daily Update (June 2011), OvidSP In Process & Other Non-Indexed Citations (June 2011), ISI Web of Science limited to Proceeding Papers and Meeting Abstracts (1945–2011), ISI BIOSIS Previews limited to Meetings (1969–2011), Scopus (1960–2011), ClinicalTrials.gov (1999–2011), Health Services Research Projects in Progress (1995–2011), VHL LILACS (1982–2011), VHL IBECS (1999–2009), ProQuest Dissertations & Theses (1861–2011), NRR Archives (2000–2007), and OpenGrey (1980–2005). All initial literature searches were completed in June and July 2011. The MEDLINE search was updated in August 2014. The detailed MEDLINE search strategy used has been reported in the study of Nayak et al. [10]. Other database search strategies are available upon request.

Supplementary literature search methods included reviewing reference lists of included studies and topical reviews to locate additional studies. We also handsearched Osteoporosis International, Endocrine Reviews, and the Journal of Bone and Mineral Research for relevant articles from 1990 to 2011.

Study selection

Inclusion and exclusion criteria were applied to yield relevant studies. We included studies that evaluated the performance of a clinical risk assessment instrument to identify individuals with central DXA-determined osteoporosis or low BMD, provided sensitivity and specificity values (or sufficient data to calculate these values) for at least one specified cutoff threshold of the evaluated instrument used to identify individuals with BMD T-scores below a specified DXA threshold, reported original data, and had adult participants. We included studies of clinical risk tools combined with quantitative ultrasound, X-ray, or other non-DXA test results in the risk tool algorithm, provided that the instrument contained a clinical risk factor component. We included studies published in any format (e.g., journal article, government report, abstract) and any language of publication and had no restrictions on study participant characteristics (other than age ≥18 years) or co-morbidities. We excluded studies that did not evaluate clinical risk assessment instrument performance in populations independent of the instrument development cohort.

We reviewed studies for inclusion in two stages, title/ abstract followed by full text. We used Google Translate translation system to translate foreign language studies. One reviewer assessed all studies for inclusion or exclusion at the title/abstract stage (DLE); a second reviewer (SN) assessed all studies for which there were questions about potential eligibility at that stage. Two reviewers (DLE and SN) assessed all studies retrieved for full-text review for eligibility.

Data extraction

Information extracted from eligible studies included participant numbers, participant characteristics, study location, clinical risk assessment instrument(s) evaluated, DXA reference sites assessed, risk instrument thresholds (cutoff values used to separate positive from negative results) assessed, DXA low BMD or osteoporosis thresholds used, sensitivity and specificity associated with each threshold, area under the receiver operating characteristic (ROC) curve (AUC) if reported, and potential sources of bias.

Data analysis

We performed random-effects meta-analysis using the DerSimonian and Laird method to calculate summary estimates of sensitivity and specificity for each separate risk assessment instrument for which there were at least three studies evaluating performance in a similar population within the same country and reporting sensitivity and specificity estimates for the same combination of risk tool cutoff threshold and DXA reference sites and threshold. All analyses were performed using Stata version 11.0 (StataCorp, College Station, TX). Furthermore, we qualitatively described findings for instruments and thresholds for which data were insufficient for meta-analysis and evaluated potential sources of bias.

Results

Literature search and study selection

The literature search yielded 22,551 separate records for review, of which 108 met inclusion criteria [11–118]. A flow diagram of the literature search and study selection process is shown in Fig. 1.

Fig. 1.

Flow diagram of literature search and study selection

Study characteristics

Included study characteristics are described in the Electronic Supplementary Material (Appendix Table 1). The most commonly assessed instruments were the Osteoporosis Self-Assessment Tool (OST) (55 studies), the Simple Calculated Osteoporosis Risk Estimation (SCORE) instrument (32 studies), the Osteoporosis Self-Assessment Tool for Asians (OSTA) (27 studies), the Osteoporosis Risk Assessment Instrument (ORAI) (26 studies), and body weight criteria (15 studies). Many studies assessed multiple clinical risk assessment instruments. Only five studies evaluated a combination of a peripheral bone density assessment test with a clinical risk assessment tool [33, 59, 86, 88, 117]. Eighty-five studies were published as full-text articles, while 23 were abstracts only. Included studies were performed in 34 different countries. The number of study participants ranged from 60 to 21,063, and a mean participant age in the 60s was most common. Most studies (78) included female participants only, 24 studies had only male participants, and 6 studies included both sexes. The majority of studies did not report participants’ medical comorbidities; 5 studies included only individuals with rheumatoid arthritis [16, 43, 44, 47, 92]. Reported osteoporosis prevalence in study populations ranged from 4.1 to 44.8 %. Studies assessed the sensitivity and specificity of a variety of risk assessment instrument and DXA threshold combinations; most studies evaluated risk instruments for identifying participants with DXA-determined osteoporosis (T-score≤2.5) and assessed DXA sites of the femoral neck, total hip, and/or lumbar spine. Included studies were published between 1998 and 2014, and a large majority (101) was published in English.

Meta-analysis of performance of clinical risk assessment instruments for identifying individuals with osteoporosis by DXA criteria

Sufficient data was available for meta-analyses of the sensitivity and specificity of the OST risk assessment instrument for women and men in the USA, and the OSTA risk assessment instrument for women in Thailand. The total number of participants in all studies included in each of these meta-analyses was 31, 779 for OST for women in the USA [45, 51, 80], 760 for OST with a threshold (risk instrument cutoff value used to separate positive from negative results) of 3 for men in the USA [11, 60, 107], 5260 for OST with thresholds of 2 and 1 for men in the USA [11, 60, 71], 3079 and 2780 for OSTA with a threshold of ≤−1 for women in Thailand with DXA reference sites of the femoral neck [26, 39, 56, 84, 87, 88, 98, 109] and lumbar spine [26, 39, 56, 84, 87, 98, 109], respectively, and 1201 for OSTA with a threshold of ≤0 for women in Thailand [39, 84, 109]. Meta-analysis results are shown in Table 1. Meta-analysis of studies evaluating OST in US postmenopausal women with a threshold of <1 to identify individuals with osteoporosis at the femoral neck provided summary sensitivity and specificity estimates of 89 % (95%CI 82–96 %) and 41 % (95%CI 23–59 %), respectively. Meta-analysis of studies evaluating OST in predominantly older men in the USAwith a threshold of 3 to identify individuals with osteoporosis at the femoral neck, total hip, or lumbar spine provided summary sensitivity and specificity estimates of 88 % (95%CI 79–97 %) and 55 % (95%CI 42–68 %), respectively. Meta-analysis of studies evaluating OSTA in postmenopausal women in Thailand using a threshold of ≤0 to identify individuals with osteoporosis at the femoral neck provided summary sensitivity and specificity estimates of 90 % (95%CI 84–95 %) and 47 % (95%CI 30–64 %), respectively. Summary sensitivity estimates were lower (and specificity estimates typically higher) for the meta-analyses performed using other OST and OSTA thresholds and DXA references site combinations for US older men and postmenopausal women in Thailand (Table 1). There was significant between-study heterogeneity in all but one of the performed meta-analyses, as demonstrated by high I-squared values.

Table 1.

Meta-analysis results

Risk instrument	Country	Population	Risk instrument threshold	DXA reference site and T-score threshold	Summary sensitivity estimate from meta-analysis (%c (95%CI; I2 valued))	Summary specificity estimate from meta-analysis (%c (95%CI; I2 valued))
OSTa	USA	Postmenopausal women [40, 45, 68]	<1	Femoral neck, ≤−2.5	89 (82–96; I2=98.8 %)	41 (23–59; I2=99.9 %)
OSTa	USA	Men [11, 52, 90]	3	Femoral neck, total hip, or lumbar spine, ≤−2.5	88 (79–97; I2=68.5 %)	55 (42–68; I2=89.6 %)
OSTa	USA	Men [11, 52, 59]	2	Femoral neck, total hip, or lumbar spine, ≤−2.5	81 (70–92; I2=83.2 %)	54 (32–76; I2=98.5 %)
OSTa	USA	Men [11, 52, 59]	1	Femoral neck, total hip, or lumbar spine, ≤−2.5	73 (62–84; I2=81.0 %)	64 (45–83; I2=98.2 %)
OSTAb	Thailand	Postmenopausal women [26, 39, 56, 84, 87, 88, 98, 109]	≤−1	Femoral neck, ≤−2.5	84 (76–92; I2=76.8 %)	61 (50–72; I2=97.4 %)
OSTAb	Thailand	Postmenopausal women [26, 39, 56, 84, 87, 98, 109]	≤−1	Lumbar spine, ≤−2.5	71 (60–82; I2=89.7 %)	62 (52–73; I2=96.6 %)
OSTAb	Thailand	Postmenopausal women [39, 84, 109]	≤0	Femoral neck, ≤−2.5	90 (84–95; I2=10.4 %)	47 (30–64; I2=97.1 %)
OSTAb	Thailand	Postmenopausal women [39, 84, 109]	≤0	Lumbar spine, ≤−2.5	83 (67–99; I2=91.1 %)	48 (36–60; I2=93.4 %)

^aOsteoporosis Self-Assessment Tool

^bOsteoporosis Self-Assessment Tool for Asians

^cRandom-effects meta-analysis using DerSimonian and Laird method

^dPercentage of variation across studies attributable to heterogeneity

There were insufficient numbers of studies evaluating other risk assessment instruments within similar populations within the same country and using the same risk tool and DXA thresholds to perform meta-analysis for other instruments or for meta-analysis of the performance of OST and OSTA in other countries or for identifying individuals with low BMD rather than osteoporosis.

Qualitative review of the performance of clinical risk assessment instruments for identifying individuals with osteoporosis or low BMD by DXA criteria

We qualitatively reviewed the performance of frequently evaluated risk assessment instruments (OST, SCORE, ORAI, OSTA, and body weight criteria) for identifying individuals with DXA-determined osteoporosis (T-score≤−2.5) or low BMD with T-scores≤−2.0 at the femoral neck, total hip, or lumbar spine—commonly accepted reference sites for the diagnosis of osteoporosis [119]. Studies that evaluated OST for women reported performance estimates ranging from a high sensitivity of 99 % and corresponding specificity of 15 % with use of a threshold of <6 to identify perimenopausal and early postmenopausal women in Denmark with osteoporosis [96] to a low sensitivity of 17.7 % (95%CI 16.0–19.5 %) and corresponding specificity of 95.7 % (95%CI 94.8–96.4 %) with use of a threshold of −3 to identify postmenopausal women in Argentina with osteoporosis [100]; AUCs ranged from 0.652 (95%CI 0.604–0.699) [72] to 0.77 [95]. Studies that evaluated OST to identify men with osteoporosis or low BMD at the femoral neck, total hip, or lumbar spine reported performance estimates ranging from a high sensitivity of 93 % and corresponding specificity of 66 % with use of a threshold of 3 to identify US male veterans with osteoporosis [11] to a low sensitivity of 6 % and corresponding specificity of 94 % with use of a threshold of −2 to identify US men with rheumatoid arthritis with osteoporosis [92]; AUCs ranged from 0.590 (95%CI 0.492–0.688) when evaluating OST with a threshold of <4 in Portuguese men aged ≥50 [73] to 0.993 in a subgroup analysis of US male veterans aged ≥80 [11].

For the SCORE instrument, performance estimates for identifying women with osteoporosis or low BMD at the femoral neck, total hip, or lumbar spine ranged from a high sensitivity of 98.9 % and corresponding specificity of 5.7 % with use of a threshold of 6 to identify postmenopausal Belgian women aged >65 with osteoporosis [15] to a low sensitivity of 44 % and corresponding specificity of 77 % with use of a threshold of >7 to identify perimenopausal and early postmenopausal women in Denmark with osteoporosis [96]; AUCs ranged from 0.64 to 0.76 [34]. For ORAI, reported sensitivity for identifying women with osteoporosis ranged from a high value of 100 % (95%CI 94.9–100 %) with corresponding specificity reported as not applicable (no individuals tested negative) with use of a threshold of ≥9 for Portuguese post-menopausal women aged ≥65 years [72] to a low sensitivity of 3 % with corresponding specificity of 98 % with use of a threshold of >11 for perimenopausal and early postmenopausal women in Denmark [96]; AUCs ranged from 0.64 (95%CI 0.58–0.70) [96] to 0.703 [32].

For the OSTA instrument for identifying women with osteoporosis at the femoral neck, total hip, or lumbar spine, performance estimates ranged from a high sensitivity of 93.0 % (95%CI 84.3–97.7 %) and corresponding specificity of 20.2 % (95%CI 12.5–30.1 %) with use of a threshold of <2 for Portuguese postmenopausal women aged ≥65 years [72] to a low sensitivity of 29.9 % (95%CI 19.3–42.3 %) and corresponding specificity of 91.1 % (95%CI 88.2–93.5 %) with use of a threshold of ≤−1 for postmenopausal Chinese women aged 45–59 [114]; AUCs ranged from 0.62 (95%CI 0.56–0.68) [84] to 0.668 (95%CI 0.619–0.716) [72]. Reported sensitivities and specificities for OSTA for identifying men with osteoporosis at the femoral neck, total hip, or lumbar spine ranged from a high sensitivity of 87.33 % and corresponding specificity of 56.20 % with use of a threshold of ≤−1 for Chinese men aged ≥50 years [69] to a low sensitivity of 38.2 % and corresponding specificity of 82.1 % with use of a threshold of <1 for Portuguese men aged ≥50 years [73]; AUCs ranged from 0.597 (95%CI 0.497–0.697) [73] to 0.676 (95%CI 0.612–0.732) [117].

Two studies evaluated body weight criteria for identifying women with osteoporosis at the femoral neck, total hip, or lumbar spine [72, 78]. The highest reported sensitivity was 88.0 % (95%CI 68.8–97.5 %) with a corresponding specificity of 43.6 % (95%CI 36.2–51.2 %) when using a threshold of <70 kg for 40–54-year-old Portuguese postmenopausal women [72], and the lowest reported sensitivity was 39.6 % (95%CI 38.5–40.6 %) with a corresponding specificity of 82.8 % (95%CI 82.0–83.6 %) when using a threshold of ≤57 kg for Canadian women aged 40–59 years [78]. AUCs ranged from 0.611 (95%CI 0.562–0.661) for Portuguese post-menopausal women [72] to 0.71 (95%CI 0.68–0.75) for a subgroup of 40–49-year-old Canadian women [78]. One study evaluated body weight criteria for identifying men with osteoporosis at the femoral neck, total hip, or lumbar spine [73]. This study, which included Portuguese men aged ≥50 years, reported a high sensitivity of 82.4 % and corresponding specificity of 35.7 % when evaluating a threshold of <80 kg (AUC 0.590 (95%CI 0.492–0.689)) and conversely a low sensitivity of 26.5 % and specificity of 89.3 % when evaluating a threshold of <65 kg (AUC 0.579 (95%CI 0.467–0.691)) [73].

Figure 2 is a plot of overall analysis results for sensitivity (true positive rate) versus 1 minus specificity (false positive rate) from studies that evaluated the performance of clinical risk assessment instruments in postmenopausal female study populations for identifying DXA-determined osteoporosis as defined by T-score≤−2.5 at the femoral neck, total hip, or lumbar spine. Risk assessment instrument performance estimates for postmenopausal women varied by study, risk instrument assessed, and risk instrument threshold assessed. The OST and SCORE instruments at several different thresholds (3 or 4 for OST and 6 or 7 for SCORE) demonstrated the highest sensitivities for identifying postmenopausal women with osteoporosis in these studies.

Fig. 2.

Scatter plot of sensitivity (true positive rate) versus 1–specificity (false positive rate) for studies evaluating clinical risk assessment instrument performance for identifying postmenopausal women with DXA-determined osteoporosis (T-score≤−2.5) at the femoral neck, total hip, or lumbar spine. Each point is labeled with the corresponding risk instrument, risk instrument cutoff threshold (cutoff value used to separate positive from negative results), and associated reference. Points are proportional to the number of study participants; however, sizes are not to scale

Performance of clinical risk assessment instruments by age

A number of studies assessed performance of SCORE, OST, ORAI, OSTA, or body weight criteria in different age subgroups when using the same risk instrument thresholds for identifying individuals with osteoporosis or low BMD at various DXA references sites. Six studies evaluated SCORE performance in younger and older perimenopausal or postmenopausal women [15, 34, 57, 75, 81, 97]; four found lower sensitivity of SCORE for younger women in their studies than older women when using the same thresholds, and all found higher specificity for younger women than older women. Three studies assessed OST in older and younger subgroups of postmenopausal women [72, 74, 80]; all found that that OST had lower sensitivity but higher specificity for younger postmenopausal women when using the same thresholds. Four studies assessed OST performance for age subgroups of men over the age of 50; three found lower sensitivity for the younger men in their analyses, and all four studies found higher specificity for the younger men subgroup [11, 60, 91, 93]. Four studies evaluated ORAI performance in younger compared to older postmenopausal women; three found lower sensitivity for younger women, and all found higher specificity for younger women [72, 74, 75, 81]. Three studies evaluated OSTA performance in younger and older perimenopausal or postmenopausal women, and all found lower OSTA sensitivity but higher OSTA specificity in younger women [72, 87, 114]. Two studies evaluated body weight criteria (<70 kg) performance in younger and older postmenopausal women; one study found higher sensitivity for women 40–55 years of age and similar specificity for younger and older women [72], and the other found similar sensitivity and specificity for younger and older postmenopausal women [74].

Study quality and potential sources of bias

We evaluated studies on several quality criteria to assess potential for bias, including sample size, recruitment of a cohort unclassified by disease (osteoporosis or low BMD) state, time between risk tool administration and DXA testing, independence of interpretation of risk tool and DXA results, and source of funding. A large majority of studies had at least 30 participants with and 30 participants without osteoporosis or low bone density, including all studies included in the meta-analyses for OST for US women. However, several studies included in meta-analyses of OSTA for women in Thailand did not have at least 30 participants with osteoporosis at either the femoral neck or lumbar spine [26, 56, 84], and two studies included in meta-analyses of OST for men in the USA [11, 107] did not have 30 participants with osteoporosis. Nearly all studies implied recruitment of participants as a cohort unclassified by disease state and did not report independence of interpretation of risk tool and DXA results. A large majority of studies did not report the time between risk instrument and DXA testing for participants. Most studies did not report their funding source; 28 studies reported pharmaceutical company funding or author association with a pharmaceutical company [11, 16–18, 20, 23, 36, 38, 40, 41, 49–51, 58, 59, 64, 71–73, 76, 78, 85, 91–93, 102, 103, 118], including three that were included in meta-analyses of OST for US men [11, 71, 93].

Discussion

Our systematic review and meta-analysis of the performance of clinical risk assessment instruments for screening for osteoporosis or low bone density found that many studies have been performed to assess performance of these tools; however, evidence for each tool’s performance when using the same risk instrument threshold and DXA threshold and reference sites within a particular country’s population is limited, with only OST (for US women and men) and OSTA (for women in Thailand) having a sufficient number of similar studies for meta-analysis of sensitivity and specificity estimates. Our meta-analysis findings showed reasonably high (88–90 %) summary sensitivity estimates for OST for postmenopausal women in the USA and OSTA for postmenopausal women in Thailand when using thresholds of <1 and ≤0, respectively, to identify women with osteoporosis at the femoral neck and for OST for older US men when using a threshold of 3 to identify men with osteoporosis at the femoral neck, total hip, or lumbar spine. However, the corresponding specificity estimates at these thresholds were low, in the range of 40–55 %. Our qualitative review of studies that evaluated risk instruments of OST, SCORE, ORAI, OSTA, and body weight criteria revealed that each showed sensitivity near or exceeding 90 % for identifying individuals with DXA-determined osteoporosis or low BMD with a T-score≤−2.0 at the femoral neck, total hip, or lumbar spine when using various risk instrument thresholds within different populations but low specificity, frequently below 40 %, at thresholds required for high sensitivity. Thus, at clinical risk tool thresholds required to identify nearly or more than 90 % of individuals with osteoporosis or low BMD with a T-score≤−2.0, many false positives can also be expected, with approximately half to most individuals without osteoporosis or DXA T-score≤−2.0 also testing positive. AUCs for commonly evaluated risk instruments to identify individuals with osteoporosis or T-score≤−2.0 at the femoral neck, total hip, or lumbar spine varied in different study populations, with values most commonly in the 0.6 to 0.8 range; an AUC of 1 indicates a perfectly accurate test for distinguishing individuals with and without the condition of interest, whereas an AUC of 0.5 indicates a useless test. In general, AUCs of 0.80–0.90 are considered good, 0.70–0.80 are considered fair, and 0.60–0.70 are considered poor [120].

Among studies that evaluated the performance of clinical risk assessment instruments in postmenopausal female study populations for identifying DXA-determined osteoporosis as defined by T-score≤−2.5 at the femoral neck, total hip, or lumbar spine, OST and SCORE evaluated at several different thresholds (3 or 4 for OST and 6 or 7 for SCORE) demonstrated the highest sensitivities. OSTA, body weight criterion, and ORAI at various thresholds demonstrated somewhat lower sensitivity among these studies. Studies that evaluated the Age, Body Size, No Estrogen (ABONE) and OSIRIS risk tools for postmenopausal women at cut points of ≥2 or <1, respectively, for diagnosis of osteoporosis at various DXA reference sites generally found insufficient sensitivity compared to other options; thus, we recommend against use of ABONE or OSIRIS at these cut points given better alternatives. In general, simpler instruments, such as OST, OSTA, or even body weight criteria alone performed as well as (and in several studies, most frequently for OST, better than) more complex instruments, and thus, we recommend use of a simple instrument, such as OST, over more complex instruments. Even body weight criteria, with its simplicity (no calculation required), performed comparably in several studies to some tools with more risk factors (e.g., ORAI), although generally not as well as OST or SCORE. However, given its simplicity and that primary care patients are weighed at almost every appointment, body weight criteria have potential for easy clinical use.

Even within our meta-analyses limited to studies done within similar populations within the same country using identical risk instrument and DXA thresholds, there was significant between-study heterogeneity. One possible source of heterogeneity is variation in participant age; for instance, of the studies included in the meta-analyses of OST for postmenopausal US women, the study by Gourlay et al. that included only older postmenopausal women aged 67 years and older [45] reported higher sensitivity and lower specificity than the other two included studies, one which included participants aged 45–81 [51] and the other which had a majority of participants younger than age 65 [80]. Our qualitative review of studies that performed age subgroup analyses for the OST, SCORE, OSTA, or ORAI risk instruments revealed that for each of these instruments when using the same threshold for different age subgroups, sensitivity for identifying individuals with osteoporosis or low BMD was lower at a majority of the time (and specificity consistently higher) for younger compared to older postmenopausal women or older men. Our findings suggest that for many instruments, different thresholds may be needed for screening individuals in different age ranges to ensure adequate sensitivity, particularly for younger (such as early postmenopausal) individuals. Another potential source of heterogeneity in our analyses is slight variation in how risk instrument scores were calculated between studies. For example, of the studies that were included in the meta-analyses of OST for US men, three calculated OST values using the formula [0.2×(weight in kilograms-age in years), truncated to give an integer] [11, 71, 107], whereas one study calculated OST using the formula [0.2×(weight in kilograms -age in years), rounding to the nearest integer] (personal communication) [60].

Our findings indicate a general pattern of osteoporosis clinical risk assessment instruments having a significant trade-off between sensitivity and specificity, and fair at best overall ability to distinguish individuals with and without osteoporosis or T-scores≤−2.0 by DXA criteria at the femoral neck, total hip, or lumbar spine. Our findings generally agree with prior more limited reviews evaluating the performance of these instruments. A systematic review by Rubin et al. that included 31 studies of risk instruments for predicting low BMD also found no consistently best performing tool, and that simple tools performed equal to or better than more complex instruments [121]. A systematic review by Nelson et al. in 2010 compared AUC performance estimates for 23 studies of risk assessment instruments to predict BMD T-score≤−2.5 and also found that most AUCs were in the range of 0.6 to 0.8, indicating modest prediction of DXA-determined osteoporosis [122]. This study also found that instruments with fewer risk factors often performed equally to or better than more complicated instruments and did not identify a clearly best performing instrument [122]. A systematic review by Rud et al. that included studies comparing OST to other tests to select women for BMD testing found that the diagnostic odds ratio did not differ significantly among OST, SCORE, and ORAI instruments [123].

Osteoporosis clinical risk assessment instruments are not currently widely used for screening by primary care physicians, unlike some risk instruments for other medical conditions—for example, the Framingham Risk Score for 10-year cardiovascular event risk. This is despite the fact that many osteoporosis risk instruments are simpler than the Framingham Risk Score, and discrimination performance of the Framingham Risk Score does not appear to be better than osteoporosis clinical risk assessment instruments, with c statistic values (analogous to AUCs) ranging from 0.63 to 0.83 in different populations [124]. Several factors may contribute to greater use of the Framingham Risk Score. First, heart disease is the leading cause of death of women and men, with mortality rates substantially higher than that associated with osteoporosis; thus, given that physicians have competing preventive care demands, it is not surprising that they may prioritize heart disease prevention. Another factor that may contribute to lower use for osteoporosis clinical risk instruments is lack of evidence for whether their standardized use would reduce fracture rates. An additional barrier is the existence of different osteoporosis clinical risk instrument cutoff thresholds to define a positive test result when screening among different populations, such as women versus men, or individuals of different ages. Such “moving-target” thresholds are an impediment for busy clinicians who have limited time in a brief patient visit to identify the appropriate threshold. This problem could be addressed by providing an easy-to-use online osteoporosis risk instrument calculator for physicians to enter key data about their patient (e.g., age and sex) and have this data automatically processed to report whether a patient’s risk instrument score is sufficient to warrant further evaluation.

Our systematic review and meta-analysis results by themselves are insufficient to answer the question of whether osteoporosis clinical risk assessment tools should be used routinely in clinical practice. This question would be best addressed with a comprehensive comparative effectiveness analysis that compares different screening tests and thresholds to identify the best strategies for patients with different key characteristics such as age and sex. It is likely that the best screening strategies would vary according to patient characteristics. Although specificity is generally poor for osteoporosis clinical risk assessment instruments at the thresholds required to identify approximately 90 % of individuals with osteoporosis or low BMD, it is possible that it may still be worthwhile to prescreen individuals with a clinical risk assessment instrument and reduce the number of people without osteoporosis or low BMD referred for DXA testing by 50 % or so. Several previous studies have found that osteoporosis risk assessment instruments can be cost-effective screening tools, despite their low specificity [125, 126]. Our findings can be applied to future comparative effectiveness analyses to evaluate whether prescreening with clinical risk assessment instruments may be a good option for patients with different characteristics. In the absence of an up-to-date comparative effectiveness analysis of all available osteoporosis screening options for women and men of different ages, evidence is currently lacking to recommended routine use of osteoporosis clinical risk assessment instruments as an initial screening test over DXA. However, these tools are a viable screening option for individuals who are not able to easily access DXA testing or who would prefer a non-DXA initial screening test. If an osteoporosis clinical risk assessment instrument is chosen for initial screening, we recommend use of a simple instrument (e.g., OST).

Our study has several limitations. First, the heterogeneity of included studies limited our ability to perform meta-analysis of all the data. Even within the meta-analyses performed, there was significant statistical heterogeneity. Another limitation was the total number of study participants included in the studies in our meta-analysis of OST for screening US men with a threshold of 3 was relatively small (760). Furthermore, we found mixed quality of the studies included in this systematic review. Moreover, publication bias is a possibility, with studies showing favorable performance results being preferentially published; however, we included abstracts in addition to full-text articles to mitigate this potential bias. Our study had several notable strengths. This study is the most comprehensive review of the performance of clinical risk assessment instruments for identifying individuals with osteoporosis or low BMD to date; we included greater than 70 more studies than any prior review on this topic that we are aware of, after performing an exhaustive literature search. Additionally, we performed meta-analyses of performance estimates of OST for women and men in the USA and OSTA for women in Thailand, including data from nearly 32,000 participants in several different studies for the meta-analyses of OST for US postmenopausal women.

In conclusion, our findings show that commonly evaluated risk instruments of OST, SCORE, ORAI, OSTA, and body weight criteria each demonstrate high sensitivity approaching or exceeding 90 % for identifying individuals with DXA-determined osteoporosis or low BMD with a T-score≤−2.0 at particular thresholds within various populations but with a trade-off of low specificity at thresholds required for high sensitivity. Simpler instruments, such as OST, generally perform as well as or better than more complex instruments. Currently, the lack of standardized cutoff thresholds for these instruments limits their potential for clinical use; thus, cut point standardization is an important area for future research. Additional studies are also needed to evaluate the comparative effectiveness and cost-effectiveness of use of clinical risk assessment instruments for initial prescreening of individuals for osteoporosis or low BMD compared with other screening strategies.