Manual Chart Review Reveals Bisphosphonate Exposure Misclassification in Atypical Femur Fracture Patients: Auditing Medical Records in Two International Cohorts

Abstract

Purpose

Variation in results across fracture studies can stem from differences in study design, case definitions, data sources and healthcare settings, but in atypical femur fracture (AFF) studies, the reported proportion of AFF patients without bisphosphonate (BP) exposure shows an exceptionally wide variation. We aimed to evaluate whether misclassification of BP exposure could explain this variability and hypothesized that manual review of medical records would narrow differences between cohorts.

Methods

This retrospective study examined two population-based AFF cohorts: one from Sweden and one from Kaiser Permanente Southern California (KPSC). The Swedish cohort included 172 patients, aged ≥55 years with AFFs diagnosed between 2008 and 2010, of whom 38 had no recorded BP use. The KPSC cohort included 306 patients, aged ≥50 years with AFFs diagnosed between 2007 and 2017, of whom 29 were classified as BP-naïve. Manual review of medical records and pharmacy data was performed to verify BP exposure status.

Results

In the Swedish cohort, manual review did not identify any additional BP exposure, leaving 38 of 172 patients classified as BP-naïve (22.1%). In contrast, in the KPSC cohort, evidence of prior BP use was identified in 20 of 29 patients initially classified as BP-naïve, further reducing the proportion of BP-naïve AFF cases from 9.5% to 2.9%.

Conclusion

Misclassification of BP exposure occurs and can substantially affect reported rates of BP-naïve AFF, but it does not fully explain the wide variation observed across studies. Additional contextual or methodological factors likely contribute and should be addressed in future AFF research.

Mini abstract

We examined if inaccuracies in reported BP treatment in two large cohorts could explain the wide discrepancy observed between different patient cohorts with atypical femur fractures. Assessing BP treatment through manual review of medical and pharmacy records increased the discrepancy, indicating that other factors likely contribute to the observed variation.

Introduction

When studies of a given diagnosis report widely varying rates, a useful first step is to examine methodological differences in study design, case definitions, data sources, and healthcare contexts before considering other explanations for the discrepancy.

In fracture research, one striking example is the wide and unexplained variation found in the reporting of atypical femur fracture (AFF). AFF are rare fractures of the femoral shaft, which in older adults, are primarily linked to bisphosphonate (BP) use. BPs reduce hip and vertebral fracture incidence in patients at high risk of fragility fracture by inhibiting bone resorption [1, 2], and long-term BP use has been strongly and consistently associated with AFF [3–5].

Meanwhile, however, AFFs have also been reported in patients who have not been exposed to BP medications (these patients are termed “BP-naïve”). When it comes to these BP-naïve patients the reporting is, by contrast, characterized by wide and unexplained variation between studies. The proportion of reported AFF cases without BP exposure has ranged from a very low 2.6% to a very high 72.0% (Table 1) [3–10]. Between these studies, differences certainly exist in study design, case definitions, data sources, and healthcare contexts. However, it is still difficult to explain the unusual degree of variation that is particular to the reporting of the proportion of BP-naïve AFF patients. A first step would be to take a closer look at whether these different studies are characterized by substantial differences in methodology. Relevant differences could arise in diagnosis classification, how AFF outcomes are determined, or in treatment classification, how the lack of exposure to BPs is ascertained, or both.

Table 1.

Prior studies demonstrating variation in proportions of AFF without BP treatment in studies with radiographic adjudication of fracture types.

Study	Study Design	Pharmacy Data Source	AFF (n)	Number of patients without BP (n)	Percentage without BP (%)
Meier et al. 2012 [6]	Case-control	EMR data	39	7	17.9
Dell et al. 2012 [7]	Retrospective cohort	EMR data	142	14	9.9
Feldstein et al. 2012 [8]	Retrospective cohort	EMR data	75	54	72.0
Schilcher et al. 2015 [5]*	Case-control and cohort study	Swedish Patient Registry	172	38	22.1
Lo et al. 2012 [9]	Retrospective cohort	EMR data	38	1	2.6
Kim et al. 2017 [10]	Cross-sectional	EMR data	147	33	22.4
Black et al. 2020 [3]*	Retrospective cohort	EMR data	306	29	9.5
Bauer et al. 2024 [4]	Case-cohort	Danish Patient Registry	189	59	31.2

^*Included in this study.

EMR = electronic medical record. AFF = atypical femur fracture. BP = bisphosphonate.

Two factors could explain possible misclassification of AFF. First, some studies classify AFF by radiographic confirmation, while others rely on diagnostic codes alone; this latter approach captures fractures that do not meet radiographic AFF criteria and are less likely to be BP-related, potentially inflating the apparent BP-naïve proportion [11]. Second, even within studies using radiographic adjudication, which is more reliable [11], the American Society for Bone and Mineral Research (ASBMR) case definition has evolved and shown variability in interpretation and application [12–16].

If we consider only this subset of studies in which AFF has been radiographically adjudicated, any differences in how ASBMR criteria are interpreted and applied, even if substantial, still appear insufficient to account for the large variation even in these reported proportions of AFF patients who are BP-naïve (Table 1). Therefore, the other possible major methodological difference between studies also needs to be considered: treatment classification—how is the lack of exposure to BPs being ascertained? Treatment misclassification could occur for BP exposure if the data being used suffers from inaccuracies. If accuracy could be improved, it might reduce the level of variation in reporting, leading to clearer findings. However, validation of data accuracy for treatment classification in large, population-based studies of AFF has not yet been sufficiently pursued.

We set out to validate the accuracy of the data on treatment classification for reportedly BP-naïve AFF patients in two of the largest existing cohorts of radiographically confirmed AFF patients: one from Sweden (the Swedish cohort) and one from Kaiser Permanente Southern California (KPSC) in the United States (the KPSC cohort). Between the two populations, the initial reported proportions of BP-naïve AFF patients had differed substantially: 22.1% in the Swedish cohort compared with 9.5% in the KPSC cohort. We tested the accuracy of these reported proportions by manually reviewing drug exposure data in medical and pharmacy records in both cohorts. In seeking to validate BP exposure status among these two populations of patients, our goal was to acquire an additional degree of insight into the observed variations across studies. Our hypothesis was that manually reviewing drug exposure data in medical and pharmacy records would correct misclassified treatment status and, thereby, reduce the previously reported differences in the proportion of BP-naïve patients between the two cohorts.

Methods

Swedish Cohort

As previously reported [5], the Swedish National Patient Register (SNPR) was used to identify women and men in Sweden aged 55 years or older (total population = 2 891 586) who had a subtrochanteric or diaphyseal femur fracture (ICD-10 code S722 or S723) between January 1, 2008 – December 31, 2010 (n= 5 694, for 2008 only women were included) (Appendix, Table S). Fractures associated with high-energy trauma (ICD-10, V-diagnosis associated with motor vehicle accidents and falls from higher than standing height) were excluded.

Ethical approval for the Swedish cohort was granted by the regional ethics board in Linköping for inclusion of patients (2011/358-31) and retrieval of medical records (2022-04383-02).

AFF determination

Radiographs of all femur fractures identified via diagnosis codes were adjudicated according to the 2014 ASBMR major criteria [17]. The adjudication process was performed by three orthopedic surgeons who were blinded to all background data, identifying 172 patients with AFF [5]. The ASBMR major criteria transverse fracture line and focal cortical thickening (callus reaction) were compulsory in that case definition of AFF.

BP exposure determination

Data from the Swedish National Prescribed Drug Register (SNPDR) were used to determine the patients’ BP status. As described in the previously published study, 134 patients appeared to have used BPs prior to fracture (BP-users) and 38 did not have evidence of BP use (BP-naïve) (Figure).

Figure.

Flowchart of patient selection and radiographic adjudication in the Swedish cohort (Schilcher et al. 2015) [5] and the KPSC cohort (Black et al. 2020) [3].

*The 11 women excluded from the KPSC cohort had AFFs that occurred prior to cohort entry.

AFF = atypical femur fracture; NFF = normal femur fracture. BP = bisphosphonate. KPSC = Kaiser Permanente Southern California. SNPR = Swedish National Patient Register. SNPDR = Swedish National Prescribed Drug Register. EMR = electronic medical record. ICD = international classification of disease.

Estimate of BP exposure prevalence

The prevalence of BP treatment in the underlying Swedish population was estimated from a random sample of the population comprising 282 937 patients aged 55 years and older at the start date of the cohort: January 1, 2008, through December 31, 2015. Data on BP treatments was obtained from the SNPDR.

BP exposure validation

For these 38 BP-naïve patients with AFF, medical records between 2005 (the start of drug prescription registration in the SNPDR) to 2010 were requested from all relevant healthcare regions in Sweden and included available documentation of all visits and care periods at public healthcare departments of orthopedic surgery, general surgery, oncology, internal medicine, geriatrics, endocrinology, rheumatology and from primary care health centers. Private health care providers were identified and contacted individually. Medical records were then manually reviewed (using a standardized protocol) by our research team, or examined locally by qualified staff, to identify any mention of past or current BP treatment. In addition, we conducted an electronic health record search for antiresorptive therapies, using a predefined list of drug names, Anatomical Therapeutic Chemical (ATC) codes, and relevant procedural codes.

KPSC Cohort

The KPSC cohort has been previously described [3] and included women aged 50 years or older who were KPSC members at any time between January 1, 2007, to November 30, 2017, (n=1 097 530) and who had at least 12 months of continuous health plan enrollment prior to cohort entry. Cohort members were considered disenrolled from the health plan after 92 consecutive days of non-enrollment and were not allowed to re-enter the cohort.

The KPSC study was reviewed and approved by the KPSC Institutional Review Board under a waiver of consent.

AFF determination

Subtrochanteric and diaphyseal femur fractures were identified for cohort members via ICD-9 and ICD-10 codes (Appendix, Table S) (n=1 689) [3]. Fractures associated with an ICD-9 E-code for motor vehicle crashes (E810*-E819*) or ICD-10 V-codes within three days before or after the date of fracture were excluded. Blinded radiographic adjudication according to the 2014 ASBMR major criteria was conducted independently by two orthopedic surgeons and discordances were discussed until consensus was reached [17]. In this process a total of 317 AFF were identified, though 11 were later excluded because the AFFs occurred before the patient’s cohort entry or after the patient’s follow-up was censored for other reasons, leaving 306 confirmed AFFs for analysis (Figure).

BP exposure determination

The total duration of BP therapy during KPSC membership for both oral and parenteral alternatives and the recency of treatment were obtained from electronic pharmacy records of dispensed medications. Oral BPs were typically dispensed for 3 months at a time and we considered a patient exposed to oral BP upon receipt of a second dispensing of the medication. A patient was considered exposed to parenteral BP upon receipt of the initial injection/infusion. Of the 306 patients with adjudicated AFF, 29 appeared to have had no records of BP treatment.

BP exposure validation

For these 29 BP-naïve patients with AFF, manual chart review was conducted using the KPSC electronic health record (EHR), which was introduced region-wide in 2005. The comprehensive EHR includes documentation from all healthcare encounters delivered within KPSC and may include records for care received outside of KPSC if the patient or physician requested those documents to be added. Additionally, to capture BPs dispensed from a non-KPSC pharmacy, claims data were also searched. For each BP-naïve patient, a trained abstractor reviewed EHR records beginning on the date of the fracture outcome and working backwards into the pre-fracture period as far as possible, or until mention of BP use was found.

Results

Patient characteristics are provided in Table 2.

Table 2.

Patient characteristics in both cohorts before the medical chart review.

Patient Characteristics	Sweden	KPSC
AFF follow-up years	2008 – 2010	2007 – 2017
Adjudicated fractures	5 475	1 689
All AFF, n (% women)	172 (93.0)	306 (100) *
Age, mean (SD)	76.7 (8.2)	72.7 (8.4)
BP-naïve AFF, n (%)	38 (22.1)	29 (9.5)
- Women, n (%)	30 (78.9)	29 (100)
- Age, mean (SD)	74.8 (9.3)	56.3 (7.0)

KPSC = Kaiser Permanente Southern California. AFF = atypical femur fracture. BP = bisphosphonate

^*KPSC cohort included only women. BP treatment included denosumab. The Swedish cohort included only women in 2008 and women and men in 2009–2010.

Swedish Cohort

Access to medical records and drug charts was granted for 36 out of the 38 patients from 17 of the 21 Swedish Health Care regions. For two patients from Region Stockholm, the data security assessment yielded a rejection of our request. No parenteral or oral antiresorptive treatments were identified for any of the 36 patients with available medical and drug charts; thus, the proportion of patients without BP treatment in the original 172 Swedish AFF patients remained unchanged at 22.1% after the validation process. If the two patients without available data are assumed to have received BP treatment, the recalculated proportion would be 20.9%.

Between 2005 and 2010, 5.5% of Swedish adults aged 55 or older received BPs, most commonly alendronate (4.5%). Among hip fracture patients, 14.3% had used BPs. Use of parenteral agents was minimal: zoledronic acid (0.2% in the population; 0.5% in hip fracture patients) and denosumab (≤0.02% in both groups). Hormone therapy and other osteoporosis medications were more common, but BPs remained the primary antiresorptive treatment.

The different types of antiresorptive treatment and treatment prevalence in Sweden are provided in Table 4.

KPSC Cohort

In the KPSC cohort, manual review of EHR data was completed for all 29 patients initially classified as BP-naïve. Treatment with BP was identified in 20 patients (Table 3), and two additional patients had documented use of other bone-active medications, raloxifene (n=1) and calcitonin (n=1). Alendronate was the most frequently used agent (n=18), followed by pamidronate (n=3), ibandronate (n=2), and zoledronate (n=2). Ten patients had received a combination of BPs and/or other bone-modifying medications, while the remaining ten had received only a single BP agent, alendronate (n=9) or pamidronate (n=1). Reasons for misclassification fell into the following groups: 2 had received zoledronate infusions recorded in datasets not included in the original search algorithm; 4 had filled BP prescriptions at non-KPSC pharmacies, captured in claims data that had not been searched correctly; 2 had documented BP use predating KPSC enrollment with no internal dispensing record; 6 had clear narrative documentation of ongoing BP use in clinician notes that the original search did not capture; and 6 had appeared to initiate BP only after the fracture but, on review, had evidence of prior use in the clinical notes. Reclassification of these 20 individuals increased the proportion of AFF cases among BP-exposed individuals in the KPSC population (estimated at ~196 000) from 0.14% to 0.15%. After validation, the proportion of AFF patients without documented BP exposure decreased from 9.5% to 2.9%.

Table 3.

BP exposure among patients with radiographically adjudicated AFF in the two cohorts, before and after the validation process. Some patients had multiple BP-treatments.

	Sweden AFF, n=172		KPSC AFF, n=306
BP status	Before validation, n (%)	After validation, n (%)	Before validation, n (%)	After validation, n (%)
BP use	134 (77.9)	U	277 (90.5)	297 (97.1)
No BP use	38 (22.1)	36 (20.9)	29 (9.5)	9 (2.9)
Unknown	0	2 (1.2)	0	U
Drug type	Number of patients, mean years of treatment (SD)
Alendronate	120, 3.5 (1.2)	U	272, 6.9 (3.2)	290, 6.9 (3.2)
Risedronate	16, 3.5 (1.4)	U	10, 2.0 (2.3)	U
Clodronate	NA	NA	0	U
Etidronate	0	U	2, 2.2 (0.3)	U
Zoledronate	0	U	8, 3.9 (4.5)	10, 4.2 (4.0)
Ibandronate	2*	U	9, 2.8 (1.0)	11, 3.3 (1.3)
Pamidronate	NA	0	2, 1.2 (0.3)	5, 2.7 (1.7)
Denosumab	NA	0	0	U
Any osteoporosis medication	141, 3.6 (1.1)	U	277, 7.2 (3.4)	297, 7.2 (3.5)
Selective estrogen receptor modulator	1*	U	NA	NA
Hormone replacement therapy	50, 2.2 (1.5)	U	NA	NA
Parathyroid hormone analogs	1*	U	0	U
Strontium ranelate	2*	U	NA	NA

BP = bisphosphonate. AFF = atypical femur fracture. KPSC = Kaiser Permanente Southern California.

^{* =}no relevant statistics for duration of use due to few cases. U = unchanged, same value before and after validation of BP status. NA = not available.

Discussion

Reported proportions of BP-naïve AFF vary widely between studies, from 2.6% to 72.0%. Comparing studies is challenging because of differences in study design, case definitions, data sources, and healthcare contexts (Table 1). For example, case-control studies of AFF [6] tend to enrich for BP exposure through case selection, while cohort studies that rely on broader case identification criteria (ICD codes) and lack of radiographic adjudication can capture misclassified AFFs [18]. Cumulative BP exposure cannot be assessed at all in cross-sectional studies [10].

In our Swedish and KPSC cohorts, the BP-naïve proportion had originally been reported as 22.1% and 9.5%, respectively, both lower than the 31.2% in the Bauer et al. study [4], We hypothesized that manual review of medical and pharmacy records would correct misclassification and narrow this difference between the two cohorts.

During our manual review, we identified and corrected some misclassified treatment status cases in the KPSC cohort. However, unexpectedly, the corrections shifted the results in the opposite direction from what we had hypothesized. Improved accuracy did not lead to reduced variation between the cohorts, but to greater variation. In the Swedish cohort, the validation process identified no previously unrecorded BP use, leaving the proportion of AFF patients without documented BP treatment unchanged at 22.1%. In contrast, manual review of the KPSC records uncovered evidence of prior BP therapy in most patients initially classified as BP-naïve, reducing the percentage of AFF cases without BP exposure from 9.5% to just 2.9%. Since the proportion of BP-naïve patients became even wider after validation, the originally observed difference in proportions of BP-naïve patients appears to result not from treatment misclassification caused by data inaccuracy, but from other factors.

Our finding that 20 more AFF patients had received BP treatment in addition to the previous 277 AFF patients in the manual chart review underscores a broader methodological point: accurate information on BP treatment status is paramount, because inaccuracies may distort the association between BP and AFF, leading to incorrect conclusions about the risks and benefits of BP therapy [19]. In this specific instance, however, the misclassification had a small effect on the association between BP and AFF. While the issue of radiographic misclassification of AFF has been acknowledged in many studies, the misclassification of treatment status in large population-based studies using drug registers or medical records has not been thoroughly investigated. Our results demonstrate that electronic pharmacy records alone, even in a highly integrated healthcare system, may be insufficient to ensure reliable treatment classification, given the complexity of drug coding systems [20] and the potential for care outside the system of record. This emphasizes the importance of combined strategies to ascertain drug treatment exposure, including manual chart review, in research on AFF. More broadly, our experience illustrates that observational cohort studies require a demonstration of assurance regarding the thoroughness and reliability of data collection before conclusions about drug–outcome associations can be considered robust [21, 22].

At the same time, even though we found not less but more variation in the rates of BP-naïve patients in different cohorts, this lack of consistency does not apply to AFF patients who had been exposed to BP. As long as radiographic adjudication was used to define AFF outcome, the association between BP exposure and AFF across different cohorts itself remains very strong.

Below, we discuss some other minor factors that might be reflected in our results, some issues related to treatment classification, an issue possibly related to diagnosis classification, and strengths and limitations of our study.

Look-back period differences between the cohorts

In theory, one factor that could have influenced our findings is the difference in look-back periods between the two cohorts. In the KPSC cohort, the look-back period went back from the date of AFF to the most recent mention of BP use or until the start of the patient’s KPSC membership. By contrast, in the Swedish cohort it was limited to 2005, the start of the SNPDR. However, these differences in the look-back window between the cohorts seem very unlikely to have contributed to the observed differences, because the risk for AFF decreases by 70% for every year since last use of BP [23].

Imbalances in sex between the cohorts

Differences in sex distribution might contribute to a disparity in BP treatment prevalence between the cohorts. The KPSC cohort did not recruit men, including only women aged 50 years or older, while the Swedish cohort included both women and men (except for 2008) aged 55 and older (Table 2). Osteoporosis is more common in women, and BPs are more frequently prescribed to women [24], a finding supported by our previous work [5], as well as a study from Singapore [25]. Both studies reported a higher proportion of women among BP-related AFF cases compared to BP-naïve cases.

Ethnic imbalances between the cohorts

Ethnic differences likely contribute to the disparity in AFF rates between the cohorts. In the original KPSC study [3], 13.5% of BP treated women were of Asian descent. Asian women had a substantially higher AFF incidence of 5.95 per 10,000 person-years compared with 1.09 for White women, with a multivariate-adjusted hazard ratio of 4.84 (95% CI 3.57–6.56) [3]. This is consistent with other studies demonstrating that the risk of AFF among Asian women is 5 to 6 times higher than among women of other ethnic groups [26]. Information on race or ethnicity is not recorded in Swedish national health registers and could therefore not be included as a covariate in the present study [27]. The Swedish population, however, is predominantly of Northern European ancestry. These differences in ethnic composition should be considered when comparing AFF rates across populations and may partly account for the observed variation between cohorts.

Issues in treatment classification—the Swedish cohort

In the Swedish cohort, manual review of medical records did not reveal any additional BP use. One factor that likely contributed to this consistency was the very low prevalence of parenteral BP use in this population (0.20%, Table 4). The high validity of oral drug data in the SNPDR, including BPs, has been demonstrated in earlier studies and was confirmed by our findings [21, 28]. That said, given the increasing prevalence of parenteral treatment in Sweden [29] our results may not fully reflect more recent prescribing patterns. In future, further study of Swedish patients should rely not only on the past high validity of oral drug data but should attempt to take into account parenteral BP exposure as well.

Table 4.

Prevalence of bone modifying medications in the population and hip fracture patients in the Swedish cohort. Prevalences are given as the mean of the study period.

Drug type	Any bisphosphonate	Alendronate	Risedronate	Ibandronate	Etidronate	Zoledronic acid	Hormone replacement therapy	Denosumab	Selective estrogen receptor modulator	Any osteoporosis medication	Clodronate	PTH analogs	Strontium ranelate
Sweden 2005–2010, n (%)
Random sample of adults (n=282 937) *	15 576 (5.5)	12 768 (4.5)	3 168 (1.1)	167 (0.1)	573 (0.2)	446 (0.2)	47 259 (16.7)	5 (0.002)	577 (0.2)	58 170 (20.6)	59 (0.02)	104 (0.04)	256 (0.09)
Hip fracture (n=5 233) **	746 (14.3)	617 (11.8)	137 (2.6)	2 (0.04)	36 (0.7)	27 (0.5)	897 (17.1)	1 (0.02)	25 (0.5)	1 461 (27.9)	2 (0.04)	2 (0.04)	21 (0.4)

Individuals included may take multiple medications.

^*Random population sample (n=282 937) of Swedish adults 55 years or older (n=2 891 586).

^**Hip fracture patients (n= 5 233) identified using ICD-10 diagnostics code (S720 + S721) from population sample (n=282 937) of Swedish adults 55 years or older (n=2 891 586).

Issues in treatment classification—the KPSC cohort

In the KPSC cohort, particularities of the US healthcare system made treatment validation complex. In Europe, medications are identified consistently using Anatomical Therapeutic Chemical (ATC) codes. In the US, medications are identified variably by generic name, brand name, National Drug Code (NDC), Generic Product Identifier (GPI) codes, or codes developed within a given healthcare system. None of these systems map cleanly onto each other, creating ample opportunity for misspecification [20]. Some BPs are also marketed under different brand names for separate indications (e.g., osteoporosis vs. cancer) at different dosing schedules but under the same generic name, further complicating identification. Thorough US pharmacy searches therefore require complex combinations of names, codes, dosages, and dosing instructions to ensure complete capture [20].

Fortunately, since AFF is rare, in our study the number of individuals in the cohorts was small enough that it was possible to search for evidence of BP exposures in the electronic health records (EHR) for each individual KPSC patient. This allowed verification of BP dispensation records in the patients’ pharmacy data. We had to carefully review our coding, however, and when we did so, we found that some of the numerous search terms had been omitted. Correcting our algorithm finally resulted in correct BP exposure classification for all AFF patients in the KPSC cohort.

Our experience aligns with the limitations stated by the authors in the original publication [3] and show a limitation when sourcing treatment data solely from electronic dispensing records. While the specific sources of misclassification identified in our study reflect the data architecture of KPSC, the underlying challenges are not unique to this system. The complexity of drug coding in the US, reliance on multiple non-interoperable coding systems [20], and the potential for patients to receive care outside the system of record are challenges common to all US healthcare databases. The high validity of oral drug data in centralized national registers such as the Swedish SNPDR has been demonstrated [21, 28], whereas comparable validation of US pharmacy data has been limited [19]. In a previous study [19], uncorrected classification of oral BP use led to substantial underestimation of treatment adherence, particularly in long-term care settings, but accuracy of treatment effect estimates was significantly improved after data cleaning.

A US Food and Drug Administration (FDA) guidance document recommends the gold-standard validation technique of manual chart review for real-world data studies to improve causal inference [30]. Based on our experience studying AFFs, such manual review and validation is recommended and appears feasible when studying sufficiently rare outcomes [22]. Our findings should therefore serve as a cautionary example with broad relevance for pharmaco-epidemiological studies relying on electronic pharmacy data.

Issues in diagnosis classification—all cohorts

Investigating for potential misclassification of treatment in relation to BP exposure, despite the challenges noted above, was a worthwhile first step in trying to address the wide variation of reported rates of BP-naïve AFF. Indeed, this approach has been vindicated by the discovery that significant misclassification of BP exposure had occurred. Our study demonstrated that greater methodological rigor can be pursued in such cases. However, our finding that improved data accuracy widened the variation in reported rates of BP-naïve AFF, rather than narrowing it as we had hypothesized, suggests that it might also be necessary to cast a still-wider net in the effort to find explanations for such wide variation.

One area that warrants further investigation may be the AFF case definition. ASBMR case definitions have shown variability in interpretation and application [12–16]. Radiographic adjudication may not always distinguish AFF from insufficiency or stress fractures unrelated to BP use, where bone remodeling is altered by other metabolic disease or by mechanical overload on healthy bone [31]. Depending on recruitment, some studies may capture more insufficiency or fatigue fractures and others more BP-related AFF. Despite the relatively high reliability of the most rigorous prevailing approaches to diagnostic classification, a shift toward even greater precision in diagnosis and treatment might help narrow unexplained variation.

Strengths and limitations

Given the rarity of AFF, a major strength of this study is the use of two large cohorts, allowing investigation of a substantial number of patients. Importantly, the cohorts also shared key methodological features. In both, femur fractures were first identified using ICD diagnosis codes and then confirmed as AFF through expert radiographic adjudication according to ASBMR criteria, providing high diagnostic reliability. In addition, both studies used centrally collected electronic pharmacy data within their respective healthcare systems.

Both settings had a high degree of data integration. Sweden’s publicly funded system centralises healthcare encounters and dispensed medications in national registers. KPSC is an exception in the US: a fully integrated delivery system combining insurance, hospitals, and physician services in a closed network, yielding medical and pharmacy data completeness comparable to nationalised systems, allowing longitudinal tracking of healthcare encounters and prescriptions.

Several limitations should be considered. Despite similarities in data integration, it was not possible to apply a fully standardized approach to medical record review across cohorts. In the Swedish cohort, access to medical records depended on local providers, and in two cases data could not be obtained due to regulatory restrictions. Direct patient contact was not permitted. In the KPSC cohort, although professional data extractors reviewed records, patients may have received care or medications outside the system. Consequently, incomplete capture of exposure information cannot be entirely excluded.

Finally, residual confounding and unrecognized biases may remain in both cohorts and should be considered when interpreting the findings.

Conclusion

Despite the investigation of BP exposure classification for AFF patients initially reported as BP-naïve, substantial discrepancies in the updated proportion of BP-naïve AFF patients between the Swedish and KPSC cohorts not only persisted after manual medical record review, but widened further. Our findings suggest that discrepancies in BP exposure reporting between cohorts can only partly be explained by inaccuracies in data acquisition, indicating that other contextual or methodological factors likely contribute to the observed variation. This emphasizes the importance of multiple drug ascertainment strategies to ensure reliable data for adequate risk-benefit analysis with BP treatment.

Importantly, however, the strong association between BP use and AFF consistently demonstrated in numerous studies remains unchanged, and reinforces the biological plausibility and clinical relevance of this association regardless of differences in study design, cohort composition, or healthcare system context.