Predicting the individualized risk of nonadherence to zoledronic acid among osteoporosis patients receiving the first infusion of zoledronic acid: development and validation of new predictive nomograms

Abstract

Introduction:

Achieving optimal adherence to zoledronic acid (ZOL) among osteoporosis (OP) patients is a challenging task. Here, we aimed to develop and validate a precise and efficient prediction tool for ZOL nonadherence risk in OP patients.

Methods:

We prospectively collected and analyzed survey data from a clinical registry. A total of 1010 OP patients treated for the first time with ZOL in two separate hospitals were selected for nonadherence analysis. The evaluation included a 16-item ZOL Nonadherence Questionnaire and potential risk factors for ZOL nonadherence were assessed via univariate and multivariate analyses. We next developed and validated two distinct-stage nomograms. Discrimination, calibration, and clinical usefulness of the predicting models were assessed using the area under the curve (AUC), calibration curves, and decision curve analysis (DCA).

Results:

The total nonadherence rate was 20.30% after the first ZOL infusion. To generate a model predicting ZOL nonadherence risk, six predictors of 16 items were retained. Model 2 (AUC, 0.8486; 95% confidence interval [CI], 0.8171–0.8801) exhibited considerably more discrimination in desirable functional outcomes, relative to Model 1 (AUC, 0.7644; 95% CI, 0.7265–0.8024). The calibration curves displayed good calibration. DCA revealed that a cutoff probability of 5–54% (Model 1) and 1–85% (Model 2) indicated that the models were clinically useful. External validation also exhibited good discrimination and calibration.

Conclusions:

This study developed and validated two novel, distinct-stage prediction nomograms that precisely estimate nonadherence risk among OP patients receiving the first infusion of ZOL. However, additional evaluation and external validation are necessary prior to widespread implementation.

Introduction

Osteoporosis (OP) is manifested by low bone mass and disintegrated bone microarchitecture that severely enhance fracture risk. ¹ In mainland China, OP is diagnosed in 30–40% women and 10–20% men over 50 years of age. ² Based on the large population in China, OP and its associated fractures signify a massive societal expense that rises each year with a growing aged population. ³ In addition, OP and related fracture management also produces a considerable economic burden that is on a steep rise all over the world. ⁴

Unfortunately, low adherence is far too common in authorized anti-OP therapies. In a prior report, only ~50% of patients complied with the anti-OP therapeutic regimen after 1 year and about a third followed the regimen after 2 years. ² As expected, nonadherence to anti-OP therapy is strongly related to elevated fracture risk, morbidity, mortality, and expenses. ⁵ Fortunately, altering dosing interval of the anti-OP medication to weekly or monthly applications can improve patient compliance, but it still remains suboptimal.^6–8 Zoledronic acid (ZOL), the most commonly used intravenous bisphosphonates for diagnosed osteoporosis, suppresses osteoclast-driven bone resorption and lowers vertebral fracture risk. ⁹ But, achieving optimal clinical efficacy requires the patient to return for the next infusion. However, a number of studies reported that 30–60% OP patients discontinued ZOL treatment by the end of the first year, following initiation.^10–13

The reasons for ZOL nonadherence are multifactorial. It not only involves patients, but is also dependent on the specified therapy or healthcare system, which may influence a patient’s ability to adhere to the treatment regimen. ¹⁴ The World Health Organization (WHO) highlights the multifaceted causes of medication nonadherence, ¹⁵ which include socio-economy, patient, therapy, condition, and healthcare system. ¹⁵ Often times, nonadherence is the end result of multiple causes. Moreover, a patient’s perception and motivation for compliance depends on a self-perceived risk–benefit ratio. ¹⁵ Here, we generated a nonadherence prediction model, using readily available and relevant data, to aid clinicians in personalizing treatment strategies that encourage compliance.

Methods

Study patients

This is a retrospective evaluation of ZOL nonadherence, based on medical data from a clinical registry. This work is registered in the Chinese Clinical Trial Registry (ChiCTR2000036375). Among 1595 consecutive OP patients treated with ZOL in the Affiliated Kunshan Hospital of Jiangsu University (AKSHJU), between March 2018 and June 2020, we chose, for analysis, patients who met our strict inclusion criteria, the details of which are as follows: (1) the first infusion of ZOL (n = 1328) and (2) able to monitor and record data, as required by this study (n = 1221). In all, 211 patients were eliminated from analysis. The reasons were as follows: (1) allergy to any components of ZOL (n = 12), (2) serum total calcium level of < 2.13 mmol/L (8.5 mg/dL) (n = 30), (3) prior usage of other types of bisphosphonates (n = 91), (4) atrial fibrillation and other serious cardiovascular diseases (n = 34), and (5) loss to follow-up (n = 44). Ultimately, our analysis included 1010 patients. Among them, 756 patients from the main hospital of AKSHJU were assigned to the development set (DS), which included 157 nonadherent patients and 599 adherent patients. In addition, 254 patients from the You-yi branch hospital of AKSHJU were recruited as the validation set (VS), and it included 48 nonadherent and 206 adherent patients (Figure 1). We obtained written informed consent from all study subjects or their caregivers and our work received approval from the ethical committee at the Affiliated Kunshan Hospital of Jiangsu University (approval No. 2021-06-015-K01). Patient information was initially documented for hospital’s quality improvement purposes. Data analyzers were blinded to the identity of patients. Finally, we followed the guidelines of the Declaration of Helsinki and its recent amendments.

Figure 1.

Flowchart.

OP, osteoporosis; ZOL, zoledronic acid.

OP management

The OP treatment regimen followed institutional protocols and were guided by doctors directly involved with patient cases. All patients received routine OP management services or OP liaison services (OLS), which entailed extensive assessment and care encompassing two primary services: medication management (MMS) and fracture liaison (FLS). Enrollment in the FLS program at AKSHJU was based on the following criteria: (1) age ⩾ 50 years; (2) newly identified fragility fracture; (3) local resident and expected to complete the 2-year outpatient follow-up evaluations. Patients who experienced fracture from major trauma, cancer, femoral shaft atypical fracture, or those who participated in a different fracture or OP therapy study were eliminated from this study. Patients were recruited for the MMS program if they met the following requirement: (1) aged ⩾ 50 years; (2) newly initiated OP medication or poor compliance to current OP medication; (3) local resident expected to complete the 2-year outpatient follow-up evaluations. The exclusion criteria for this program resembled that of the FLS program.

Our hospital specially developed an electronic system for the OLS program, which consists of three independent modules: coordinators, medical staffs, as well as patients and their families (Software Copyright ID Number: 2021SR0760693, 2021SR0802762, and 2021SR0791538, CN). The FLS program of the electronic management system was designed according to the 13 Best Practice Framework (BPF) standards established by the International Osteoporosis Foundation (IOF).^16,17 These actions allowed coordinators to readily coordinate the electronic systems for standardized, personalized, and efficient OP management.

Survey structured questionnaire

A multidisciplinary team of experts developed this structured, self-administered questionnaire (16-item ZOL Nonadherence Questionnaire [ZOLNQ-16], see Supplementary File 1) related to the potential predictors of medication nonadherence. The potential predictors were stratified according to the WHO’s five categories of medication adherence. ¹⁵ The effectiveness of ZOLNQ-16 was examined. Our aim was to test this questionnaire under different conditions to assess its precision and feasibility. The ZOLNQ-16 questionnaire consists of 16 items encompassing socio-economic (4 items), patient-associated (5 items), therapy-associated (3 items), healthcare system-associated (2 items), and condition-associated (2 items) areas. ZOLNQ-16 can be divided into two stages, based on the patients’ follow-up time in relation to the initial ZOL treatment. The stage 1 questionnaire can be completed as early as 1 week after initial ZOL infusion, and it includes items 1 to 13. The stage 2 questionnaire can be completed between 1 month and 12 months after the initial ZOL infusion, and it includes items 1 to 16. The ZOLNQ-16 is designed to be easy to complete and attain optimal adherence and results. Hence, it consists of a small number of items that takes approximately 8 min to answer.

Definition of items

The detailed definitions of items in ZOLNQ-16 are summarized in Table 1. Musculoskeletal pain evaluation was based on a numeric rating scale (NRS; 0 = no pain, 10 = the worst possible pain) ¹⁸ before (NRS1, item 10) and after (NRS2) ZOL treatment. Since the baseline NRS scores were vastly different among patients, the difference value between the two NRS (NRSD, NRS2 − NRS1; item 16) was employed to assess the extent of pain relief. Acute-phase response (APR, item 11) was defined as fever, musculoskeletal pain, and other flu-like symptoms occurring within 72 h of ZOL infusion.^19,20

Table 1.

Sixteen-item Zoledronic Acid Nonadherence Questionnaire (ZOLNQ-16).

Item	Factor	Variable	Definition
1	Socio-economic factor	Gender
2	Socio-economic factor	Age
3	Socio-economic factor	BMI
4	Socio-economic factor	Insurance	Receiving medical insurance for ZOL purchase.
5	Condition-related factor	Diabetes	Previous use of glucose-lowering medication or glycosylated hemoglobin A1c ⩾ 6.5%
6	Condition-related factor	Hypertension	Previous use of antihypertensive medication, systolic blood pressure > 140 mm Hg, diastolic blood pressure > 90 mm Hg
7	Patient-related factor	Smoking	Current or ex-smoker for the past 12 months
8	Patient-related factor	Drinking	Drinking once or more per week for the past 12 months
9	Patient-related factor	Previous fracture	Suffering from fragile fractures prior to ZOL treatment
10	Patient-related factor	NRS1	NRS score before ZOL infusion
11	Therapy-related factor	APR	Fever, musculoskeletal pain and other flu-like symptoms occurred within 72 h after ZOL infusion
12	Healthcare team and system-related factor	Prophylaxis for APR	Receiving NSAID to relieve the pain and fever for prophylactic reasons immediately following ZOL infusion
13	Healthcare team and system-related factor	OLS	Receiving osteoporosis liaison services
14	Patient-related factor	Inconvenience	Perception of medication regime being complicated
15	Therapy-related factor	Fracture	Any fracture after ZOL infusion
16	Therapy-related factor	NRSD	NRSD = NRS2 − NRS1; NRS2, NRS score after ZOL infusion

APR, acute-phase response; BMI, body mass index; NSAID, nonsteroidal anti-inflammatory drug; NRS, numeric rating scale; OLS, osteoporosis liaison service; ZOL, zoledronic acid.

Data collection

All patients completed ZOLNQ-16 at 12 months after the initial ZOL infusion by themselves or with the help of others. Data were prospectively collected over the phone or electronically by highly skilled OP coordinators. At 14 months after initial ZOL infusion, all patients were contacted to confirm whether the next expected ZOL dose was administered or not. Nonadherence was defined as a delay of >56 days in receiving the next expected dose.^21,22 All participating coordinators were specially trained to maintain standardization of study procedures to minimize systematic error.

Statistical analysis

Data analyses were done with the R Studio software (R version 3.6.1) and a p value < 0.05 was set as the significance threshold. The 16-item characteristics between adherence and nonadherence cohorts were analyzed via χ² test or independent sample t test. Risk factor analyses employed both univariate and multivariate logistic regression analyses. Statistically significant variables from univariate analysis were entered into multivariate logistic regression analysis (p value < 0.1). Multicollinearity among the covariates was tested by examining whether the variance inflation factor (VIF) is ⩾4, and the multivariate logistic regression analysis was employed to build the models. We developed both early- (Model 1) and late-stage models (Model 2), depending on the stage of predictors included. Along with each model, we also expressed odds ratios with 95% confidence intervals (CI) to signify the impact of each risk factor.

Using independent variables regression coefficients from the DS, we established two nomogram prediction models associated with nonadherence.^23,24 The prediction models were assessed for discrimination and calibration. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve was computed to establish discrimination of both models of nonadherence prediction. In addition, we also established the optimal threshold for nonadherence prediction. Using ROC analysis, we also generated values representing optimal sensitivity and specificity for prediction of a given threshold. We established that an AUC of 0.5 ± 0.75 is acceptable, whereas AUC > 0.75 represents outstanding discrimination. ²⁵ We next determined the integrated discrimination index (IDI) and net reclassification index (NRI) between the two models.^26,27 Significant differences between ROC curves were further confirmed with the DeLong’s test for two associated ROC curves (R package pROC). ²⁸ Calibration was assessed with calibration curves.

Furthermore, the decision curve analysis (DCA) was performed to assess the clinical feasibility of these models. This was done by quantifying the net benefits at various threshold probabilities in the DS. ²⁹ The net benefit was calculated by subtracting the proportion of all patients who are false positive from the proportion of the patients who are true positive and by weighing the relative harm of forgoing interventions compared with the negative consequences of an unnecessary intervention. ³⁰

Then, internal validation was completed using bootstrapping (BS) techniques (1000 BS resampling). Finally, external validation was performed in the VS using ROC and calibration curves using the above methods.

Sample size calculation

We calculated a minimum requirement of 88–107 participants to achieve a 99% power in our ROC investigation, carrying a two-sided α of 0.05, considering that the AUC is typically 0.75, and 30–60% OP patients discontinued ZOL treatment by the end of the first year, after initiation.^10–13

Results

Sample characteristics

Please refer to Table 2 for all patient information, including the ZOL nonadherence risk-related 16 item characteristics from DS and VS. The ages of all participants were between 50 and 95 years, with an average of 68.12 years (SD 9.08). Among them, 82.87% (n = 837) were female. In addition, the total nonadherence rate was 20.30% after the first infusion of ZOL.

Table 2.

The 16-item characteristics of the development and validation sets.

Item	Variables	Development set (n = 756)			Validation set (n = 254)
		Adherence (n = 599)	Nonadherence (n = 157)	p value	Adherence (n = 206)	Nonadherence (n = 48)	p value
1	Gender, n (%)			0.147			0.04
	Male	100 (16.69%)	34 (21.66%)		27 (13.11%)	12 (25.00%)
	Female	499 (83.31%)	123 (78.34%)		179 (86.89%)	36 (75.00%)
2	Age, n (%)			0.597			0.147
	<60 years	105 (17.53%)	34 (21.66%)		31 (15.05%)	14 (29.17%)
	60–69 years	224 (37.40%)	60 (38.22%)		84 (40.78%)	17 (35.42%)
	70–79 years	195 (32.55%)	46 (29.30%)		60 (29.13%)	11 (22.92%)
	>79 years	75 (12.52%)	17 (10.83%)		31 (15.05%)	6 (12.50%)
3	BMI, mean (SD), min, kg/m2	22.12 (3.07)	21.98 (3.40)	0.632	22.25 (2.89)	22.04 (2.91)	0.649
4	Insurance, n (%)			< 0.001			< 0.001
	No	74 (12.35%)	64 (40.76%)		19 (9.22%)	23 (47.92%)
	Yes	525 (87.65%)	93 (59.24%)		187 (90.78%)	25 (52.08%)
5	Diabetes, n (%)			0.298			0.138
	No	533 (88.98%)	135 (85.99%)		190 (92.23%)	41 (85.42%)
	Yes	66 (11.02%)	22 (14.01%)		16 (7.77%)	7 (14.58%)
6	Hypertension, n (%)			0.918			0.737
	No	445 (74.29%)	116 (73.89%)		154 (74.76%)	37 (77.08%)
	Yes	154 (25.71%)	41 (26.11%)		52 (25.24%)	11 (22.92%)
7	Smoking, n (%)			0.484			0.641
	No	531 (88.65%)	136 (86.62%)		185 (89.81%)	42 (87.50%)
	Yes	68 (11.35%)	21 (13.38%)		21 (10.19%)	6 (12.50%)
8	Drinking, n (%)			0.993			0.715
	No	534 (89.15%)	140 (89.17%)		188 (91.26%)	43 (89.58%)
	Yes	65 (10.85%)	17 (10.83%)		18 (8.74%)	5 (10.42%)
9	Previous fracture, n (%)			0.877			0.589
	No	389 (64.94%)	103 (65.61%)		142 (68.93%)	35 (72.92%)
	Yes	210 (35.06%)	54 (34.39%)		64 (31.07%)	13 (27.08%)
10	NRS1, n (%)			0.157			0.452
	No pain (0)	24 (4.01%)	10 (6.37%)		11 (5.34%)	4 (8.33%)
	Mild pain (1–3)	234 (39.07%)	49 (31.21%)		75 (36.41%)	13 (27.08%)
	Moderate pain (4–6)	278 (46.41%)	84 (53.50%)		94 (45.63%)	22 (45.83%)
	Severe pain (7–10)	63 (10.52%)	14 (8.92%)		26 (12.62%)	9 (18.75%)
11	APR (any of the below symptom clusters), n (%)			< 0.001			0.298
	No	287 (47.91%)	49 (31.21%)		103 (50.00%)	20 (41.67%)
	Yes	312 (52.09%)	108 (68.79%)		103 (50.00%)	28 (58.33%)
	Fever, n (%)			< 0.001			0.23
	No	378 (63.11%)	74 (47.13%)		143 (69.42%)	29 (60.42%)
	Yes	221 (36.89%)	83 (52.87%)		63 (30.58%)	19 (39.58%)
	Musculoskeletal pain, n (%)			0.018			0.863
	No	425 (70.95%)	96 (61.15%)		140 (67.96%)	32 (66.67%)
	Yes	174 (29.05%)	61 (38.85%)		66 (32.04%)	16 (33.33%)
	Other flu-like symptoms, n (%)			< 0.001			0.185
	No	528 (88.15%)	115 (73.25%)		179 (86.89%)	45 (93.75%)
	Yes	71 (11.85%)	42 (26.75%)		27 (13.11%)	3 (6.25%)
12	Prophylaxis for APR, n (%)			0.01			0.002
	No	230 (38.40%)	78 (49.68%)		70 (33.98%)	28 (58.33%)
	Yes	369 (61.60%)	79 (50.32%)		136 (66.02%)	20 (41.67%)
13	OLS, n (%)			< 0.001			< 0.001
	No	258 (43.07%)	114 (72.61%)		94 (45.63%)	37 (77.08%)
	Yes	341 (56.93%)	43 (27.39%)		112 (54.37%)	11 (22.92%)
14	Inconvenience, n (%)			< 0.001			0.003
	No	497 (82.97%)	110 (70.06%)		169 (82.04%)	30 (62.50%)
	Yes	102 (17.03%)	47 (29.94%)		37 (17.96%)	18 (37.50%)
15	Fracture, n (%)			< 0.001			< 0.001
	No	595 (99.33%)	132 (84.08%)		206 (100.00%)	45 (93.75%)
	Yes	4 (0.67%)	25 (15.92%)		0 (0.00%)	3 (6.25%)
	NRS2, n (%)			< 0.001			< 0.001
	No pain (0)	56 (9.35%)	12 (7.64%)		25 (12.14%)	4 (8.33%)
	Mild pain (1–3)	344 (57.43%)	62 (39.49%)		101 (49.03%)	16 (33.33%)
	Moderate pain (4–6)	175 (29.22%)	68 (43.31%)		76 (36.89%)	18 (37.50%)
	Severe pain (7–10)	24 (4.01%)	15 (9.55%)		4 (1.94%)	10 (20.83%)
16	NRSD, n (%)			< 0.001			0.008
	<−2	161 (26.88%)	17 (10.83%)		55 (26.70%)	7 (14.58%)
	−2 or −1	201 (33.56%)	48 (30.57%)		67 (32.52%)	13 (27.08%)
	0	122 (20.37%)	43 (27.39%)		44 (21.36%)	10 (20.83%)
	1 or 2	105 (17.53%)	40 (25.48%)		38 (18.45%)	14 (29.17%)
	>2	10 (1.67%)	9 (5.73%)		2 (0.97%)	4 (8.33%)

NRSD = NRS2 − NRS1 (NRS1, NRS score before ZOL infusion; NRS2, NRS score after ZOL infusion).

APR, acute-phase response; BMI, body mass index; NRS, numeric rating scale; OLS, osteoporosis liaison service; ZOL, zoledronic acid.

Nomogram development

Based on univariate analysis, the ZOL nonadherence risk factors included insurance, APR, prophylaxis for APR, OLS, inconvenience, fracture, and NRSD. Variables reaching significance in univariate analysis were then entered into the non-conditional binary multivariate logistic regression. Subsequently, we revealed that insurance, APR, OLS, inconvenience, fracture, and NRSD were independent risk factors for medication nonadherence (Table 3). We next conducted collinearity analysis on the aforementioned independent risk factors (VIFs = 1) and showed no multicollinearity among these factors.

Table 3.

Univariate and multivariate logistic regression models of ZOL nonadherence in the development set.

Item	Variables	Univariate analysis		Multivariate analysis
		OR (95% CI)	p value	OR (95% CI)	p value
1	Gender
	Male	Reference
	Female	0.72 (0.47, 1.12)	0.1485
2	Age
	<60 years	Reference
	60–69 years	0.83 (0.51, 1.34)	0.4389
	70–79 years	0.73 (0.44, 1.20)	0.2169
	>79 years	0.70 (0.36, 1.35)	0.2846
3	BMI	0.99 (0.93, 1.04)	0.6318
4	Insurance
	No	Reference		Reference
	Yes	0.20 (0.14, 0.31)	<0.0001	0.16 (0.10, 0.25)	<0.0001
5	Diabetes
	No	Reference
	Yes	1.32 (0.78, 2.21)	0.2989
6	Hypertension
	No	Reference
	Yes	1.02 (0.68, 1.52)	0.9177
7	Smoking
	No	Reference
	Yes	1.21 (0.71, 2.04)	0.4842
8	Drinking
	No	Reference
	Yes	1.00 (0.57, 1.76)	0.9933
9	Previous fracture
	No	Reference
	Yes	0.97 (0.67, 1.41)	0.8766
10	NRS1
	No pain (0)	1
	Mild pain (1–3)	0.50 (0.23, 1.12)	0.0916
	Moderate pain (4–6)	0.73 (0.33, 1.58)	0.4176
	Severe pain (7–10)	0.53 (0.21, 1.36)	0.189
11	APR
	No	Reference		Reference
	Yes	2.03 (1.40, 2.95)	0.0002	2.46 (1.55, 3.91)	0.0001
12	Prophylaxis for APR
	No	Reference		Reference
	Yes	0.63 (0.44, 0.90)	0.0108	0.72 (0.47, 1.10)	0.1315
13	OLS
	No	Reference		Reference
	Yes	0.29 (0.19, 0.42)	<0.0001	0.25 (0.16, 0.40)	<0.0001
14	Inconvenience
	No	Reference		Reference
	Yes	2.08 (1.39, 3.11)	0.0004	3.02 (1.83, 4.99)	<0.0001
15	Fracture
	No	Reference		Reference
	Yes	28.17 (9.64, 82.31)	<0.0001	37.75 (11.47, 124.28)	<0.0001
16	NRSD
	<−2	Reference		Reference
	−2 or −1	2.26 (1.25, 4.08)	0.0068	2.64 (1.32, 5.28)	0.0062
	0	3.34 (1.82, 6.14)	0.0001	5.19 (2.50, 10.77)	<0.0001
	1 or 2	3.61 (1.94, 6.70)	<0.0001	6.46 (3.05, 13.68)	<0.0001
	>2	8.52 (3.04, 23.87)	<0.0001	19.50 (5.60, 67.89)	<0.0001

NRSD = NRS2 − NRS1 (NRS1, NRS score before ZOL infusion; NRS2, NRS score after ZOL infusion).

APR, acute-phase response; BMI, body mass index; CI, confidence interval; NRS, numeric rating scale; OLS, osteoporosis liaison service; OR, odds ratio; ZOL, zoledronic acid.

Based on the logistic multivariate regression analysis, the six independent risk factors were finally included (Table 4). We next constructed an early-stage nomogram prediction model (Model 1) and a late-stage nomogram prediction model (Model 2) for the prediction of patient ZOL nonadherence risk. Schemes of two developed nomograms are shown in Figure 2. Model 1 can be used as early as 1 week after initial ZOL infusion and it includes early-stage predictors (insurance, APR, and OLS). In contrast, Model 2 can be used between 1 month and 12 months after the initial ZOL infusion, and it was constructed with Model 1 adding late-stage predictors (inconvenience, fracture, and NRSD). Both nomograms are presented in Figure 3. The nomogram can be used to retrieve values for each predictor, which can be summed up to receive a total score. The estimated risk corresponding to the total score reveals the likelihood of ZOL nonadherence.

Table 4.

Multivariable logistic regression models in the development set.

Predictors	Model 1					Model 2
	Estimate	OR	Low 95% CI	High 95% CI	p value	Estimate	OR	Low 95% CI	High 95% CI	p value
Intercept	−0.0010	0.9990	0.6331	1.5764	0.9965	−1.6397	0.194	0.0908	0.4147	<0.0001
Insurance	−1.6906	0.1844	0.1201	0.2833	< 0.0001	−1.8723	0.1538	0.0945	0.2502	<0.0001
APR	0.7587	2.1355	1.4206	3.2102	0.0003	0.9541	2.5964	1.6398	4.1111	<0.0001
OLS	−1.2798	0.2781	0.1845	0.4193	< 0.0001	−1.3911	0.2488	0.1570	0.3943	<0.0001
Inconvenience						1.1082	3.0289	1.8351	4.9993	<0.0001
Fracture						3.6455	38.3008	11.6986	125.3958	<0.0001
NRSD
<−2						Reference
−2 or −1						0.9835	2.6738	1.3354	5.3537	0.0055
0						1.6644	5.2826	2.5446	10.967	<0.0001
1 or 2						1.8743	6.5160	3.0776	13.7962	<0.0001
>2						2.9188	18.5196	5.3332	64.3099	<0.0001

NRSD = NRS2 − NRS1 (NRS1, NRS score before ZOL infusion; NRS2, NRS score after ZOL infusion).

APR, acute-phase response; CI, confidence interval; NRS, numeric rating scale; OLS, osteoporosis liaison service; OR, odds ratio; ZOL, zoledronic acid.

Figure 2.

Schemes of two developed nomograms for the prediction of ZOL nonadherence risk. Model 1 (black) can be used as early as 1 week after initial ZOL infusion and it includes three predictors. Model 2 (red) can be used from 1 month to 12 months after the initial ZOL infusion. Its construction is similar to Model 1, with the addition of three late-stage predictors. NRSD = NRS2 − NRS1 (NRS1, NRS score before ZOL infusion; NRS2, NRS score after ZOL infusion; NRS, numeric rating scale).

APR, acute-phase response; OLS, osteoporosis liaison service; ZOL, zoledronic acid.

Figure 3.

Two distinct-stage individualized nomograms for the prediction of ZOL nonadherence risk in osteoporosis patients. Model 1 (early-stage) includes insurance, APR, and OLS. Model 2 (late-stage) includes insurance, APR, OLS, inconvenience, fracture, and NRSD. The nomogram is used as follows: First, each predictor is located on the corresponding axis, then a line is drawn from the predictor to the points axis to obtain the point value; subsequently, all points from all predictors are added together and another line is drawn from the total point axis to the lower line of the nomogram to establish ZOL nonadherence risk. NRSD = NRS2 − NRS1 (NRS1, NRS score before ZOL infusion; NRS2, NRS score after ZOL infusion; NRS, numeric rating scale).

APR, acute-phase response; OLS, osteoporosis liaison service; ZOL, zoledronic acid.

For example, in Model 1, the scores were provided as follows: an OP patient who received OLS (0 point), had no medical insurance for ZOL purchase (100 points), and experienced APR after ZOL infusion (45 points). The cumulative score of the various predictive indicators was 0 + 100 + 45 = 145, and the corresponding predicted risk of nonadherence was 0.37 (37%) (Figure 4).

Figure 4.

An example prediction nomogram for the estimation of ZOL nonadherence risk in osteoporosis patients.

APR, acute-phase response; OLS, osteoporosis liaison service; ZOL, zoledronic acid.

Model performance

The Model 1 threshold was 0.1865, carrying a sensitivity of 77.07% and specificity of 66.78%. The Model 2 threshold was 0.1672, carrying a sensitivity of 84.08% and specificity of 69.45%. According to the AUC differences between the two models, Model 2 (AUC, 0.8486; 95% CI, 0.8171–0.8801) displayed remarkably high discrimination of desirable functional outcome, compared with Model 1 (AUC, 0.7644; 95% CI, 0.7265–0.8024; Table 5; Figure 5(a)). In addition, to estimate the discrimination performance by late-stage predictors more sensitively, we compared the probabilities of events and nonevents of models using the relative IDI and category-free NRI. Model 2 showed significant improvement of both IDI and NRI as compared with Model 1.

Table 5.

Comparisons of the two models by AUC, IDI, and NRI in the development set.

	Model 1	Model 2	p value
AUC (95% CI)	0.7644 (0.7265, 0.8024)	0.8486 (0.8171, 0.8801)	<0.0001
Relative IDI (95% CI)	Reference	0.0968 (0.0338, 0.1597)	0.0026
Category-free NRI (95% CI)	Reference
Event		0.0701 (0.0139, 0.1262)	0.0145
Nonevent		0.0267 (−0.0014, 0.0548)	0.0621
Total		0.0968 (0.0340, 0.1595)	0.0025

AUC, area under the curve; CI, confidence interval; IDI, integrated discrimination index; NRI, net reclassification index.

Figure 5.

Evaluation of two models for the prediction of ZOL nonadherence risk in the development set. The ROC curve AUC of (a) calculated to assess the discriminative nature of both models. The AUC of Model 1 (black) and Model 2 (red) were 0.7644 (95% CI, 0.7265–0.8024) and 0.8486 (95% CI, 0.8171–0.8801), respectively. There was statistical significance between two models (p < 0.0001). The calibration curves assessed calibrations of Model 1 (b) and Model 2 (c).

AUC, area under the curve; ROC, receiver operating characteristic; ZOL, zoledronic acid.

Based on the internal bootstrap validation method, at a probability of 0–0.45, the Model 1 bias-corrected curve was comparable to the ideal curve. But, at a high probability (>0.45), Model 1 may overestimate the probability of ZOL nonadherence [Figure 5(b)]. Upon addition of late-stage predictors, the Model 2 bias-corrected curve was close to the ideal curve, and good agreement was observed between predictions and observations (Figure 5(c)), demonstrating a good fit.

DCA

The DCA comparing clinical feasibility of Models 1 and 2 are presented in Figure 6. The probability cutoff for ZOL nonadherence was placed in the x-axis, whereas the standard net benefit of employing the model was placed in the y-axis. Based on the DCA, threshold probabilities of 5–54% for Model 1 and 1–85% for Model 2 were more beneficial than the intervention-all or intervention-none schemes. Besides, Models 2 was better than 1 within the limits of this range.

Figure 6.

Decision curve analysis for the nonadherence models. The y-axis represents standardized net benefit. The blue (Model 1) and red (Model 2) line denotes ZOL nonadherence risk models. The thin oblique solid line assumes that all patients are noncompliant to ZOL. The thick horizontal solid line assumes that all patients are compliant to ZOL. Based on the decision curve, a cutoff of 5–54% for Model 1 and 1–85% for Model 2 are far more beneficial to patients than the intervention-all or intervention-none schemes.

ZOL, zoledronic acid.

Model validation

We performed BS-corrected ROC analysis on both models, and the results are presented in Supplementary File 2. High AUC value was still reached in the interval validation. Furthermore, the two models were next externally validated using ROC and calibration curves in VS (Figure 7). The Model 1 AUC for VS was 0.7714 (95% CI 0.6954–0.8473), carrying a sensitivity of 43.75% and a specificity of 97.09% at the optimal threshold. Alternately, the Model 2 AUC for VS was 0.8480 (95% CI: 0.7807–0.9154), carrying a sensitivity of 64.58% and a specificity of 93.20% at the optimal threshold. Finally, the calibration curves of both nomograms exhibited satisfactory agreement between the estimated and observed values.

Figure 7.

External validation for ZOL nonadherence nomograms using ROC curves (a) and calibration curves (b and c) corresponding to the validation set.

ROC, receiver operating characteristic; ZOL, zoledronic acid.

APR risk findings

To further predict APR risk, all potential predictors in stage 1 (item 1 to 13 except 12) were employed to establish a prediction model in the DS by using the above methods. Unfortunately, the performance was poor (AUC, 0.6281; 95% CI, 0.5883–0.6679). Table 6 summarizes the results of univariate and multivariate analyses of APR risk. Based on multivariate analysis of APR risk, age (>79 years old [OR, 0.32; 95% CI, 0.18–0.59; p = 0.0003]), prophylaxis for APR (OR, 0.64; 95% CI, 0.47–0.87; p = 0.0042), and OLS (OR, 0.0212; 95% CI, 0.51–0.05; p = 0.0212) offer independent protection against APR.

Table 6.

Univariate and multivariate analyses of the APR risk in the development set.

Item	Variables	Univariate analysis		Multivariate analysis
		OR (95% CI)	p value	OR (95% CI)	p value
1	Gender
	Male	Reference		Reference
	Female	0.91 (0.62, 1.33)	0.6243	1.19 (0.69, 2.05)	0.5424
2	Age
	<60 years old	Reference		Reference
	60–69 years old	0.84 (0.55, 1.28)	0.4118	0.90 (0.58, 1.38)	0.6204
	70–79 years old	0.69 (0.45, 1.06)	0.0895	0.68 (0.43, 1.06)	0.0875
	>79 years old	0.32 (0.19, 0.56)	<0.0001	0.32 (0.18, 0.59)	0.0003
3	BMI	0.96 (0.92, 1.01)	0.0977	0.97 (0.92, 1.02)	0.2006
4	Insurance
	No	Reference		Reference
	Yes	1.02 (0.71, 1.48)	0.8995	1.18 (0.79, 1.76)	0.4249
5	Diabetes
	No	Reference		Reference
	Yes	0.70 (0.45, 1.09)	0.1172	0.82 (0.49, 1.36)	0.4406
6	Hypertension
	No	Reference		Reference
	Yes	0.60 (0.43, 0.83)	0.0023	0.89 (0.60, 1.31)	0.5472
7	Smoking
	No	Reference		Reference
	Yes	1.27 (0.81, 1.99)	0.3017	1.48 (0.82, 2.68)	0.1978
8	Drinking
	No	Reference		Reference
	Yes	1.21 (0.76, 1.93)	0.418	1.23 (0.71, 2.13)	0.4586
9	Previous fracture
	No	Reference		Reference
	Yes	1.51 (1.11, 2.05)	0.0079	1.36 (0.99, 1.87)	0.0586
10	NRS1
	No pain (0)	Reference		Reference
	Mild pain (1–3)	0.61 (0.29, 1.27)	0.186	0.56 (0.26, 1.20)	0.1369
	Moderate pain (4–6)	0.67 (0.32, 1.40)	0.2905	0.62 (0.29, 1.33)	0.2234
	Severe pain (7–10)	0.95 (0.41, 2.22)	0.9139	0.91 (0.38, 2.17)	0.8309
12	Prophylaxis for APR
	No	Reference		Reference
	Yes	0.63 (0.47, 0.84)	0.0019	0.64 (0.47, 0.87)	0.0042
13	OLS
	No	Reference		Reference
	Yes	0.72 (0.54, 0.96)	0.0249	0.70 (0.51, 0.95)	0.0212

NRS1 indicates NRS score before ZOL infusion.

APR, acute-phase response; BMI, body mass index; CI, confidence interval; NRS, numeric rating scale; OLS, osteoporosis liaison service; OR, odds ratio; ZOL, zoledronic acid.

Discussion

Nomograms are increasing popular as prognostic tools in oncology and medicine. They utilize user-friendly digital interfaces, enhanced precision, and comprehensible prognoses to build enhanced treatment plans. ³¹ This study offers a novel look into the application of nomograms in OP and ZOL nonadherence. We established and validated two new, stage-specific prediction models for nonadherence risk in OP patients receiving the first infusion of ZOL, using merely three or six easily available variables. External validation confirmed satisfactory discrimination and calibration power. In particular, the high Model 2 AUC in VS verified that the nomograms can be extensively and precisely used.

Based on our analysis, about 20% of OP patients failed to receive a second ZOL infusion. This result is consistent with the Curtis et al. ¹⁰ study, where it was reported that nonadherence reached 18% among U.S. Medicare enrollees. In other studies, nonadherence was relatively high, compared with our study. A retrospective observational study, using data from Humana, a US administrative healthcare claims database, revealed that among the 3128 OP patients receiving the first ZOL infusion, nonadherence was 59.2% at 12 months and 79.8% at 24 months. ¹¹ A study of a German database revealed that the proportion of patients that remained on ZOL treatment at 12 months was 65.6% (nonadherence = 34.5%). ¹³ In a study examining 259 Korean patients, only 36% (nonadherence = 64%) received a second ZOL infusion. ¹² Although, compared with other treatment option, an annual intravenous administration of 5 mg ZOL is a highly efficient and favorable, ³² it is not uncommon to discontinue ZOL administration from the second year, based on the above data.

A systematic review identified 24 factors with 139 sub-factors that influence medication compliance in OP patients, based on the WHO’s five-domain framework. ³³ Among the condition-associated factors regulating medication noncompliance was polypharmacy, and enhanced medication compliance usually resulted from a history of falls. Patient-associated factors resulting in poor medication compliance was old age and OP-related misconception. Therapy-associated factors promoting medication noncompliance were frequent dosing and medication side effects. Healthcare system–associated factors driving medication noncompliance were care under different medical specialties and lack of patient education. Finally, among the socio-economic–associated factors influencing medication noncompliance was smoking history and absence of medical insurance coverage. ³³

Similar to the above results, the predictors of ZOL nonadherence identified in our study were insurance, APR, OLS, inconvenience, fracture, and NRSD. The nomograms suggested that lack of medical insurance coverage, APR occurrence, lack of OLS management, perception of medication regime being complicated, new fractures, and no pain relief after treatment are the main factors regulating ZOL nonadherence risk in OP patients.

Lack of medical insurance was the major socio-economic predictor of ZOL nonadherence. This concurs with an additional four studies.^34–37 It is essential to assess patient financial situation before the prescription of expensive anti-OP medications. In case of patients in need of financial aid, they must be referred to appropriate service providers, such as hospital social services for financial assistance. In addition, regional governments must cover cost-effective anti-OP medications in patient medical insurance.

The occurrence of APR was found, unsurprisingly, as a predictor of ZOL nonadherence. This is in accordance with other publications^38–41 that indicated that patients often forgo medications for fear of undesirable side effects. Although an adult may be familiar with the advantages of long-term medical compliance in managing OP, its benefits are not immediately apparent. Therefore, presence of side effects like APR dramatically reduces their motivation to follow through with the treatment regimen. So, regular evaluation and management of APR, with the help of a prediction model, are essential. Despite our analysis demonstrating poor performance of APR risk prediction, several independent protective factors associated with APR were identified. Older age (>79 years old) is among the first independent protective factors of APR, which is in accordance with the Ding et al. ²⁰ study. Our prior assumption was that the relatively low proinflammatory cytokines present in the older population lead to the difference in APR. ⁴² The second independent protective factor for APR was nonsteroidal anti-inflammatory drug (NSAID) usage immediately after ZOL infusion to alleviate pain and fever associated with prophylactics. This is consistent with a clinical study. ⁴³ OLS management was the third independent protective factor of APR. Once patients were enrolled into our hospital OLS, they were advised to drink more water. Adequate hydration may play an important role in reducing APR occurrence. Hence, clinicians can take special steps to mitigate APR prior to therapy or design alternative routes of drug administration under an elevated APR risk. In future, laboratory indicators can also be introduced as predictors, such as serum 25(OH)D levels, to better develop the APR risk prediction model.

Among healthcare team and system-related factors, lack of OLS management was found to be a predictor of ZOL nonadherence. Previous studies demonstrated that early interventions like medication reminders, drug monitoring, and family support can benefit patients with elevated noncompliance risk.^44,45 When it comes to adhering to an infrequent dose intravenous therapy, the most important factor is to ensure the patient is scheduled and shows up to the appointment for an infusion treatment. ⁴⁶ Therefore, employing low-cost reminding tools can augment compliance in high-risk patients. ⁴⁵ Therefore, our healthcare team developed an electronic management system for the OLS program. Prior to the commencement of the first treatment, the electronic OLS system automatically generates a paper liaison leaflet containing the patient’s fracture/BMD/lab data and long-term treatment plan. In addition, coordinators, clinicians, OP patients, and their families can easily log into the electronic system using mobile phones and computers. This electronic OLS system provides intervention, as well as education, exercise training, medication adjustment, and referral to physiatrists. Meanwhile, automated medication reminders can schedule the following infusion appointment and send reminders to the patient to ensure adherence. If a patient fails to respond, the system can remind the coordinator to call by phone. The OLS system also provides a green reservation channel for patients to come to the hospital for regular follow-ups. The final results of our study demonstrated that a dedicated healthcare team and properly designed system are key to enhancing medication compliance in OP patients.

Similar to other studies,^47–49 our analysis also confirmed that a patient’s false perception of a complicated treatment regimen can give rise to medication noncompliance. This may be due to the fact that during the 3–5 years of ZOL treatment, patients need to come to the hospital every year for an intravenous ZOL infusion. In addition, supplements like calcium and vitamin D must be taken throughout the treatment period. Polypharmacy is a well-recognized issue in the older population as they are often afflicted with multiple chronic diseases and are under numerous medications at the same time. ¹⁵ The need to remember and consume large number of medications on a daily basis, over a prolonged period of time, can be overwhelming to the elderly, thus giving the perception of a complicated treatment regimen. This can negatively affect medication compliance. Though ZOL has longer dosing intervals, it is still necessary to strengthen patient education and provide necessary support.

Experiencing fracture and no pain relief after ZOL infusion are two of the main therapy-related indicators of ZOL nonadherence. It may be attributed to a patient’s lack of understanding of therapeutic benefit, which is a well-established factor associated with poor medication adherence.^50–52 Typically, nonadherence increases when treatment does not appear as effective as patients’ expectations. In this case, healthcare providers need to constantly guide patients to the realization that OP is a chronic disease and anti-OP treatment is a long and arduous task. In addition, patients need to be aware that a single fracture while on therapy does not indicate treatment failure. ⁵³ However, it is also necessary for clinicians to consider alternative therapy or reassessment for causes of secondary OP in patients experiencing two or more refractures or significant bone loss during therapy. ⁵³

As expected, OP patients who adhere to the treatment plan experience better outcomes than patients who were nonadherent.^54,55 Therefore, establishing patient nonadherence prediction models can guide personalized risk assessment and interventions. Thus, we established a precise ZOL noncompliance risk prediction tool that can aid clinicians in evaluating the ZOL noncompliance risk, design personalized treatment plans, prevent additional interventions in low-risk patients, and avoid delays or discontinuity in intervention. Supporting interventions like patient education, monitoring and supervision, alteration in medication regimen, providing patient support, and interdisciplinary collaborations, ⁵⁶ or an OLS system (as was done in our hospital) can highly benefit patients at elevated noncompliance risk who are starting their ZOL infusions. Moreover, this can also aid researchers in selecting optimal OP patients (exhibiting good adherence to anti-OP therapy) for clinical research. Likewise, models can also be used to exclude patients with poor compliance in retrospective studies, thereby securing more reliable analysis.

Our work encountered certain limitations. First, the data were mostly self-reported, which introduces certain social desirability and recall bias, and the values may not be accurate. ⁵⁷ However, inaccuracies in reported nonadherence are believed to be minimal since independent, experienced, and trained OP coordinators, not patients’ healthcare providers, were responsible for the follow-up evaluations. Furthermore, in order to minimize potential recall bias, two distinct-stage prediction models were generated, depending on the phase of treatment the patients were in. In addition, from the patient’s perspective, when reporting ZOL nonadherence information, it was believed that ZOL was easily identified during the telephone or network follow-ups, due to its unique characteristics (intravenous administration, once a year). Second, the patients or clinicians were not blinded to the intervention, which may contribute to both selection and observational biases. Third, the follow-up duration was relatively short (14 months). This study did not include assessments of long-term ZOL nonadherence risk. Fourth, risk factor analysis of the current study did not include all potential factors that can regulate ZOL nonadherence. Possible predictors, such as living status and education level, need to be identified in the future. Finally, despite extensive examination of our models with VS, the feasibility of employing our nomograms on OP populations from other regions and countries remains to be examined.

Conclusion

The total nonadherence rate was 20.30% after the first infusion of ZOL. This study developed and validated two novel, distinct-stage prediction nomograms, with satisfactory precision to aid clinicians in accessing nonadherence risk among OP patients receiving the first infusion of ZOL. Lack of medical insurance coverage, APR occurrence, lack of OLS management, perception of medication regime being complicated, new fractures, and no pain relief after treatment are among the main factors determining ZOL noncompliance. Corresponding interventions, such as OLS management, can enhance ZOL compliance in high-risk patients. Additional prospective validation, involving a large patient population, is warranted to confirm our results.