A nomogram for predicting immunoglobulin-resistant Kawasaki disease in children

Abstract

Objective

This case–control study focused on the establishment and internal validation of a risk nomogram for intravenous immunoglobulin (IVIG)-resistant Kawasaki disease (KD) using the Kawasaki Disease Database.

Methods

The Kawasaki Disease Database is the first public database for KD researchers. A prediction nomogram for IVIG-resistant KD was constructed using multivariable logistic regression. Then, the C-index was used to assess the discriminating ability of the proposed prediction model, a calibration plot was drawn to evaluate its calibration, and a decision curve analysis was adopted to assess its clinical usefulness. Bootstrapping validation was performed for interval validation.

Results

The median ages of IVIG-resistant and -sensitive KD groups were 3.3 and 2.9 years, respectively. Predicting factors incorporated into the nomogram were coronary artery lesions, C-reactive protein, percentage of neutrophils, platelets, aspartate aminotransferase, and alanine transaminase. Our constructed nomogram exhibited favorable discriminating ability (C-index: 0.742; 95% confidence interval: 0.673–0.812) and excellent calibration. Moreover, interval validation achieved a high C-index of 0.722.

Conclusions

The as-constructed new IVIG-resistant KD nomogram that incorporated C-reactive protein, coronary artery lesions, platelets, percentage of neutrophils, alanine transaminase, and aspartate aminotransferase may be adopted for predicting the risk of IVIG-resistant KD.

Introduction

Kawasaki disease (KD), a systemic self-limiting angiitide, usually occurs in children aged less than 5 years. Coronary artery abnormality is a major complication of KD and has been identified as a major factor leading to acquired heart disease.¹ At present, acute KD is primarily treated with 2 g/kg intravenous immunoglobulin (IVIG) and a high dose of aspirin. However, 10% to 20% of patients do not respond to this treatment and exhibit persistent fever. Additionally, other treatments do not have satisfactory efficacy in patients who have failed the original IVIG therapy and the incidence of coronary artery abnormality remains remarkably high.^2,3 Consequently, identifying the high-risk pediatric population is urgently required.

The low incidence of KD makes the recruitment of eligible patients challenging. To overcome this issue, the Korean Kawasaki Disease Genetics Consortium (KKDGC) was founded in 2008 to collect the clinical data of KD patients for large studies. Moreover, the Kawasaki Disease Database (http://www.kawasakidisease.kr/) was established to facilitate the utilization of clinical data obtained by the KKDGC.⁴ At present, nomograms are extensively utilized to predict the prognostic outcomes of diseases such as cancer, thus assisting clinicians in vital decision-making regarding treatment.⁵ Nomograms show higher accuracy and superior performance compared with previously proposed prediction methods⁶ and provide busy clinicians with a simple approach to predicting functional outcomes. Therefore, this study focused on establishing a prediction nomogram for distinguishing IVIG-resistant KD cases from IVIG-sensitive KD cases.

Materials and methods

Data sources and sample collection

This study conformed to the principles of the Declaration of Helsinki and was approved by the Ethics Committee of The First People’s Hospital of Yangtze University (No.KY20211201). The Kawasaki Disease Database covers the clinical data of KD cases from 13 hospitals involved in the KKDGC. Plasma samples extracted from blood and related clinical data are included in all samples. The KKDGC members had designed a data collection form in which 51 variables were used to obtain clinical data from KD patients. The data collection form included the following categories: (i) personal details, (ii) clinical symptoms and signs, and (iii) laboratory test results. To protect patient privacy, the institutional review boards of every institute involved in blood and clinical sample collection approved each experiment and protocol of this project. Patients with incomplete clinical data were excluded. The reporting of this study conforms to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.⁷

Statistical analysis

Data including disease, demographic, and treatment information were presented as counts and percentages. R software (Version 4.0.4; https://www.R-project.org) was utilized for statistical analysis.

The least absolute shrinkage and selection operator (LASSO) algorithm is used for dimensionality reduction for high-dimensional data. In this study, the LASSO algorithm was adopted to select the best prediction risk factors in IVIG-resistant KD patients.⁸ Characteristics whose coefficients were not zero in the LASSO model were chosen.⁹ Then, selected characteristics were incorporated to construct a prediction model for multivariable logistic regression. Possible predicting factors were then used to build a risk prediction model for IVIG-resistant KD cases.

Calibration of the constructed nomogram was performed by plotting the calibration curves to predict the risk of IVIG-resistant KD. The significant test statistic suggested imperfect model calibration.¹⁰ We determined the Harrell’s C-index to quantify the discriminating ability of the constructed IVIG-resistant KD risk prediction nomogram. Then, we conducted bootstrapping validation with 1000 iterations on the IVIG-resistant KD prediction nomogram to determine the relatively adjusted C-index.¹¹ Furthermore, we measured the clinical usefulness of the constructed nomogram using decision curve analysis to measure the net benefits associated with several threshold probabilities.¹²

Results

Patient features

In total, 507 KD cases were enrolled and classified as IVIG-resistant or -sensitive KD. IVIG-resistant KD was defined as KD patients with a persistent or recurrent fever ≥38°C at any time between 36 hours and 2 weeks after initial IVIG treatment accompanied by one or more main symptoms.¹ Table 1 summarizes patient data, which includes demographic, disease, and treatment characteristics.

Table 1.

Differences between demographic and clinical characteristics of IVIG-resistant and IVIG-sensitive KD groups.

	Differences		n (%)
Characteristics		IVIG resistance KD (n = 59)	IVIG sensitive KD (n = 448)	Total (n = 507)
Age, years	≤1	8 (13.60)	180 (40.18)	188 (37.08)
	>1	51 (86.40)	268 (59.82)	319 (62.92)
Sex	Female	17 (40.48)	101 (22.54)	118 (23.27)
	Male	42 (59.52)	347 (77.46)	389 (76.73)
KD type	Complete	46 (77.97)	342 (76.34)	388 (76.53)
	Incomplete	13 (22.03)	106 (23.66)	119 (23.47)
CAL	Yes	13 (22.03)	41 (9.15)	54 (10.65)
	No	46 (77.97)	407 (90.85)	453 (89.35)
CRP	≤10	25 (42.37)	339 (75.67)	364 (71.79)
	>10	34 (57.63)	109 (24.33)	143 (28.21)
N	≥80%	27 (45.76)	76 (16.96)	103 (20.32)
	<80%	32 (54.24)	372 (83.04)	404 (79.68)
Platelet	≤300	27 (45.76)	156 (34.82)	183 (36.09)
	>300	32 (54.24)	292 (65.18)	324 (63.91)
AST	≥100	16 (27.12)	68 (15.18)	84 (16.57)
	<100	43 (72.88)	380 (84.82)	423 (83.43)
ALT	≥80	26 (44.07)	128 (28.57)	154 (30.37)
	<80	33 (55.93)	320 (71.43)	353 (69.63)

IVIG: intravenous immunoglobulin; KD: Kawasaki disease; CAL: coronary artery lesions; CRP: C-reactive protein; N: neutrophils; AST: aspartate aminotransferase; ALT: alanine transaminase.

Selection of Characteristics

In total, 19 demographic, disease, and treatment characteristics were combined into six possible predicting factors based on 507 cases (ratio: ∼4:1; Figure 1, a and b) and the non-zero coefficients obtained from the LASSO model. Characteristics were coronary artery lesions, C-reactive protein, percentage of neutrophils, platelets, aspartate aminotransferase, and alanine transaminase (Table 2).

Figure 1.

Demographic and clinical feature selection using the LASSO binary logistic regression model. (a) Optimal parameter (lambda) selection in the LASSO model used fivefold cross-validation via minimum criteria and (b) LASSO coefficient profiles of the relevant features. LASSO: least absolute shrinkage and selection operator.

Table 2.

Factors predicting the risk of IVIG-resistant KD.

	Prediction model
Intercept and variable	β	Odds ratio (95% CI)	P -value
Intercept	1.859	6.415 (2.785–15.906)	<0.001
CAL	1.077	2.935 (1.340–6.167)	0.005
CRP	−0.785	0.456 (0.248–0.835)	0.011
N, %	−1.373	0.253 (0.135–0.476)	<0.001
Platelet	0.236	1.267 (0.691–2.296)	0.439
AST	−0.562	0.570 (0.247–1.321)	0.187
ALT	−0.256	0.775 (0.378–1.644)	0.493

IVIG: intravenous immunoglobulin; KD: Kawasaki disease; CAL: coronary artery lesions; CRP: C-reactive protein; N: neutrophils; AST: aspartate aminotransferase; ALT: alanine transaminase.

Establishment of the prediction nomogram

Table 2 shows the logistic regression results for C-reactive protein, coronary artery lesions, platelets, percentage of neutrophils, aspartate aminotransferase, and alanine transaminase. We then established a model incorporating these independent predicting factors and displayed the model in the form of a nomogram (Figure 2).

Figure 2.

Risk nomogram developed for intravenous immunoglobulin-resistant Kawasaki disease.

Excellent performance of the IVIG-resistant KD risk prediction nomogram

The calibration curve for the as-constructed IVIG-resistant KD risk prediction nomogram showed favorable consistency in predicting IVIG-resistant KD risk (Figure 3). The prediction nomogram achieved a C-index value of 0.742 (95% CI: 0.673–0.812), which was revised to 0.722 after bootstrapping validation and suggested a favorable discriminating ability for the model. The constructed IVIG-resistant KD risk prediction nomogram further achieved favorable prediction ability.

Figure 3.

Calibration curves of the intravenous immunoglobulin-resistant Kawasaki disease risk nomogram prediction in the study. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram; a closer fit to the diagonal dotted line represents a better prediction.

Clinical usefulness

Figure 4 displays the decision curve analysis of the constructed IVIG-resistant KD nomogram. As shown by the decision curve, at the threshold probabilities of >25% and <87% for the doctor and the patient, respectively, the proposed IVIG-resistant KD prediction nomogram was beneficial in predicting the risk of IVIG-resistant KD. In this range, comparable net benefits were achieved, with some overlaps, with our as-constructed IVIG-resistant KD risk prediction nomogram.

Figure 4.

Decision curve analysis for the intravenous immunoglobulin-resistant Kawasaki disease risk nomogram. The decision curve shows that if the threshold probabilities of a patient and a doctor are 25% and 87%, respectively, using the nomogram to predict intravenous immunoglobulin-resistant Kawasaki disease risk provides more benefit than the intervention-all-patients scheme or the intervention-none scheme.

Discussion

We first constructed a nomogram to predict the risk of IVIG-resistant KD in the Korean population. The new IVIG-resistant KD risk prediction nomogram was established and validated in this study using six available variables. Risk factors related to treatment and disease characteristics were included in the nomogram to predict individual IVIG-resistant KD risk. Our results offer a precise approach to predicting the risk of IVIG-resistant KD among KD cases. According to our internal validation, the nomogram we constructed displayed favorable calibration and discriminating ability. In particular, a high C-index value was obtained from internal validation, suggesting that the as-constructed nomogram may be extensively and precisely adopted given the large sample size.¹

As revealed by our constructed nomogram, C-reactive protein >10, coronary artery lesions, platelets ≤300, neutrophils ≥80%, alanine transaminase ≥80, and aspartate aminotransferase ≥100 were identified as the critical factors for determining the risk of IVIG-resistant KD among KD cases. Liu et al. were the first to establish the nomogram model of IVIG-resistant KD in China at the end of 2021. In their study, 1240 patients with common KD and 158 children with IVIG-resistant KD were selected. The calculation using a machine learning model showed that total bilirubin, procalcitonin, alanine aminotransferase, and platelet count were independent risk factors for IVIG-resistant KD. Study results supported that the nomogram model of IVIG-resistant KD had good accuracy and predictive ability (C-index = 0.87).¹³ In early 2022, Huang et al. established a nomogram model of IVIG-resistant KD for the cities of Suzhou and Fuzhou in Eastern China. In their study, 1293 patients with ordinary KD and 158 children with IVIG-resistant KD were selected and nine predictors (hemoglobin, percentage of neutrophils, C-reactive protein, platelet count, serum albumin, serum sodium, serum alkaline phosphatase, coronary artery injury, and incomplete KD) were screened using LASSO regression. The nomogram model of IVIG-resistant KD established according to the predictors above had good predictive ability (C-index = 0.75).¹⁴ Differing scoring systems have differing predicted energy efficiency in the same country, region, or population. Our study attempted to establish a nomogram model of IVIG-resistant KD based on the Korean population that would be of practical significance in guiding clinical work.

Certain study limitations should be noted. First, we obtained data from only one database that may not represent all KD cases; moreover, our study did not represent all IVIG-resistant KD cases. Second, our as-constructed nomogram exhibited excellent robustness in internal validation as verified by bootstrap testing. However, external validation was not performed. Therefore, the generalizability of the nomogram to KD populations in other regions is unknown.

Overall, this work establishes a new nomogram with high accuracy that may be used to predict the risk of IVIG-resistant KD among KD cases upon treatment initiation. Nonetheless, external validation with additional KD populations is needed.