Abstract
Objective
Features of multisystem inflammatory syndrome in children (MIS‐C) overlap with other syndromes, making the diagnosis difficult for clinicians. We aimed to compare clinical differences between patients with and without clinical MIS‐C diagnosis and develop a diagnostic prediction model to assist clinicians in identification of patients with MIS‐C within the first 24 hours of hospital presentation.
Methods
A cohort of 127 patients (<21 years) were admitted to an academic children's hospital and evaluated for MIS‐C. The primary outcome measure was MIS‐C diagnosis at Vanderbilt University Medical Center. Clinical, laboratory, and cardiac features were extracted from the medical record, compared among groups, and selected a priori to identify candidate predictors. Final predictors were identified through a logistic regression model with bootstrapped backward selection in which only variables selected in more than 80% of 500 bootstraps were included in the final model.
Results
Of 127 children admitted to our hospital with concern for MIS‐C, 45 were clinically diagnosed with MIS‐C and 82 were diagnosed with alternative diagnoses. We found a model with four variables—the presence of hypotension and/or fluid resuscitation, abdominal pain, new rash, and the value of serum sodium—showed excellent discrimination (concordance index 0.91; 95% confidence interval: 0.85‐0.96) and good calibration in identifying patients with MIS‐C.
Conclusion
A diagnostic prediction model with early clinical and laboratory features shows excellent discrimination and may assist clinicians in distinguishing patients with MIS‐C. This model will require external and prospective validation prior to widespread use.
INTRODUCTION
A novel pediatric inflammatory syndrome, now known as multisystem inflammatory syndrome in children (MIS‐C), emerged shortly following the onset of the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) pandemic (1, 2, 3). MIS‐C occurs several weeks after SARS‐CoV‐2 infection, and the presenting symptoms include fever and multiorgan involvement including the mucocutaneous, gastrointestinal, and/or cardiovascular systems (1). Clinical experience has shown improved patient outcomes with early identification and treatment, but the overlap between presenting features of MIS‐C and other febrile illnesses makes the diagnosis challenging. To address this problem, we aimed to build a diagnostic prediction model using clinical and laboratory data readily accessible within 24 hours of hospital presentation that can inform early diagnosis of MIS‐C in hospitalized children.
PATIENTS AND METHODS
Patient selection
We performed retrospective chart reviews of patients aged 0 to younger than 21 years with clinical suspicion for MIS‐C admitted to Vanderbilt University Medical Center (VUMC) between the release of our institutional clinical practice guideline on June 10, 2020, and April 8, 2021. Patients were identified through the pediatric infectious diseases consultation list of patients who met potential criteria for MIS‐C because all patients for whom there was any clinical concern for MIS‐C (fever, clinical involvement of >2 systems, and no alternative explanation) received a pediatric infectious diseases consultation per institutional clinical guidelines. This study was approved by the institutional review board at VUMC with a waiver of informed consent. Methods and results are reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis Guidelines (4) (Supplementary Table 1).
Data collection
Demographic, clinical, laboratory, radiographic, and cardiac characteristics collected within the first 24 hours of presentation to VUMC were extracted from each patient's electronic medical record and stored in a secure Research Electronic Data Capture (Vanderbilt University) (5). Clinical and historical data were manually extracted from the admission “history and physical,” and recorded elements included demographic information (age, sex, race, and ethnicity), biometric data, medical history, SARS‐CoV‐2 disease or exposure, fever, hypotension and/or need for fluid resuscitation (defined as vasopressor use, systolic blood pressure <10th percentile, or receipt of fluid bolus [Supplementary Table 2]), respiratory insufficiency (defined as supplemental oxygen requirement or mechanical ventilation), and organ system–specific historical and examination features. Laboratory data collection was automated and manually reviewed for completeness and accuracy. The initial reported value was recorded for laboratory studies repeated within the first 24 hours of admission. All cardiac information was assembled and reviewed by pediatric cardiology subspecialists (AS, DEC, and WM). Demographic, illness, and laboratory characteristics among patients were summarized using frequency (percentage) for categorical variables and mean (standard deviation) for continuous variables. Comparisons of patient characteristics between individuals with and without MIS‐C were conducted using either the t‐test with unequal variances or Pearson's χ2 test, as appropriate.
MIS‐C diagnosis and concordance with public health surveillance definitions
Final diagnoses were recorded from discharge documents based on the decision of the patient's primary service. An MIS‐C diagnosis was determined by the patient's clinical team, guided by published MIS‐C case definitions by the Centers for Disease Control and Prevention (CDC) and World Health Organization (WHO) along with evolving literature on the MIS‐C description (6, 7). Characteristics of the patients who were diagnosed with MIS‐C but did not meet one or both case definitions can be found in Supplementary Table 3. Alternative diagnoses were assigned by the child's medical team based on additional evaluation. All children diagnosed with Kawasaki disease (KD) met American Heart Association criteria for complete or incomplete KD requiring treatment (8). Records of patients with a diagnosis of MIS‐C underwent independent review by a pediatric rheumatologist (AP) and pediatric infectious diseases physician (NBH) with 100% agreement. All patients were reassessed by the CDC and WHO case definitions (6, 7) within the first 24 hours from presentation to determine sensitivity, specificity, and percent agreement using Cohen's κ coefficient.
Model outcome and candidate predictors
The primary outcome was MIS‐C diagnosis dichotomized as 1) MIS‐C and 2) non‐MIS‐C. Candidate predictors were selected a priori using clinical input from coauthors (MTC, SEK, JAC, AP, NBH). Twelve baseline candidate predictors were considered: hypotension and/or fluid resuscitation (yes or no), diarrhea (yes or no), headache (yes or no), abdominal pain (yes or no), rash (yes or no), sodium level (mmol/l), b‐type natriuretic peptide (BNP) level (pg/ml), neutrophil/lymphocyte ratio, platelet count (×103/μl), troponin level (ng/ml), erythrocyte sedimentation rate (mm/hour), and absolute eosinophil count (×103/μl). Sample size calculations using the three sample size criteria proposed by Riley et al (9) are provided in Supplementary Table 4.
Model development and internal validation
Given the limitations of multiple imputation in model development, we used single imputation for the main analysis. A total of 99 children (78%) had complete data for every candidate predictor. Redundancy analysis to examine collinearity among predictors was conducted using hierarchical clustering with Spearman's rank correlation (10). Nonlinearity was evaluated with a global chunk test using restricted cubic splines with three knots for continuous predictors (sodium, BNP, neutrophil/lymphocyte ratio, platelets, troponin, and absolute eosinophils).
Model building was conducted using logistic regression for a binary outcome. Bootstrap backward selection was performed to identify the most important predictors for MIS‐C. Variables selected in more than 80% of the 100 bootstrap repetitions were included in our final model (11, 12).
Model performance was evaluated with discrimination and calibration measures. Discrimination, which measures how well the model can distinguish MIS‐C from non‐MIS‐C, was calculated using the concordance index (c‐index) and shown graphically with a receiver operating characteristic curve. Model calibration, which measures the agreement between observed and predicted outcomes (13, 14) was assessed using a calibration plot, slope, and intercept (15). Overall model performance was evaluated using the Brier Score (ranges from 0 [perfect accuracy] to 1 [inaccuracy]). Internal validation was conducted using enhanced bootstrapping with 200 replications to estimate the optimism‐corrected performance measures (10). Final model predictions account for shrinkage according to the heuristic shrinkage [(X2 − df)/X2] factor to correct for overfitting and uncertainty induced by the variable selection process (10).
Sensitivity analysis
A sensitivity analysis was performed to evaluate the final model performance with single imputation versus multiple imputation and to compare models treating continuous covariates linearly versus models with restricted cubic splines with three knots.
RESULTS
MIS‐C and non‐MIS‐C cohorts
During the study period, 127 patients were admitted to our hospital with clinical suspicion for MIS‐C (Table 1). Among these, 45 (35.4%) were diagnosed with MIS‐C and 82 (64.6%) were diagnosed with alternative conditions (Supplementary Figure 1). The most common alternative diagnoses were KD and non‐SARS‐CoV‐2 viral syndrome (21% and 15%, respectively). Among patients diagnosed with MIS‐C, 87% (39 of 45) met the CDC's case definition and 96% (42 of 45) fulfilled WHO criteria within the first 24 hours of presentation. In patients who received an alternative diagnosis, 18% (15 of 82) and 9% (7 of 82) met CDC and WHO criteria, respectively. When evaluating Cohen's κ coefficient, MIS‐C diagnosis in our cohort showed 65% agreement with the CDC's case definition and 85% agreement with the WHO's case definition (Table 2).
Table 1.
Demographic, illness, and laboratory characteristics of patients with and without MIS‐C
| Characteristic | All children (N = 127) | MIS‐C (n = 45) | Non‐MIS‐C (n = 82) | P |
|---|---|---|---|---|
| Age, years, mean (SD) | 9.2 (5.6) | 9.6 (4.3) | 9.0 (6.2) | 0.538 |
| Sex, male, no. (%) | 76 (59.8) | 26 (57.8) | 50 (61.0) | 0.725 |
| Race, no. (%) | ||||
| White | 90 (70.9) | 29 (64.4) | 61 (74.4) | 0.387 |
| Black | 14 (11.0) | 7 (15.6) | 7 (8.5) | |
| Other | 23 (18.1) | 9 (20.0) | 14 (17.1) | |
| Ethnicity, no. (%) | ||||
| Hispanic or Latino | 16 (12.6) | 6 (13.3) | 10 (12.2) | 0.853 |
| Weight, percentile, mean (SD) | 60.5 (32.1) | 65.4 (29.4) | 57.9 (33.3) | 0.191 |
| Height, percentile, mean (SD) | 55.3 (31.7) a | 61.1 (30.5) | 52.1 (32.2) b | 0.123 |
| Body mass index, mean (SD) | 19.7 (6.7) a | 20.2 (7.5) | 19.5 (6.2) b | 0.587 |
| SARS‐CoV‐2 history, no. (%) | ||||
| SARS‐CoV‐2 exposure | 54 (42.5) | 31 (68.9) | 23 (28.0) | <0.001 |
| Duration since SARS‐CoV‐2 exposure and disease, mean (SD) | 5.2 (3.8) c | 5.3 (3.0) d | 5.0 (4.9) e | 0.756 |
| SARS‐CoV‐2 detection f | 51 (40.2) | 38 (84.4) | 13 (15.9) | <0.001 |
| SARS‐CoV‐2 IgG | 43 (46.7) | 35 (92.1) | 8 (14.8) | <0.001 |
| SARS‐CoV‐2 RT‐PCR or antigen testing | 9 (7.6) | 4 (10.3) | 5 (6.3) | 0.390 |
| Symptoms, no. (%) | ||||
| Fever | 122 (96.1) | 45 (100) | 77 (93.9) | 0.160 |
| Fever duration, days, mean (SD) | 6.1 (5.8) g | 5.2 (1.9) h | 6.7 (7.2) | 0.093 |
| Maximum temperature, °F, mean (SD) | 103.1 (1.4) i | 103.4 (1.1) j | 103.0 (1.6) k | 0.155 |
| Hypotension or fluid resuscitation | 50 (39.4) | 35 (77.8) | 15 (18.3) | <0.001 |
| Mucocutaneous symptoms, no. (%) | 99 (78.0) | 38 (84) | 61 (74) | 0.191 |
| Sore throat | 31 (24.4) | 10 (22.2) | 21 (25.6) | 0.671 |
| Bilateral conjunctival injection | 53 (41.7) | 24 (53.3) | 29 (35.4) | 0.050 |
| Oral mucosal changes | 49 (38.6) | 17 (37.8) | 32 (39.0) | 0.890 |
| Unilateral cervical adenopathy (>1.5 cm) | 4 (3.1) | 1 (2.2) | 3 (3.7) | 0.999 |
| Hepatomegaly or splenomegaly | 4 (3.1) | 0 (0) | 4 (4.9) | 0.296 |
| Rash (any) | 70 (55.1) | 30 (66.7) | 40 (48.8) | 0.053 |
| Cardiac symptoms, no. (%) | ||||
| Abnormal cardiac examination results | 46 (36.2) | 22 (48.9) | 24 (29.3) | 0.028 |
| Respiratory symptoms, no. (%) | 56 (44.1) | 19 (42.2) | 37 (45.1) | 0.753 |
| Cough | 41 (32.3) | 14 (31.1) | 27 (32.9) | 0.834 |
| Dyspnea | 21 (16.5) | 7 (15.6) | 14 (17.1) | 0.826 |
| Abnormal respiratory examination results | 17 (13.7) | 7 (15.6) | 10 (12.2) | 0.652 |
| Gastrointestinal symptoms, no. (%) | 101 (80.2) | 40 (88.9) | 61 (74.4) | 0.053 |
| Diarrhea | 53 (41.7) | 24 (53.3) | 29 (35.4) | 0.050 |
| Abdominal pain | 69 (54.3) | 34 (75.6) | 35 (42.7) | <0.001 |
| Nausea | 51 (40.2) | 23 (51.1) | 28 (34.1) | 0.062 |
| Vomiting | 69 (54.3) | 27 (60.0) | 42 (51.2) | 0.342 |
| Abnormal abdominal examination results | 37 (29.1) | 18 (40.0) | 19 (23.2) | 0.046 |
| Acute abdomen | 3 (2.4) | 2 (4.4) | 1 (1.2) | 0.286 |
| Blood in stool | 1 (0.8) | 0 (0) | 1 (1.2) | 0.999 |
| Neurological symptoms, no. (%) | ||||
| Headache | 67 (52.8) | 29 (64.4) | 38 (46.3) | 0.051 |
| Neck pain | 19 (15.1) | 8 (17.8) | 11 (13.4) | 0.510 |
| Lymphatic symptoms, no. (%) | ||||
| Edema of hands and/or feet | 20 (15.7) | 6 (13.3) | 14 (17.1) | 0.580 |
| Arthritis | 5 (3.9) | 1 (2.2) | 4 (4.9) | 0.655 |
| Arthralgia | 18 (14.2) | 5 (11.1) | 13 (15.9) | 0.464 |
| Myalgia | 39 (30.7) | 20 (44.4) | 19 (23.2) | 0.013 |
| Laboratory panel, mean (SD) | ||||
| White blood cell count, ×103/μl | 11.3 (6.8) | 10.4 (5.4) | 11.8 (7.5) | 0.216 |
| Hemoglobin, g/dl | 11.6 (1.7) | 120781.2) | 11.4 (1.9) | 0.103 |
| Platelet count, ×103/μl | 260.5 (146.8) | 186.7 (63.3) | 300.9 (163.3) | <0.001 |
| Neutrophils, % | 71.5 (17.9) l | 82.1 (7.0) e | 65.3 (19.3) m | <0.001 |
| Absolute neutrophils, ×103/μl | 8.3 (5.6) | 8.7 (4.6) | 8.0 (6.1) | 0.490 |
| Lymphocytes, % | 18.4 (14.8) | 10.7 (7.0) | 22.7 (16.1) | <0.001 |
| Absolute lymphocytes, ×103/μl | 1.9 (2.0) | 1.0 (0.6) | 2.4 (2.2) | <0.001 |
| Neutrophil/lymphocyte ratio | 9.1 (13.4) l | 10.9 (6.9) e | 8.2 (15.9) m | 0.194 |
| Serum sodium, mmol/l | 134.5 (3.6) | 132.8 (2.9) | 135.4 (3.7) | <0.001 |
| Blood urea nitrogen, mmol/l | 14.8 (11.5) n | 16.4 (11.1) | 13.9 (11.6) o | 0.231 |
| Creatinine, mg/dl | 0.9 (1.2) | 0.9 (0.9) | 1.0 (1.3) | 0.837 |
| Blood urea nitrogen/creatinine ratio | 18.6 (8.2) n | 19.9 (7.7) | 17.9 (8.5) o | 0.164 |
| Albumin, g/dl | 3.5 (0.5) n | 3.4 (0.5) | 3.6 (0.5) o | 0.131 |
| AST, units/l | 62.4 (106.8) n | 45.6 (23.7) | 71.8 (131.4) o | 0.084 |
| ALT, units/l | 50.2 (95.8) n | 35.7 (24.9) | 58.3 (117.5) o | 0.100 |
| Lactate dehydrogenase, units/l | 429.1 (225.7) p | 384.3 (125.0) j | 466.8 (279.9) q | 0.065 |
| C‐reactive protein, mg/l | 135.3 (104.1) n | 190.5 (98.4) | 104.7 (94.6) o | <0.001 |
| Erythrocyte sedimentation rate, mm/hr | 47.2 (29.7) g | 49.5 (23.0) e | 45.9 (33.1) k | 0.486 |
| Troponin, ng/ml | 0.6 (3.3) r | 0.8 (4.0) | 0.5 (2.7) s | 0.658 |
| B‐type natriuretic peptide, pg/ml | 265.4 (640.5) t | 452.1 (778.4) e | 149.7 (510.3) u | 0.025 |
| Ferritin, ng/ml | 729.3 (1484.0) v | 637.2 (447.5) j | 799.5 (1936.9) w | 0.550 |
| Fibrinogen, mg/dl | 505.3 (162.5) x | 568.7 (147.1) y | 447.7 (155.4) z | <0.001 |
| Neutrophils with vacuolization or toxic granulation, no. (%) | 33 (26.0) | 18 (40) | 15 (18.3) | 0.008 |
| Echocardiogram abnormal, no. (%) | 52 (49.1) aa | 24 (60.0) bb | 28 (42.4) cc | 0.079 |
| LV ejection fraction, mean (SD) | 57.2 (10.8) dd | 53.2 (9.0) ee | 60.1 (11.1) q | 0.002 |
| Coronary artery ectasia, no. (%) | ||||
| RCA | 6 (7.7) ff | 0 gg | 7 (14.0) e | 0.033 |
| LMCA | 2 (2.3) x | 0 ee | 2 (4.1) hh | 0.504 |
| Left circumflex | 2 (12) ii | 0 jj | 2 (25) kk | 0.467 |
| Left anterior descending | 6 (9.4) ll | 0 mm | 6 (15.0) bb | 0.076 |
| Coronary z score, mean (SD) | ||||
| RCA | 1.2 (3.2) hh | 0.7 (0.8) nn | 1.6 (4.1) oo | 0.247 |
| LMCA | 0.1 (1.0) q | −0.1 (0.9) pp | 0.3 (1.0) qq | 0.146 |
| Left circumflex | none obs | none obs | none obs | |
| Left anterior descending | 1.3 (3.3) bb | 0.2 (1.0) rr | 1.8 (4.0) ss | 0.068 |
Note: Data are n (%), unless otherwise specified. P values were calculated using two‐sample t‐tests allowing unequal variances for continuous variables and Pearson's χ2 test for categorical variables (α ≤ 0.05).
Abbreviations: ALT, alanine aminotransferase; AST, aspartate aminotransferase; IgG, immunoglobulin G; LMCA, left main coronary artery; LV, left ventricle; MIS‐C, multisystem inflammatory syndrome in children; obs, observed; RCA, right coronary artery; RT‐PCR, reverse transcription polymerase chain reaction.
n = 125.
n = 80.
n = 105.
n = 29.
n = 44.
All 16 patients with Kawasaki disease tested negative for SARS‐CoV‐2 polymerase chain reaction, 2 had positive SARS‐CoV‐2 nucleocapsid IgG, and 2 had COVID‐19 exposure but no serologic conversion. Patients with Kawasaki disease with SARS‐CoV‐2 exposure were differentiated from those with MIS‐C based on young age, presence of coronary ectasia and/or giant aneurysm, and the presence of both leukocytosis and thrombocytosis.
n = 118.
n = 73.
n = 116.
n = 42.
n = 74.
n = 123.
n = 79.
n = 126.
n = 81.
n = 92.
n = 50.
n = 117.
n = 72.
n = 115.
n = 71.
n = 97.
n = 55.
n = 86.
n = 41.
n = 45.
n = 106.
n = 40.
n = 66.
n = 24.
n = 37.
n = 78.
n = 34.
n = 49.
n = 15.
n = 7.
n = 8.
n = 60.
n = 24.
n = 21.
n = 28.
n = 20.
n = 30.
n = 14.
n = 26.
Table 2.
Performance comparison of institutional clinical definition to public health surveillance case definitions for MIS‐C within 24 hours of admission
| Surveillance case definitions | ||||||
|---|---|---|---|---|---|---|
| CDC a | WHO b | CDC or WHO c | ||||
| MIS‐C | Non‐MIS‐C | MIS‐C | Non‐MIS‐C | MIS‐C | Non‐MIS‐C | |
| Institutional definition | ||||||
| MIS‐C | 39 | 6 | 43 | 2 | 44 | 1 |
| Non‐MIS‐C | 15 | 67 | 7 | 75 | 17 | 65 |
| Concordance agreement | ||||||
| Percent d | 0.83 e | 0.93 f | 0.86 g | |||
| Chance h | 0.52 e | 0.53 f | 0.51 g | |||
| κ i | 0.65 e | 0.85 f | 0.71 g | |||
Abbreviations: CDC, Centers for Disease Control and Prevention; CRP, c‐reactive protein; ECHO, echocardiogram; ESR, erythrocyte sedimentation rate; MIS‐C, multisystem inflammatory syndrome in children; NT‐proBNP, N terminal pro hormone B‐type natriuretic peptide; PT, prothrombin time; PTT, partial thromboplastin time; RT‐PCR, reverse transcription polymerase chain reaction; WHO, World Health Organization.
Individual aged <21 years presenting with 1) fever (≥38°C [measured and/or self‐reported]) for ≥24 hours; 2) laboratory evidence of inflammation, elevated C‐reactive protein level, elevated erythrocyte sedimentation rate, elevated fibrinogen level, elevated procalcitonin level, elevated D‐dimer level, elevated ferritin level, elevated lactic acid dehydrogenase level, elevated interleukin 6 level, neutrophil count, low lymphocyte count, or low albumin level; 3) multisystem organ system involvement (≥2); and 4) evidence of COVID‐19 by positive RT‐PCR, antigen, or serologic testing or contact with a person with confirmed COVID‐19 ≤4 weeks prior to the onset of symptoms (6).
Individual aged 0‐19 years presenting with 1) fever (≥38°C [measured and/or self‐reported]) for ≥3 days; 2) two of the following: (a) rash or bilateral nonpurulent conjunctivitis or mucocutaneous inflammation signs; (b) hypotension or shock; (c) features of myocardial dysfunction pericarditis, valvulitis, or coronary abnormalities (including ECHO findings or elevated troponin or NT‐proBNP levels); (d) evidence of coagulopathy by PT, PTT, and elevated D‐dimers levels; or (e) acute gastrointestinal problems; 3) elevated markers of inflammation (ESR, CRP, or procalcitonin); 4) evidence of COVID‐19 by positive RT‐PCR, antigen, or serologic testing or likely contact with a person with confirmed COVID‐19; and 5) no other obvious microbial cause of inflammation (including bacterial sepsis, staphylococcal, or streptococcal shock syndromes) (7).
Individual met either CDC and/or WHO surveillance case definition.
Percent agreement: p0, the relative observed agreement among definitions; (a + d)/total.
CDC criteria.
WHO criteria.
CDC or WHO criteria.
Chance agreement: pe, the hypothetical probability of chance agreement; [((a + b)/total) × ((a + c)/total)] + [((c + d)/total) × (b + d)/total)].
κ: measure of reliability designed to correct for chance agreement; (p0 − pe)/(1 − pe).
Demographic characteristics
There were no baseline demographic differences between patients with and without MIS‐C in our cohort (Table 1). Both groups had few comorbid conditions, with asthma or reactive airway disease being the most reported (2 [4%] in MIS‐C; 10 [12%] in non‐MIS‐C).
Clinical characteristics
All patients with MIS‐C had fever. There were no differences in fever peak or duration between groups. All patents with MIS‐C had a known history of SARS‐CoV‐2 exposure or disease, but this was not a unique feature (100% MIS‐C vs. 40% non‐MIS‐C; P < 0.001). Patients with MIS‐C were more likely to present with hypotension and/or require fluid resuscitation (Table 1). No significant differences between groups in the evaluation of mucocutaneous features (skin, oral mucosa, conjunctiva) or respiratory, neurologic, or lymphatic systems were noted. Patients with MIS‐C were more likely to have abnormal cardiac examination results (49% MIS‐C vs. 29% non‐MIS‐C; P = 0.028), which was most often characterized by tachycardia. They were also more likely to have abdominal pain (reported by patient and/or abdominal tenderness on examination; 76% MIS‐C vs. 43% non‐MIS‐C; P < 0.001) and abnormal abdominal examination results (40% MIS‐C vs. 23% non‐MIS‐C; P = 0.046). Myalgia was more frequently seen in patients with MIS‐C (44% vs. 23%; P = 0.013).
Laboratory and cardiac characteristics
Patients with MIS‐C had a significantly lower mean serum sodium level (132.8 mmol/l vs. 135.4mmol/l; P < 0.001), platelet count (186.7 × 103/μl vs. 300.9 × 103/μl; P < 0.001), lymphocyte percentage (10.7% vs. 22.7%; P < 0.001), and absolute lymphocyte count (1.0 × 103/μl vs. 2.4 × 103/μl; P < 0.001) compared to the non‐MIS‐C group (Table 2). Those with MIS‐C had a higher mean neutrophil percentage (82.1% vs. 65.3%; P < 0.001), C‐reactive protein (CRP) level (190.5 mg/l vs. 104.7 mg/l; P < 0.001), BNP level (452.1 pg/ml vs. 149.7 pg/ml; P = 0.025), and fibrinogen level (568.7 mg/dl vs. 447.7 mg/dl; P < 0.001) compared to the non‐MIS‐C group. Neutrophils with vacuolization or toxic granulation were seen more often in the patients with MIS‐C than in patients with non‐MIS‐C (40% vs. 18.3%; P < 0.007). Patients with MIS‐C had a lower mean left ventricle ejection fraction than patients with non‐MIS‐C (53.2% vs. 60.1%; P = 0.002) (Table 1). None of the patients with MIS‐C were found to have coronary artery abnormalities, whereas 8 of 16 (50%) patients with KD had ectasia of at least one coronary artery. All patients with MIS‐C were treated with intravenous (IV) immunoglobulin, 88.9% (40 of 45) were given IV methylprednisolone, and 33% (15 of 45) received anakinra. All patients with MIS‐C survived, but 2 of 82 (2.4%) with non‐MIS‐C diagnoses died because of infectious complications of malignancy.
Model performance
After bootstrap backward selection, the most important predictors for MIS‐C were hypotension and/or fluid resuscitation (yes or no), abdominal pain (yes or no), new rash associated with current illness (yes or no), and serum sodium level (mmol/l). Bootstrap inclusion frequencies, a measure of variable importance, are presented in Supplementary Table 5. The model demonstrated great discrimination with a c‐index of 0.91 (95% confidence interval [CI]: 0.85‐0.96; Figure 1A) and good model calibration with a near‐diagonal calibration curve at predicted risks between 0.0 and 0.8 (Figure 1B); departure between 0.2 and 0.4 is likely due to the sparseness of patients in our study population. The Brier Score was 0.12 (good accuracy). After internal validation, the optimism‐corrected c‐index was 0.89 (95% CI: 0.84‐0.95) and optimism‐corrected slope and intercept were 0.89 and –0.04, indicating minimal overfitting.
Figure 1.
Multisystem inflammatory syndrome in children (MIS‐C) prediction model characteristics depicting model discrimination with a receiver operating characteristic curve (A) and calibration plot (B) of observed and predicted outcome probabilities. CI, confidence interval.
After applying a heuristic shrinkage factor of 0.94, predicted risks from the final model can be applied to an external population by using a nomogram to determine the risk a child has MIS‐C (Figure 2). Based on the clinical symptoms and laboratory values at hospital admission, a sum of the total points for each important predictor is added to determine the risk a child has MIS‐C. For example, if a child has hypotension and/or fluid resuscitation (points = 81), a rash (points = 44), no abdominal pain (points = 0), and a sodium level of 130 mmol/l (points = 70), then the risk that a child has MIS‐C is approximately 0.6 (total points = 81 + 44 + 0 + 70 = 171).
Figure 2.
Nomogram to estimate a patient's risk of multisystem inflammatory syndrome in children (MIS‐C). aDefined as a patient requiring vasopressor support or systolic blood pressure <10th percentile for age and/or demonstrating clinical need to receive fluid bolus.
In addition to the nomogram, predicted risks can also be calculated with the following formula: probability (MIS‐C = 1) = 1/[1 + exp (−Xβ × 0.94)], where Xβ = 20.305 + 2.943 × [hypotension and/or fluid resuscitation] + 1.697 × [rash] + 1.376 × [abdominal pain] + (−0.180) × [serum sodium]. Figure 3 provides a distribution for the predicted probability from our model for each MIS‐C and non‐MIS‐C diagnosis.
Figure 3.
Predicted probability of multisystem inflammatory syndrome in children (MIS‐C) in patients hospitalized with inflammatory disorders at Vanderbilt University Medical Center (VUMC) June 10, 2020, to April 8, 2021, Nashville, TN. “Other” includes appendicitis (n = 1), abscess (n = 1), cholangitis (n = 1), community‐acquired pneumonia (n = 1), group A Streptococcus pharyngitis (n = 1), lymphoma (n = 1), periodic fever syndrome (n = 1), renal failure (n = 1), Stevens–Johnson syndrome (n = 1), urticaria (n = 1), vaping‐related lung injury (n = 1), toxin‐mediated disease or tick‐borne illness or non‐SARS‐CoV‐2 viral syndrome (n = 1), tick‐borne illness or Kawasaki disease (n = 1), tick‐borne illness or viral infection (n = 2), systemic lupus erythematosus (n = 2), mycoplasma infection (n = 2), systemic onset juvenile idiopathic arthritis (n = 2), gastroenteritis (n = 2), lymphadenitis (n = 2), bacterial enteritis (n = 2), allergic reaction (n = 2), acute interstitial nephritis (n = 2), non‐SARS‐CoV‐2 myocarditis (n = 3), bacteremia (n = 3), acute COVID‐19 (n = 4), urinary tract infection (n = 4), bacterial toxin–mediated disease (n = 4), and tick‐borne illness (n = 5).
Sensitivity analysis
The final model with multiple imputation had similar discrimination to our singularly imputed final model with a c‐index of 0.91 (95% CI: 0.84‐0.95). Similarly, a final model including a cubic spline–restricted serum sodium (three knots) had a slightly better discrimination than our final model treating serum sodium linearly (c‐index: 0.93; 95% CI: 0.89‐0.98). However, to avoid overfitting due to the small sample size, we report the final model treating serum sodium as a linear predictor.
DISCUSSION
In our study comparing individuals with MIS‐C to those with non‐MIS‐C, our predictive modeling tool was able to identify individuals with MIS‐C among an otherwise undifferentiated group of febrile patients within the first 24 hours of hospital presentation. In our diagnostic prediction model, we discriminated MIS‐C from all other inflammatory syndromes (eg, infection, malignancy, and new‐onset rheumatic disease), systematically evaluated all patients per our institutional guidelines allowing for near complete collection of an extensive set of laboratory and cardiac data, and selected candidate predictors based on accessibility and availability to clinicians within the first 24 hours. In addition, we used logistic regression modeling with stability investigation and applied heuristic shrinkage rather than machine learning approaches. Machine learning approaches tend to be challenging to externally validate, require extremely large data sets, and are often challenging to interpret (10). Our final diagnostic prediction model included four variables: abdominal pain, rash, hypotension and/or fluid resuscitation, and the serum sodium value. These variables combine to provide excellent discrimination with a c‐index of 0.91 and good calibration.
Additional literature dedicated toward describing and distinguishing MIS‐C is expanding rapidly in both volume and complexity (1, 16, 17, 18, 19, 20). Machine learning methods have been used by Geva et al (18) to group patients hospitalized with COVID‐19‐related illness into phenotypic clusters with varying likelihood of MIS‐C and by Lam et al (20) to assign risk scores for MIS‐C and KD. Kline et al was the first group to describe sensitivity and specificity to laboratory testing when they found that a CRP level greater than 4.5 mg/dl and an absolute lymphocyte count less than 1.5 × 103/μl had 86% sensitivity and 91% specificity when screening for MIS‐C in a population of patients presenting to the emergency department (19). A recent study from Godfred‐Cato et al provides a quantitative scoring system to help distinguish MIS‐C from COVID‐19, KD, and toxic shock syndrome (21).We found that our patients with MIS‐C had significantly more abdominal pain than those without MIS‐C. This finding matches with that of Roberts et al, who found the same in a single‐center retrospective analysis of patients evaluated for MIS‐C (22). Although nonspecific exanthem is associated with many childhood illnesses, rash has been described as a feature of MIS‐C and was reported by Whittaker et al in 52% of patients with MIS‐C (23). Our investigation and a study by Carlin et al (16) also found that rash pointed toward a diagnosis of MIS‐C, whereas Godfred‐Cato et al recently showed that rash favored a diagnosis of KD (21).
The third variable in our model was the need for fluid resuscitation and/or hypotension. This reflects the myocardial dysfunction seen frequently in MIS‐C (24) and, in our analysis, outperformed other cardiac findings, such as BNP, ejection fraction, and troponin. Although the administration of bolus IV fluids is common in hospitalized children, we identified it to be much more common in patients with MIS‐C. Secondary analysis showed that 14 of 17 (82.4%) patients in our cohort fulfilled this variable because of the administration of fluid bolus only. This finding is consistent with findings from the Godfred‐Cato et al (21) study, in which decreased cardiac function was more suggestive of MIS‐C, and was also supported by Roberts et al (22).
We found there was a significant difference in mean serum sodium levels between patients with and without MIS‐C. This may reflect the presence of elevated interleukin 6 levels seen in MIS‐C (25, 26) and the known effects of this cytokine on osmoregulation through the nonosmotic release of vasopressin (27). Hyponatremia is well known in MIS‐C but has not been included in other studies as a discriminating feature. Furthermore, serologic evidence of SARS‐CoV‐2 infection (nucleocapsid immunoglobulin G) was not included as a predictor because of inconsistent availability at early time points and a projected decline in specificity due to the rising prevalence of SARS‐CoV‐2 infection in the population.
The CDC and WHO criteria were designed for high sensitivity, but as our data attests, have poor specificity for patients with alternative inflammatory disorders. Our model can be used in conjunction with the CDC and WHO case definitions to improve specificity and functions best when applied to sick, febrile, undifferentiated patients for whom there is concern for, but not definite evidence of, MIS‐C. It succeeds well at differentiating MIS‐C and KD, with 14 of 16 (88%) patients with KD demonstrating less than 50% probability of having MIS‐C. Six patients without MIS‐C were identified by our model to have a greater than 70% probability of having the condition. Among these patients, there were two with bacteremia and one of each with acute COVID‐19, lymphoma, tick‐borne illness, and bacterial toxin–mediated disease.
The final diagnosis of MIS‐C was determined using CDC and WHO case definitions in addition to new literature on MIS‐C characterization. Eight patients were diagnosed as having MIS‐C by their clinical team without meeting one or both available case definitions (Supplementary Table 3) within 24 hours of presentation, and seven of these patients failed to meet case definitions at discharge. Further review shows that two patients (patient 1 and patient 2, Supplementary Table 3) did not meet fever duration criteria. Among these, patient 1 had no report of fever before presentation but had fever in the emergency department on presentation. patients 1 and 2 both had severe presentation consistent with MIS‐C and were placed on immune modulation shortly after admission, which may have shortened the fever course and prevented them from meeting the case definitions. Five patients did not meet the case definitions because of longer duration from COVID‐19 disease or exposure (range 35‐49 days) but were diagnosed with MIS‐C as new literature emerged supporting longer latency between exposure and MIS‐C in some patients (28). The remaining patient, who did not satisfy CDC case definition because of single organ system involvement at presentation, was found to have the rapidly progressive multisystem involvement that is typical of MIS‐C over the subsequent 24 hours and met both case definitions by discharge. Because there is clinical ambiguity in differentiation of MIS‐C from non‐MIS‐C, it is possible that the final VUMC diagnosis was incorrect for some study patients. We elected against the use of a comparison cohort of inflammatory disorders prior to the COVID‐19 pandemic because this analysis would be limited by nonuniform evaluation of patients prior to our standardized algorithm for MIS‐C.
Limitations of our model include the retrospective nature of data collection, which led to some incomplete medical records. Although we chose the first 24 hours of presentation for the dual purposes of standardization and building a model with clinical use early in a patient's hospital presentation, we could not standardize the point in any given patient's illness at which they chose to seek care. This may have prevented us from identifying features that would have been present if the patient sought care at a different time.
Our study also represents a single site in the mid‐South, and the frequency and scope of inflammatory conditions, such as certain infectious agents (including tick‐borne and fungal infections), will not be identical to that in other parts of the country. In this study, because of the lack of an external population, we were unable to externally validate the prediction model; however, we internally validated our prediction model with bootstrap methods (10).
We used early clinical and laboratory features to inform the design of a clinical diagnostic prediction model with excellent discrimination and good calibration. The diagnostic model aims to assist clinicians in distinguishing patients with MIS‐C from those with alternate diagnoses. The effects of changing prevalence of MIS‐C in the community and presence of both emerging variants and widespread pediatric vaccination on the performance of this model remain unclear. This model will require external and prospective validation prior to application in hospital and clinical settings.
AUTHOR CONTRIBUTIONS
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. M. T. Clark had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design
M. T. Clark, Rankin, D. E. Clark, Halasa, Connelly, Katz.
Acquisition of data
M. T. Clark, Rankin, Peetluk, Gotte, Herndon, McEachern, Smith, D. E. Clark, Hardison, Patrick, Halasa, Connelly, Katz.
Analysis and interpretation of data
M. T. Clark, Rankin, Peetluk, Esbenshade, Patrick, Halasa, Connelly, Katz.
Supporting information
Disclosureform
Appendix S1 Supplementary Information
ACKNOWLEDGMENTS
None.
This work was supported by the National Center for Advancing Translational Sciences (grant UL1‐TR‐000445). Dr. Rankin's work was supported by the NIH (award TL1‐TR‐002244). Dr. Patrick's work was supported by the NIH National Institute of Child Health and Development K12‐HD‐087023 (research scholar). The other authors received no additional funding.
Drs. Clark and Rankin contributed equally to this work. Drs. Connelly and Katz contributed equally to this work.
Author disclosures are available at https://onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1002%2Facr2.11509&file=acr211509-sup-0001-Disclosureform.pdf.
REFERENCES
- 1. Feldstein LR, Rose EB, Horwitz SM, et al. Multisystem inflammatory syndrome in U.S. children and adolescents. N Engl J Med 2020;383:334–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Toubiana J, Poirault C, Corsia A, et al. Kawasaki‐like multisystem inflammatory syndrome in children during the covid‐19 pandemic in Paris, France: prospective observational study. BMJ 2020;369:m2094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Verdoni L, Mazza A, Gervasoni A, et al. An outbreak of severe Kawasaki‐like disease at the Italian epicentre of the SARS‐CoV‐2 epidemic: an observational cohort study. Lancet 2020;395:1771–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015;350:g7594. [DOI] [PubMed] [Google Scholar]
- 5. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)–a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Centers for Disease Control and Prevention . Information for healthcare providers about multisystem inflammatory syndrome in children (MIS‐C). 2021. URL: https://www.cdc.gov/mis/mis-c/hcp/index.html.
- 7. World Health Organization . Multisystem inflammatory syndrome in children and adolescents temporally related to COVID‐19. 2020. URL: https://www.who.int/news‐room/commentaries/detail/multisystem‐inflammatory‐syndrome‐in‐children‐and‐adolescents‐with‐covid‐19.
- 8. McCrindle BW, Rowley AH, Newburger JW, et al. Diagnosis, treatment, and long‐term management of Kawasaki disease: a scientific statement for health professionals from the American Heart Association [published erratum appears in Circulation 2019;140:e181–4]. Circulation 2017;135:e927–99. [DOI] [PubMed] [Google Scholar]
- 9. Riley RD, Ensor J, Snell KI, et al. Calculating the sample size required for developing a clinical prediction model. BMJ 2020;368:m441. [DOI] [PubMed] [Google Scholar]
- 10. Harrell FE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. 2nd ed. New York: Springer; 2015. [Google Scholar]
- 11. Heinze G, Wallisch C, Dunkler D. Variable selection: a review and recommendations for the practicing statistician. Biom J 2018;60:431–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Heymans MW, van Buuren S, Knol DL, et al. Variable selection under multiple imputation using the bootstrap in a prognostic study. BMC Med Res Methodol 2007;7:33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Van Calster B, Nieboer D, Vergouwe Y, et al. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol 2016;74:167–76. [DOI] [PubMed] [Google Scholar]
- 15. Van Calster B, McLernon DJ, van Smeden M, et al. Calibration: the Achilles heel of predictive analytics. BMC Med 2019;17:230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Carlin RF, Fischer AM, Pitkowsky Z, et al. Discriminating multisystem inflammatory syndrome in children requiring treatment from common febrile conditions in outpatient settings. J Pediatr 2021;229:26–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Corwin DJ, Sartori LF, Chiotos K, et al. Distinguishing multisystem inflammatory syndrome in children from Kawasaki disease and benign inflammatory illnesses in the SARS‐CoV‐2 pandemic. Pediatr Emerg Care 2020;36:554–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Geva A, Patel MM, Newhams MM, et al. Data‐driven clustering identifies features distinguishing multisystem inflammatory syndrome from acute COVID‐19 in children and adolescents. EClinicalMedicine 2021;40:101112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Kline JN, Isbey SC, McCollum NL, et al. Identifying pediatric patients with multisystem inflammatory syndrome in children presenting to a pediatric emergency department. Am J Emerg Med 2022;51:69–75. [DOI] [PubMed] [Google Scholar]
- 20. Lam JY, Roberts SC, Shimizu C, et al. Multicenter validation of a machine learning algorithm for diagnosing pediatric patients with multisystem inflammatory syndrome and Kawasaki disease [preprint]. medRxiv 2022. [PMC free article] [PubMed] [Google Scholar]
- 21. Godfred‐Cato S, Abrams JY, Balachandran N, et al. Distinguishing multisystem inflammatory syndrome in children from COVID‐19, Kawasaki disease and toxic shock syndrome. Pediatr Infect Dis J 2022;41:315–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Roberts JE, Campbell JI, Gauvreau K, et al. Differentiating multisystem inflammatory syndrome in children: a single‐centre retrospective cohort study. Arch Dis Child 2022;107:e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Whittaker E, Bamford A, Kenny J, et al. Clinical characteristics of 58 children with a pediatric inflammatory multisystem syndrome temporally associated with SARS‐CoV‐2. JAMA 2020;324:259–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Sperotto F, Friedman KG, Son MB, et al. Cardiac manifestations in SARS‐CoV‐2‐associated multisystem inflammatory syndrome in children: a comprehensive review and proposed clinical approach. Eur J Pediatr 2021;180:307–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Consiglio CR, Cotugno N, Sardh F, et al. The immunology of multisystem inflammatory syndrome in children with COVID‐19. Cell 2020;183:968–981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Gruber CN, Patel RS, Trachtman R, et al. Mapping systemic inflammation and antibody responses in multisystem inflammatory syndrome in children (MIS‐C). Cell 2020;183:982–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Papanicolaou DA, Wilder RL, Manolagas SC, et al. The pathophysiologic roles of interleukin‐6 in human disease. Ann Intern Med 1998;128:127–37. [DOI] [PubMed] [Google Scholar]
- 28. Cirks BT, Rowe SJ, Jiang SY, et al. Sixteen weeks later: expanding the risk period for multisystem inflammatory syndrome in children. J Pediatric Infect Dis Soc 2021;10:686–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Disclosureform
Appendix S1 Supplementary Information