Current Prognostic Models Fail to Reliably Distinguish Kawasaki Disease From Pediatric Multisystem Inflammatory Syndrome Temporally Associated With COVID-19

Abstract

Background:

Along with the emergence of coronavirus disease 2019 (COVID-19), a hyperinflammatory multisystemic syndrome in children called PIMS appeared. Despite its close similarity to the Kawasaki disease (KD), the conclusion of PIMS being a distinct disease entity was reached quickly. Since then, research projects have been implemented to find differences and reliable distinction parameters with conflicting results. Therefore, the aim of this study was to explore existing prognostic models on their ability to distinguish the diagnoses.

Methods:

Two externally developed prognostic models were applied to 2 distinct datasets: one consisting solely of 26 KD and PIMS cases from Bayreuth, the other comprising 632 cases of both diagnoses reported by pediatricians from all over Germany and Innsbruck, Austria. The potential of the models to distinguish the diagnoses on a phenotypic level was evaluated based on their sensitivities and specificities, κ-coefficients by Cohen and correlation analysis.

Results:

The sensitivity and specificity of both prognostic models did not display a reliable distinguishability between KD and PIMS. Moreover, correlation analysis cannot confirm the correlation of most parameters utilized in the models with the PIMS diagnosis. The only parameter showing a positive correlation in both datasets was the patients’ older age.

Conclusions:

These findings are congruent with the general inconclusiveness in research concerning the distinction of KD and PIMS. We could not reproduce a distinguishability using the models or a correlation between the utilized parameters and PIMS except for one. Henceforth, research will need to focus on nonphenotypical properties for differentiation of KD and PIMS.

The emergence of a new virus in December 2019, the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), caused a global wave of severe pneumonia cases [coronavirus disease 2019 (COVID-19)]. Unexpectedly, life was dominated by social distancing and further constrictions.¹ Along with the increasing number of COVID-19 specialists observed what initially seemed to be a concurrent increase in Kawasaki disease (KD) cases. The first description of increased incidence during the pandemic came from London in May 2020 where Riphagen et al² observed 8 children, 4–14 years old, suffering from hyperinflammatory shock. Soon, a link to a SARS-CoV-2-infection a few weeks before the systemic inflammation was described. Thus, physicians and researchers concluded that the observed immune dysregulation must be an inherent disease entity. It was first named pediatric multisystem inflammatory syndrome temporally associated with COVID19 (PIMSTS) and later named multisystem inflammatory syndrome in children (MIS-C).

Both KD and PIMS are exuberant immune reactions that cause a variety of symptoms, such as mucocutaneous lesions and cardiac involvement. This variety and overlapping of symptoms make differentiation difficult.³ Additionally, KD patients often do not meet the criteria for KD diagnosis; instead, the diagnosis reads incomplete KD. Especially infants often present without classic symptoms.⁴

In the past 5 years, the definition of PIMS has changed several times. The first case definition, which was formulated by the US Centers for Disease Control and Prevention, had to ensure fast identification and medical interception of at-risk patients.⁵ At present, there are 3 not entirely conclusive definitions by the US Centers for Disease Control and Prevention, the World Health Organization, and the UK’s Royal College of Paediatrics and Child Health. Today, this heterogeneity of definitions renders it difficult to compare or produce retrospective research results. More importantly, diagnosing PIMS remains confusing, as well.^6,7

The main criterion differentiating between KD and PIMS was a foregoing SARS-CoV-infection before PIMS. Detecting this infection retrospectively is mostly conducted by using SARS-CoV-2 testing and the history of exposure. This led to misdiagnosing antibody-/exposure-positive patients who met the criteria for incomplete or complete KD as PIMS.⁸ Currently, most people have been infected at least once and/or have been vaccinated. Subsequently, the specificity of the antibodies decreased significantly and ceased to be a diagnostic criterion.⁹

Therefore, researchers have deemed it important to find other criteria for distinction. Patients diagnosed with PIMS are on average older than those with KD. However, the median age of PIMS patients has never been constant, but has shifted along with the different viral variants.¹⁰ PIMS is more likely to be found in people of African–American or Hispanic background, whereas KD is known for its increased prevalence in Asian people. KD patients present more often with mucocutaneous changes, cervical lymphadenopathy and coronary artery aneurysms. Instead, PIMS patients develop gastrointestinal symptoms and reduced left ventricular ejection fraction (LVEF), mitral valve insufficiency and pericardial effusion more dominantly.^11,12 The acute disease progression is more serious with PIMS as more patients go into hypovolemic shock or progress to have respiratory dysfunction, require a longer treatment and are more often in an intensive care unit.¹² In the long-term, the prognosis for KD patients is poorer, as coronary aneurysms can persist for a long time and are the most common cause of acquired heart disease in children in industrialized countries.^13,14 Hypercoagulative states are possible in both diseases.¹⁵ However, the use of anticoagulants, interleukin inhibitors, inotropic substances and steroids was reported more frequently in the PIMS group.¹²

The reported laboratory differences have not always been consistent.^{3,8,9,14–16} Both diagnoses were accompanied by increased markers of inflammation. KD is associated with leukocytosis, thrombocytosis and elevated liver enzymes. In contrast, PIMS is more commonly associated with thrombocytopenia, neutrophilia, hyponatremia, hypalbuminemia and elevated cardiac enzymes.^3,17

Based on these differences, prediction models have been developed. These models include different sets of criteria and are the result of different approaches, as one was produced by machine-learning, while another concentrated on using objectifiable criteria only. So far, most of them have been solely validated internally and have not been applied in clinical routine.^9,18–20

Our study aimed to explore the potential of 2 existing prediction models to phenotypic distinguishability of both syndromes.

METHODS

The research described in this manuscript was conducted in accordance with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Friedrich-Alexander-University, Erlangen-Nürnberg (January 22, 2025, 24-465-BR).

Two distinct datasets were utilized in this study. The first dataset was locally collected from the ORBIS data pool of the Children’s Hospital in Bayreuth. Comprising 26 cases in total, it includes all KD and PIMS cases that have been recorded in the clinic so far with the first case dating back to 2004. The second dataset is owed to the generosity of the working group Hufnagel et al¹⁴ from the German Society of Paediatric Infectiology (Deutsche Gesellschaft für Pädiatrische Infektiologie, DGPI). They initially collected these data from pediatric departments all over Germany and Innsbruck, Austria, for their PIMS-Survey from January 2020 to April 2023. The data provided by them for the purpose of this study stems from March 2020 to April 2022, comprising a total of 961 cases. Recruiting these data was organized via voluntary reports by physicians after the patient’s discharge.²¹ Bayreuth did not report their cases to the DGPI working group. The data extraction and inclusion were performed exclusively for the purpose of this study. Therefore, it was decided to analyze the data separately from the DGPI dataset.

Patients were selected based on their diagnosis. The definition of PIMS used in the PIMS-Survey was from the World Health Organization.^14,21,22 The definition for KD was taken from the American Heart Association.^14,23 For improved comparability, we used the same definition. Originally collected for a different purpose, the documented and reported data were at the discretion of the individual physician and case. Thus, not every case contains the information necessary for the application of the models, and we had to exclude the 239 cases with incomplete data, leaving 632 cases at hand. As the cases with incomplete data are randomly distributed over age and gender groups, causing a major bias by their exclusion was ruled out.

The prediction models were chosen based on their performance, namely the area under the curve (AUC), at internal validation, ready availability and application process. The selectable models were from the working groups Starnes et al,²⁰ Kostik et al,¹⁸ Lam et al¹⁹ and Godfred-Cato et al.⁹ Lam et al¹⁹ used machine-learning to produce their model called KIDMATCH and is, to our knowledge, the only model externally validated. However, as the model is not available to the public, it cannot be applied here.¹⁹ Besides KIDMATCH, all models used an easily applicable equation to calculate a risk score, which comes with a corresponding cutoff value leading to a diagnosis.^9,18,20 The model KMDScore by Kostik et al¹⁸ achieved an AUC-value of 0.927. The models by Starnes et al²⁰ and Godfred-Cato et al⁹ both scored an AUC of 0.96. Consequently, by fulfilling all the criteria the last 2 models were elected.

The selection of data was determined by the parameters utilized in the models: The parameters considered by Starnes et al²⁰ in their model were age, serum sodium, C-reactive protein (CRP), alanine-aminotransferase (ALT), blood platelets and LVEF. Their focus in developing this model was to use objectifiable parameters. According to Starnes et al,²⁰ the values are collected and then put into the following equation, resulting in a “PIMS-risk” in percent: 1/[1 + EXP(30.802 + 0.309 × age − 0.242 × sodium − 0.009 × platelet_count + 1.523 × contractibility − 0.005 × ALT + 0.004 × CRP) × 0.982)].²⁰ In diagnostics of the patients here, echocardiography had a more precautional than analytic goal and LVEF was often not assessed. However, with this parameter being a binary value we were able to substantiate it with any description of cardiac contractibility, such as left ventricular end-systolic diameter or “preserved contractibility.” Reduced heart contractibility is rated with 1 point. If it is preserved, zero points are assigned. Also, the serum sodium levels were not incorporated in the PIMS-Survey, and restricted the analysis of its dataset to the model by Godfred-Cato et al.⁹

Godfred-Cato et al’s model entails only binary values: platelet count <150 × 10³/L (2 points), abdominal pain, headache, pericardial effusion, CRP >10 mg/L (one point each), rash (minus 1 point) and mucocutaneous lesions (minus 2 points). The points are added and translated into a “PIMS-risk” in percent, as well.⁹

Statistical analysis was performed using SPSS for Windows Version 11 version 24H2. Unlike the validations previously conducted by the model producing working groups, AUC values were not used for evaluation here. Instead, we calculated the sensitivity and specificity of both models for their assessment. With 2 diagnoses on hand, we chose to accredit the KD diagnosis to the value of sensitivity and the diagnosis of PIMS to the value of specificity. A comparison between the performance of both tests was solely possible for the dataset from Bayreuth using Cohen‘s κ-coefficient. This coefficient helps ascertain the congruence between the results and the gold standard, namely the individual physician‘s established diagnosis.²⁴ To conduct further analysis, correlations between the diagnosis and the parameters of the models were calculated using Pearson’s coefficient for linear values and Kendall’s τ for binary values.²⁵ The confidence interval (CI) was 95%, respectively. Finally, as means of correction for multiple testing the P values of the correlation analysis were adjusted performing the Benjamini-Hochberg procedure.

RESULTS

Of 26 cases from Bayreuth (cohort 1), 21 were diagnosed previously by the attending physicians with KD, and the remaining 5 cases were diagnosed with PIMS. The age of the KD patients was 2.29 ± 2.59 years (95% CI) and of the PIMS patients 7.00 ± 5.61 years (95% CI), respectively. The model of Starnes et al²⁰ assigned the KD diagnosis to 22 cases and the PIMS diagnosis to 3 cases, leaving 1 case undiagnosed. In total, 21 patients were diagnosed with KD by their physicians in accordance with the model. Thus, this model achieved a sensitivity of 90.5% and specificity of 60.0%.

On the other hand, the model by Godfred-Cato et al⁹ diagnosed 17 patients with KD and 3 with PIMS. The age of PIMS patients was 8.52 ± 4.14 years (95% CI) and of KD patients 6.80 ± 4.27 years (95% CI). With 6 unassigned cases, only 15 of the 26 cases were identified in agreement with the physicians’ diagnosis, while 5 cases were assigned another diagnosis. Accordingly, this model achieved a sensitivity of 66.7% and a specificity of 80.0%. For comparison, the κ-coefficient resulted in rather small values of 0.347 for the model by Starnes et al²⁰ and of 0.059 for the model by Godfred-Cato et al.⁹

Among the data from the DGPI (cohort 2) of 632 cases, 493 were diagnosed with KD and 139 with PIMS. The model by Godfred-Cato et al⁹ assigned 331 cases in accordance with the individual physicians‘ diagnoses, 120 cases were left unassignable and 181 cases were subordinated to a different diagnosis. More precisely, 233 KD cases and 279 PIMS cases were identified. Subsequently, the resulting sensitivity is 44.6% and the specificity is 20.1%.

Additionally, the correlation between the diagnosis of PIMS and the parameters used in the models was tested. Parameters with a positive correlation to PIMS proven statistically significant were age and pericardial effusion within the dataset from Bayreuth. Age was the only variable with a significant correlation confirmed in the DGPI dataset. However, other variables showed significant correlation with PIMS here: Rash and mucocutaneous lesions correlated positively with the diagnosis. The correlation coefficients’ values rank mostly in the lower to middle field of the correlation scale (Table 1) with the highest being 0.740 (pericardial effusion, P value < 0.001). Aside from age as the only parameter showing significance in both datasets, the conclusion of this analysis is that for none of the parameters a strong correlation with PIMS can be affirmed.

Table 1.

Summary of Statistical Results of the 2 Datasets (Bayreuth and DPGI)

Characteristic	Bayreuth		DGPI
	KD	PIMS	KD	PIMS
Number of cases	21 (80.8%)	5 (19.2%)	493 (78.0%)	139 (22.0%)
Male gender	16 (76.2%)	3 (60.0%)	323 (65.5%)	89 (64.0%)
Median age	2 years	7 years	7 years	9 years
- Correlation (Pearson)	0.506 (P value: 0.008)*		0.166 (P value: < 0.001)*
Median serum sodium	131.9 mmol/l	132.6 mmol/l	n/a	n/a
- Correlation (Pearson)	0.084 (P value: 0.682)		n/a
Median ALT	78.8 U/l	26.2 U/l	56.9 U/l	42.2 U/l
- Correlation (Pearson)	−0.251 (P value: 0.217)		−0.58 (P value: 0.152)
Median platelet count	350.0 x10^9/l	325.6 x10^9/l	308.9 x10^9/l	254.6 x10^9/l
- Correlation (Pearson)	−0.065 (P value: 0.757)		−0.084 (P value: 0.035)†
Platelet count < 150 × 109/L	2 (9.5%)	0 (0.0%)	215 (43.6%)	64 (46.0%)
Median CRP	86 mg/l	102 mg/l	217 mg/l	184 mg/l
- Correlation (Pearson)	0.093 (P value: 0.652)		−0.021 (P value: 0.595)
CRP > 100 mg/L	7 (33.3%)	3 (60.0%)	366 (74.2%)	102 (73.4%)
Reduced cardiac function - Correlation (Kendall’s τ)	1 (4.8%)	1 (20.0%)	198 (40.2%)	57 (41.0%)
	0.225 (P value: 0.260)		0.009 (P value: 0.823)
Pericardial effusion - Correlation (Kendall’s τ)	0 (0.0%)	3 (60.0%)	150 (30.4%)	39 (28.1%)
	0.740 (P value: < 0.001)*		−0.021 (P value: 0.590)
Headache - Correlation (Kendall’s τ)	3 (14.3%)	1 (20.0%)	80 (16.2%)	31 (22.3%)
	0.062 (P value: 0.755)		0,066 (P value: 0.097)
Abdominal pain - Correlation (Kendall’s τ)	7 (33.3%)	3 (60.0%)	236 (47.9%)	77 (55.4%)
	0.216 (P value: 0.280)		0.062 (P value: 0.117)
Rash - Correlation (Kendall’s τ)	17 (81.0%)	4 (80.0%)	472 (95.7%)	57 (41.0%)
	−0.010 (P value: 0.962)		0.614 (P value: < 0.001)*
Mucocutaneous lesions - Correlation (Kendall’s τ)	16 (76.2%)	5 (100.0%)	460 (93.3%)	32 (23.0%)
	0.238 (P value: 0.234)		0.695 (P value: < 0.001)*
Model by Starnes et al20	22 (84.6%)	3 (11.5%)	n/a	n/a
- Sensitivity - Specificity - κ-coefficient	90.5% 60.0% 0.347 (P value: 0.043)*		n/a n/a n/a
Model by Godfred-Cato et al9	17 (65.4%)	3 (11.5%)	233 (36.9%)	279 (44.1%)
- Sensitivity - Specificity - κ-coefficient	66.7% 80.0% 0.059 (P value: 0.619)		44.6% 20.1% 0.226 (P value: < 0.001)*

The table depicts the number of cases, which fulfill each of the criteria of the 2 different prognostic models for each dataset and each diagnosis, respectively. Additionally, a correlation analysis has been performed for each criterion and diagnosis.

^*Statistically significant results (95% CI).

^†Results are not considered statistically significant anymore after correction for multiple testing (Benjamini-Hochberg). The sensitivity and specificity for each model and the cases identified correctly are also listed here. The kappa coefficient serves as a parameter for performance comparison between the models.

DISCUSSION

This study examined the phenotypic distinguishability of the 2 diagnoses, KD and PIMS. The 2 prognostic models could not reliably differentiate between KD and PIMS, as shown by the calculated sensitivity and specificity measures. One of the models (by Godfred-Cato et al⁹) was applicable to both datasets and, in comparison, scored worse being applied to the larger dataset from the DGPI.

Furthermore, we performed a correlation analysis to gather information on the ability of each parameter to distinguish the diagnoses. The only parameter showing a consequent positive correlation with PIMS diagnosis throughout both cohorts was age. However, this parameter must be viewed with caution, since the median age and mostly affected age groups have been shifting with the different SARS-CoV Variants. The number of hospitalized infants has been rising and fewer teenagers have been admitted to the hospital.²⁶ Another variable with a significant correlation in cohort 1 was pericardial effusion, favoring the diagnosis of PIMS. This finding has already been discussed by Starnes et al,²⁰ as they also did not reproduce this result. Interestingly, the dataset from the DGPI also demonstrated the opposite with pericardial effusion being more common in KD patients. The only parameters consistently utilized in all prognostic models to date are CRP and platelet count.^9,18–20 None of these factors appeared to have a positive correlation with the diagnosis of PIMS here.

In general, the data used in this study mostly presented the same peculiarities for each diagnosis as observed in previous research. We were again able to confirm augmented presentation of dermatological and mucosal changes in KD patients, while PIMS cases rather exhibited involvement of the abdominal and thoracic organs.^{3,10,14,16,27} An exception besides pericardial effusion is the blood platelet count. In cohort 2, the count is reduced in PIMS cases being congruent with research, whereas in cohort 1, none of the PIMS cases but all the KD cases showed diminished platelets. In addition, serum sodium is not a commonly observed parameter in research so far, but has been utilized by Starnes et al²⁰ and Clark et al²⁸ in their model. In this study, we were not able to reproduce their findings of lower serum sodium levels in PIMS patients.

In concordance with these results, a theory of a disease spectrum including major hyperinflammatory syndromes, such as KD and PIMS, has already been proposed in research.^14,29 Nevertheless, the assessment of this study is limited to the respective phenotypes. While phenotypically the results do not oppose such theory, on their own, they are insufficient to establish causality.

With regard to the pathomechanism, parallels have been discovered as well: Although the exact pathomechanism behind KD is not fully understood yet, Rowley et al³⁰ detected viral-like cytoplasmic inclusion bodies in acute KD tissue. Analogically, in PIMS patients, residual COVID-19 viral particles have been observed to persist in infected tissues as well and seem to trigger the immune system to produce an exaggerated inflammatory response.^10,15 Second, the effectiveness of intravenous immunoglobulins in KD patients demonstrates the presence of ready antibodies against the causative agent in the general population. Therefore, this agent must be common and likely to cause subliminal infections in the normal population.^23,30 The condition of being genetically susceptible to contracting KD has been attributed to multiple genetic polymorphisms.⁴ Parallelly, genetic susceptibility in PIMS patients was also proposed, describing defects in the OAS1/2 or Rnase L genes. These proteins and endoribonuclease are responsible for degrading single-stranded viral or human RNA and therefore contribute to the phagocyte response to SARS-CoV-2.³¹

Other findings by Goetzke et al³² describe the concurrent increase in TGFβ1 serum levels as a potential driver of the hyperinflammatory immune response in PIMS patients. An expansion of EBV-specific T-cells in PIMS patients was observed. But, according to Goetzke et al,³² the increased TGFβ1 levels impair T-cell reactivity and ultimately the elimination of infected cells, whilst reactivating latent EBV in B-lymphocytes. Thus, children with latent EBV are proposed to have an increased risk of developing PIMS.³² Interestingly, increased TGFβ1 levels and single-nucleotide polymorphisms of the respective gene associated with genetic susceptibility have also been described in KD patients.^33,34

Nevertheless, significant differences cannot be ignored: While the discussed findings suggest similar pathomechanisms for both disease entities, they do not explain the differences in epidemiological distribution. Furthermore, there has been no association described between KD and EBV. However, a comprehensive discussion of pathomechanisms and spectrum theory is beyond the scope of this manuscript.

This study has both strengths and limitations: A major strength of this study is the size of the DGPI dataset. Unlike many studies, it collected both PIMS and KD cases simultaneously, thus avoiding the bias of different reporting times. These are the major limitations of the smaller dataset from Bayreuth. However, as the results presented were rather consistent throughout both datasets, a larger distortion can be excluded. A common disadvantage for studies utilizing data collected in the work field is individual reporting, and applies here: How much data was to be reported, as well as when measurements such as LVEF were to be taken, was totally at the individual physician‘s discretion. Additionally, this caused the exclusion of a significant number of cases due to incomplete data, but by the random distribution of the excluded cases, a greater bias was avoided. As mentioned before, the gold standard used as reference in this study was again the individual physician’s diagnosis, which results from the changing disease definition over time in case of PIMS. Finally, as serum sodium was not included in the PIMS-Survey, this study was also restricted to the application of only one model on this specific dataset.

Finally, this study could not confirm the phenotypic criteria of distinction between KD and PIMS. Existing prognostic models are yet unable to reliably distinguish the diagnoses based on the phenotypes of both diseases. In conclusion, their distinction needs to rely on nonphenotypical properties. Developing a test, if possible, which is also applicable in everyday hospital settings, is going to be the objective for future research.

ACKNOWLEDGMENTS

Special gratitude is owed to the working group of the “Deutsche Gesellschaft für Pädiatrische Infektiologie,” since this study utilized inter alia the data from their “PIMS-Survey.” Its results were previously published and are available both on their website (https://dgpi.de/pims-survey-update/) and as an article of the Nature Publishing Group journal “Scientific reports” named “A comparison of pediatric inflammatory multisystem syndrome temporarily-associated with SARS-CoV-2 and Kawasaki disease” (https://www.nature.com/articles/s41598-022-26832-5). Herefore, I specifically thank Prof. Dr. med. Markus Hufnagel (Freiburg, Germany) and Prof. Dr. med. Reinhard Berner (Dresden, Germany).