Abstract
Background
To construct an accurate, reliable, and simple scoring system of improving HFMD diagnosis.
Material/Methods
Based on the following 3 steps, a simple scoring diagnostic system was built: (1) we selected basic markers (age and sex), markers recommended in HFMD diagnosis guidelines, and significant biomarkers among severity groups found in a large dataset; (2) we used positive constituent ratio for determining scores of each marker; and (3) we applied receiving operating curve in an external dataset to determine the optimal cut-off score.
Results
The selected markers were sex, age, fever, skin rashes, nervous system disorder, respiratory system disorder, digestive system disorder and cardiopulmonary complications, C-reactive-protein, White Blood Cell, Creatinine Kinase, Creatinine Kinase Isoenzyme, Gamma-Glutamyl Transpeptidase, Albumin, Globulin, Albumin/Globulin Ratio, Natrium, Chloride, Calcium, and Glucose. A simple scoring system with 3.9684 as the lower cut-off was constructed. The AUC was 0.918 (95% CI: 0.874–0.963, P<0.01). The sensitivity, specificity, and Youden Index, which were based on the validation dataset of 200 subjects (80 cases, 120 non-cases with skin rashes or fever), were 0.95, 0.90, and 0.85, respectively.
Conclusions
This simple scoring system is an effective method to diagnose HFMD.
MeSH Keywords: Epidemiologic Study Characteristics as Topic; Hand, Foot and Mouth Disease; Research Design
Background
Hand, foot, and mouth disease (HFMD) is an acute infectious disease caused by Enterovirus A, of which the most commonly reported genotypes are Coxsackievirus A6, A10, and A16 (CV-A6, CV-A10, CV-A16) and Enterovirus A71 (EV-A71) [1]. HFMD commonly occurs in preschool children. In China, children under 3 years old are at the highest risk of HFMD [2]. The main clinical manifestations of HFMD are maculopapular rash and vesicles on the hands, feet, mouth, or buttocks. Some rare HFMD cases have other symptoms that are related to EV-A71, such as viral meningitis, encephalitis, encephalomyelitis, pulmonary edema, and circulatory disorder. The primary causes of death are brainstem encephalitis and neurogenic pulmonary edema.
Since its original identification in New Zealand in 1957, HFMD has been frequently reported worldwide [3]. Additionally, it has become a public health priority in China before its first large-scale epidemic in Fuyang City, Anhui Province in 2008 [4,5]. During the epidemic season, kindergarten, nursery, and family can be infected by intra-familial and intra-community transmissions [6]. The enterovirus of HFMD transmits rapidly, is strongly contagious, and is a main cause of asymptomatic infection; thus, the disease can be potential dissemination and controlling its transmission is challenging [7].
The clinical manifestations of HFMD are diverse and nonspecific. Therefore, it is difficult to perform an accurate tentative diagnosis in clinical practice. During the epidemic season, pre-school children who have rashes on hands, feet, mouth, or buttocks, and who are with or without fever, are suspected to have HFMD according to the diagnostic criteria of HFMD [1]. Although the main symptoms of HFMD are skin rashes with or without fever, it can be easily confused with other types of pyretic or exanthematous diseases, such as measles, urticaria, rubella, exanthema subitum, varicella, scarlatina, herpangina, and herpes simplex. Laboratory examinations are usually used to confirm suspected HFMD cases, but the detection rates are highly dependent on factors such as quality of specimens and incubation periods [8]. As a result, the diagnosis of HFMD can be misdiagnosed or missed, which delays delivery of optimal treatments.
With advances in the development and application of mathematical methods, quantitative methods, and statistical methods in medicine, analysis methods such as diagnostic scale, discrimination model, regression analysis, and system evaluation are being integrated into epidemiological and clinical research. Scoring systems have been applied to evaluate the risk of mortality in intensive care units (ICU) and are integrated into the intensive care process. Scoring systems allow between- and within-individual comparisons over time and also provide useful information for comparing the severity of illness in patients who were enrolled into clinical trials [9]. Two mortality risk scoring systems – Pediatric Risk of Mortality (PRISM) and the Pediatric Index of Mortality (PIM) scores – are widely applied in pediatrics [10,11]. A number of studies, mainly in developed countries, as well as in a few resource-restricted settings, have validated PIM and its updated version, PIM2 scores [12,13]. Previous studies in China have established scoring systems based on the internationally applied scoring systems (PIM or PCIS) and the clinical manifestation of HFM, to assess the severity of HFMD [14,15]. To improve HFMD disease management by detecting disease at early stages, our study developed a simple scoring system to quickly and accurately diagnose HFMD, especially where laboratory-confirmed tests are not available.
Material and Methods
Ethics statement
The present study was approved by the Ethics Committee of the First Affiliated Hospital of Shantou University Medical College. The methods carried out were accordance with the approved guidelines as well as the guidelines for the protection of human subjects. Due to the retrospective nature of the study, informed consent was not required.
Training dataset
Data on subjects were retrospectively collected from medical records of all the clinically diagnosed HFMD pediatric inpatients, aged at 14 years old or younger, who visited the pediatric department of the First Affiliated Hospital of Shantou University Medical College between January 2012 and December 2014. The diagnostic criteria of HFMD were based on the HFMD diagnosis and treatment guideline (2010 edition) of the Ministry of Health, China [1]: (1) HFMD emerge mainly in summer and autumn in preschool children, especially in infants; (2) Patients have fever accompanied with rash on hands, feet, mouth, or buttocks. Some patients may have no fever; (3) In a minority of severe patients who have atypical rash and are difficult to diagnose, clinical diagnosis primarily depends on etiologic or serologic test results; and (4) It is not recommended to diagnose HFMD for patients without rash.
HFMD cases were categorized into 3 groups (mild, severe, and critical) [1]. Patients with any neurological complications (e.g., aseptic meningitis, encephalitis, encephalomyelitis, and acute flaccid paralysis or autonomic nervous system dysregulation) were categorized as severe cases. Patients with any 1 of the following 3 groups of clinical manifestations were categorized as critical cases: (1) frequent cramp, coma, and cerebral hernia; (2) cardiopulmonary complications (e.g., dyspnea, cyanosis, frothy sputum, cardiopulmonary edema, and pulmonary rale; and (3) shock or other dysfunctions in the circulatory system. Probable or confirmed HFMD patients that were neither critical nor severe were categorized as mild cases.
All data were collected from medical records of the patients’ first physical examination and blood test on the first day of hospital admission, including: (1) sex and age; (2) clinical symptoms such as skin rashes, fever, nervous system, respiratory system, digestive system, and cardiopulmonary complications; and (3) biomarkers such as HBDH, LDH, LDH1, AST, ALT, GGT, ALP, CHE, MAO, AFU, TP, ALB,GLB, Tbil, Dbil, Ibil, A/G, K+, Na+, Cl−, Ca2+, CO2CP, BUN, Cr, CRP, WBC, CK, CK-MB, and GLU. The diagnosis of clinical symptoms was based on the clinical diagnosis of their first physical examination. The measurements of biomarkers were based on the blood samples of their first laboratory test since admission. In addition, patients were considered to have skin rashes if they had maculopapular or vesicular rash on locations such as hands, feet, mouth, buttocks, elbows, trunk, or face. Patients were considered to have a nervous system disorder if they had altered mental status, somnolence, skittishness, headache, vomiting, delirium or coma, extremity tremor, myoclonus, nystagmus, ataxia, oculomotor disorder, atony or acute flaccid paralysis, convulsions, startle, seizures, or neck resistance.
Development of the scoring system
A simple scoring system for HFMD diagnosis was developed in the following steps: (1) We selected relevant markers including basic markers (age and sex), markers listed in HFMD clinical guidelines of China, and biochemical markers that were significantly different among severity groups in the training dataset; (2) We estimated the score of each marker by using positive constituent ratio to construct the simple scoring system; and (3) We determined the optimal cut-off of the simple scoring system in the validation dataset.
Markers selection through data analysis of training dataset
All biochemical markers in the training dataset were common biochemical markers used in clinical practice.
Analysis of variance (ANOVA) was used to analyze the association between HFMD and biochemical markers. When a biochemical marker was found to be statistically different among the 3 groups, multiple comparisons were used for pairwise comparison of the mean difference. Bonferroni corrections were applied to adjust for multiple comparisons. In the cases of homogenous variance, Fisher’s least significant difference (LSD) and Student-Newman-Keuls (SNK) tests were used for pairwise multiple comparison; in the cases of heterogeneous variance, Dunnett analysis (T3) was used for pairwise multiple comparison. For mean square analysis of all groups, all 2-sided P<0.01 were considered statistically significant; for pairwise multiple comparison, all 2-side P<0.05 were considered statistically significant. Markers that were significantly different in 2 of the 3 groups were selected as markers of the simple scoring system.
Assign score to markers
The score of each marker was estimated using positive constituent ratio. The age and sex of the 1404 cases were detailed and accurately recorded; hence, their scores were established based on 1404 cases of the sample. The other 18 selected markers were completely recorded in 985 cases; therefore, their scores were established based on 985 cases of the sample.
Age and sex were categorical markers. Of the 1404 cases of this study, 915 (65.17%) were male and 489 (34.83%) were female, which means if the patient was male, his sex score would be 0.6517; if the patient was female, the sex score would be 0.3483. Similarly, the score of age group 0–3 years old, 4–6 years old, and 7–14 years old were 0.7479, 0.2321, and 0.020, respectively.
The score of 6 clinical manifestations (fever, skin rashes, nervous system disorder, respiratory system disorder, digestive system disorder, and cardiopulmonary complications) were separately obtained by calculating the proportion of cases with positive test results. For example, if 882 of the 985 cases had fever, then the percentage of fever was (882/985)×100%=89.54%; which means the score of fever in those pediatric patients who had fever was 0.8954, and the score of fever for pediatric patient who did not have fever was 0. The calculation of score for the remaining second type of markers followed the same rules.
The scores of the 12 laboratory parameters were obtained by calculating the proportion of abnormal cases in the whole sample. For example, if among the 985 cases, 356 CRP values were abnormal, then the proportion of abnormal CRP was (356/985) ×100%=36.14%, which means the score of CRP for pediatric patient who had abnormal CRP value was 0.3614, and the score of CRP for pediatric patient who had normal CRP value was 0. The calculation of score for the remaining third type of markers followed the same principle.
The formula of the scoring system was S=X1+X2+X3+ … +X20, where X1, X2, X3, … +X20 were the score of each marker, and S was the HFMD diagnosis score.
Validation dataset
Confirmed HFMD cases by IgM antibody assays whose age were 14 years old or younger were retrospectively and randomly sampled from the inpatients in the Pediatric Department of the First Affiliated Hospital of Shantou University Medical College in 2015. To assess the performance of our scoring system in distinguishing HFMD cases from other diseases such as measles, urticaria, rubella, exanthema subitum that share similar symptoms as HFMD, non-cases in the validation dataset were chosen among patients of other diseases (e.g., measles and rubella) that showed symptoms of skin rashes and with or without fever. All non-cases were age 14 years or younger, with fever or skin rashes, and were confirmed to be other diseases without HFMD.
A total of 200 participants, including 80 confirmed HFMD cases and 120 non-cases, were enrolled into the validation dataset. Among 80 cases, the serotypes were EV-A71 (32 cases), CV-A16 (22 cases) and cross-contamination of both viruses (26 cases). The types of non-cases were measles (9 cases), rubella (12 cases), exanthema subitum (8 cases), and other similar diseases (91 cases).
EVs detection
Blood samples were clinically collected from serum when patients visited the clinic or at admission. EV71-IgM and CV-A16-IgM antibody in serum and cerebrospinal fluid specimens were detected using immune colloidal gold technique, according to the manufacturer’s instructions. EV71-IgM and CV-A16-IgM antibody detection kits (Wantai Biological Pharmacy Enterprise Co., Ltd., Beijing, China) were used for the detection.
Discrimination analysis
The diagnostic accuracy of the simple scoring system was assessed in 3 steps: (1) We calculated the score of all subjects in the validation dataset; (2) We calculated the sensitivity, specificity, Youden Index (YI) and AUC; and (3) We determined the optimal cut-off score that corresponded to the largest YI. The outcomes (case, non-case) were masked, and then the score of all subjects in the validation dataset was calculated using the simple scoring system. The sensitivity and 1-specificity of all cut-off values of ROC curve were exported to a spreadsheet to obtain the optimal cut-off score that corresponded to the largest YI (Supplementary Table 1). The scoring system was performed with complete data and missing data at different levels of severity to assess the robustness of its sensitivity.
The flowchart is shown in Figure 1.
Figure 1.
The flowchart of creating a simple score system. HFMD – hand, foot and mouth disease; ROC – receiver operating curve; X1, X2, …, X20 – the score of the 20 markers; S – score of hand, foot and mouth disease.
Examples of how the scoring system calculates the scores of patients are shown in Figure 2. All statistical analyses were performed using IBM SPSS version 21 (SPSS, Inc., Chicago, IL, USA).
Figure 2.
Examples of using the simple scoring system to diagnose HFMD cases. CRP – C-reactive protein; WBC – white blood cell; CK – creatinine kinase; CK-MB – creatinine kinase isoenzyme; GGT – gamma-glutamyl transpeptidase; ALB – albumin; GLB – globulin; A/G – albumin/globulin ratio; Na+ – natrium ion; Cl− – chloride ion; Ca2+ – calcium ion; GLU – glucose; HFMD – hand, foot and mouth disease; X1, X2, …, X20 – the score of the 20 markers; S – score of hand, foot, and mouth disease.
Results
Subject characteristics of training dataset
A total of 1404 cases (1104 mild, 252 severe, and 48 critical) were recruited into the training dataset (Table 1). The frequency (percentage) of male and female were 915 (65.2%) and 489 (34.8%), respectively. The age of cases ranged from 0 to 14 years old. Subjects who were 3 years old or younger had the highest incidence, which were 1050 cases (74.8%); subjects who were 4–6 years old had the second highest incidence, which were 326 cases (23.2%); and subjects who were 7–14 years old were the least, which were 28 cases (2.0%). There were 799 cases (56.9%) from rural areas and 605 cases (43.1%) from urban areas. In total, 152 cases (10.8%) had a history of HFMD.
Table 1.
Subject characteristics of training and validation dataset.
| Dataset | Training dataset (n=1404) | Validation dataset (n=200) | |
|---|---|---|---|
| HFMD cases (n=80) | Non-cases (n=120) | ||
| Sex (n and%) | |||
| Male | 915 (65.2) | 45 (56.3) | 72 (60.0) |
| Female | 489 (34.8) | 35 (43.7) | 48 (40.0) |
| Age (n and%) | |||
| 0–3 y | 1050 (74.8) | 58 (72.5) | 107 (89.2) |
| 4–6 y | 326 (23.2) | 21 (26.3) | 12 (10.0) |
| 7–14 y | 28 (2.0) | 1 (1.2) | 1 (0.8) |
| Clinical manifestations (n and%) | Based on 985 cases | ||
| Fever | 882 (89.5) | 71 (88.8) | 68 (56.7) |
| Skin rashes | 967 (98.2) | 80 (100.0) | 63 (52.5) |
| Nervous system disorder | 234 (23.8) | 47 (58.8) | 32 (26.7) |
| Respiratory system disorder | 294 (29.9) | 25 (31.3) | 23 (19.2) |
| Digestive system disorder | 103 (10.5) | 17 (21.3) | 9 (7.5) |
| Cardiopulmonary complications | 6 (0.6) | 3 (3.8) | 1 (0.8) |
Subject characteristics of validation dataset
A total of 200 subjects (80 subjects with HFMD, 120 subjects without HFMD) were randomly selected into the validation dataset (Table 1). Among HFMD cases, the frequency of males and females were 45 (56.3%) and 35 (43.7%), respectively; their mean age was 1.56 years old, ranging from 0 to 14 years old, in which 58 (72.5%) were under 3 years old; 71 subjects (88.8%) were mild cases and 9 subjects (11.2%) were severe cases. Among subjects without HFMD, the frequency of males and females were 72 cases (60.0%) and 48 cases (40.0%), respectively; their mean age was 1.48 years old, ranging from 0 to 14 years old.
Markers selection
Summary statistics of biochemical markers that were significantly different among the mild, severe, and critical cases in the training dataset are shown in Table 2. White Blood Cell (WBC), Creatinine Kinase (CK), Creatinine Kinase Isoenzyme (CK-MB), Glucose (GLU), C-Reactive Protein (CRP), Gamma-Glutamyl Transpeptidase (GGT), Albumin (ALB), Globulin (GLB), Albumin/Globulin ratio (A/G), Natrium Ion (Na+), Chloride Ion (Cl−), and Calcium Ion (Ca2+) were significantly different among the groups (mild, severe, and critical) with P<0.01. Those biochemical markers were included as markers of the simple scoring system. A total of 20 markers were included in the simple scoring system, including basic markers (age and sex), markers selected from the HFMD diagnosis and treatment guidelines of China and markers that were found to be significantly different among the groups in the training dataset.
Table 2.
Mean comparison of biochemistry parameters.
| Biochemical markers | Mild (n=1104) | Severe (n=252) | Critical (n=48) | P |
|---|---|---|---|---|
| HBDH | 258±81.2 | 265±73.3 | 277±92.8 | 0.195 |
| LDH | 348±102 | 360±102 | 371.5±150 | 0.114 |
| LDH1 | 91.5±36.3 | 92.5±31.9 | 106±48.2 | 0.034 |
| AST | 40.5±15.7 | 43.3±42.2 | 45.8±22.1 | 0.115 |
| ALT | 19.7±22.5 | 25.0±65.2 | 20.6±12.2 | 0.112 |
| GGT | 11.0±7.09+ | 11.8±8.08+ | 14.7±17.0#* | 0.006 |
| ALP | 229±74.4 | 238±127 | 242±72.6 | 0.247 |
| CHE | 8.57±1.95 | 8.65±1.95 | 7.98±1.75 | 0.147 |
| MAO | 8.70±1.94 | 5.40±3.56 | 4.98±3.13 | 0.384 |
| AFU | 22.3±7.20 | 22.0±8.16 | 23.3±6.59 | 0.606 |
| TP | 65.9±6.12 | 67.0±6.47 | 67.7±7.27 | 0.016 |
| ALB | 42.7±3.78+ | 42.8±3.56+ | 40.1±4.67#* | <0.000 |
| GLB | 23.3±5.34*+ | 24.2±5.74#+ | 27.6±7.89#* | <0.000 |
| Tbil | 8.94±5.05 | 8.48±4.73 | 8.25±4.54 | 0.332 |
| Dbil | 1.95±1.54 | 1.85±1.15 | 1.68±0.882 | 0.341 |
| Ibil | 7.02±4.46 | 6.63±4.08 | 6.58±4.20 | 0.401 |
| A/G | 1.94±0.631+ | 1.88±0.527+ | 1.58±0.492#* | 0.001 |
| K+ | 4.33±0.535 | 4.37±0.509 | 4.41±0.696 | 0.441 |
| Na+ | 137±3.63*+ | 136±3.73# | 134.6±4.40# | <0.000 |
| Cl− | 102±3.34* | 101±3.77# | 100.6±4.20 | <0.000 |
| Ca2+ | 2.42±0.171+ | 2.40±0.174 | 2.32±0.238# | 0.001 |
| CO2CP | 19.0±2.98 | 18.9±3.10 | 19.1±3.63 | 0.855 |
| BUN | 3.29±1.29 | 3.13±1.14 | 3.03±1.37 | 0.103 |
| Cr | 39.2±15.8 | 37.2±15.0 | 42.8±16.8 | 0.049 |
Data are summarised as mean ±SD.
Compared with Mild Group P<0.05,
Compared with Severe Group,
Compared with Critical Group.
HBDH – α-hydroxybutyrate acid; LDH – lactate dehydrogenase; LDH1 – isoenzyme; AST – aspartate aminotransferase; ALT – alanine transaminase; GGT – gamma-glutamyl transpeptidase; ALP – alkaline phosphate; CHE – cholinesterase; MAO – monoamine oxidase; AFU – α-L-fucosidase; TP – total protein; ALB – albumin; GLB – globulin; Tbil – total bilirubin; Dbil – direct bilirubin; Ibil – indirect bilirubin; A/G – albumin/globulin ratio; K+ – potassium ion; Na+ – natrium; Cl− – chloride ion; Ca2+ – calcium ion; CO2CP – carbon dioxide containing power; BUN – blood urea nitrogen; Cr – creatinine.
The assigned scores of 20 markers are shown in Table 3, they were sex (male: 0.6517; female: 0.3483), age (0–3 years old: 0.7479; 4–6 years old: 0.2321; 7–14 years old: 0.020), fever (0.8954), skin rashes (0.9817), nervous system disorder (0.2376), respiratory system disorder (0.2985), digestive system disorder (0.1046), cardiopulmonary complications (0.0061), C-Reactive Protein (CRP, 0.3614), WBC (0.4964), Creatinine Kinase (CK, 0.1533), Creatinine Kinase Isoenzyme (CK-MB, 0.1797), GGT (0.0061), ALB (0.2274), GLB (0.3563), A/G (0.2650), Na+ (0.3299), Cl− (0.1848), Ca2+ (0.1858), and Glucose (GLU, 0.2782).
Table 3.
Score of markers.
| Markers | Category | Frequency | Score |
|---|---|---|---|
| Sex | Male | 915 | 0.652 |
| Female | 489 | 0.348 | |
| Age | 0–3 years | 1050 | 0.748 |
| 4–6 years | 326 | 0.232 | |
| 7–14 years | 28 | 0.020 | |
| Fever | Symptomatic | 882 | 0.895 |
| Asymptomatic | 0 | ||
| Skin rashes | Symptomatic | 967 | 0.982 |
| Asymptomatic | 0 | ||
| Nervous system disorder | Symptomatic | 234 | 0.238 |
| Asymptomatic | 0 | ||
| Respiratory system disorder | Symptomatic | 294 | 0.299 |
| Asymptomatic | 0 | ||
| Digestive system disorder | Symptomatic | 103 | 0.105 |
| Asymptomatic | 0 | ||
| Cardiopulmonary complications | Symptomatic | 6 | 0.0061 |
| Asymptomatic | 0 | ||
| CRP | Abnormal | 356 | 0.361 |
| Normal | 0 | ||
| WBC | Abnormal | 489 | 0.496 |
| Normal | 0 | ||
| CK | Abnormal | 151 | 0.153 |
| Normal | 0 | ||
| CK-MB | Abnormal | 177 | 0.180 |
| Normal | 0 | ||
| GGT | Abnormal | 6 | 0.0061 |
| Normal | 0 | ||
| ALB | Abnormal | 224 | 0.227 |
| Normal | 0 | ||
| GLB | Abnormal | 351 | 0.356 |
| Normal | 0 | ||
| A/G | Abnormal | 261 | 0.265 |
| Normal | 0 | ||
| Na+ | Abnormal | 325 | 0.330 |
| Normal | 0 | ||
| Cl− | Abnormal | 182 | 0.185 |
| Normal | 0 | ||
| Ca2+ | Abnormal | 183 | 0.186 |
| Normal | 0 | ||
| GLU | Abnormal | 274 | 0.278 |
| Normal | 0 |
(I) age and sex, score=(/1404); (II) 6 clinical manifestation, score= (/985); (III) 12 lab parameters, score= (/985), where n=the number of abnormal cases. CRP – c-reactive protein; WBC – white blood cell; CK – creatinine kinase; CK-MB – creatinine kinase isoenzyme; GGT – gamma-glutamyl transpeptidase; ALB – albumin; GLB – globulin; A/G – albumin/globulin ratio; Na+ – natrium ion; Cl− – chloride ion; Ca2+ – calcium ion; GLU – glucose.
The score of all cases without missing data (985 cases) ranged from 1.65 to 6.07 of 419 cases that had missing data; 364 cases were mild, 37 cases were severe, and 18 cases were critical.
Discrimination analysis of ROC curve
Figure 3 reveals the AUC of the proposed HFMD diagnosis scoring system. The results showed that the simple scoring system had high diagnostic accuracy for HFMD [20] (sensitivity=0.95, specificity=0.90, AUC=0.918, 95% CI= 0.874–0.963, P<0.01). The optimal cut-off point of the simple scoring system that corresponded to the largest YI was 3.9684. The largest YI (sensitivity + specificity − 1) was 0.850. There were 4 (5%) false-negative cases, which were all mild cases. There were 12 (10%) false-positive cases, which were severe pneumonia, Kawasaki disease, urticaria, and herpangina. At prevalence=40% (80 HFMD cases out of 200 subjects), the positive and negative predictive values of the scoring system were 86.4% and 96.4%, respectively. Therefore, at prevalence=40%, the probability of being cases among subjects who tested positive was 86.4%, and the probability being non-cases among subjects who tested negative was 96.4%.
Figure 3.
The Receiver Operating Characteristic (ROC) curve of the simple scoring system. The curve represents the receiver operating characteristics of the simple scoring system. The blank area bounded by the curve and straight line is the area under the curve of ROC.
Sensitivity analysis
Using the scoring system in cases with completed data, 3 (10.0%) of the 29 critical cases, 23 (11.0%) of the 213 severe cases, and 222 (30.0%) of the 743 mild cases were detected as non-cases. Using the scoring system in cases with missing data, 134 (36.8%) of the 364 mild cases, 11 (29.7%) of the 37 severe cases, and 9 (50%) of the critical cases were detected as non-cases (Table 4).
Table 4.
Sensitivity and false negative of the simple scoring system in HFMD cases with complete and missing data by severity levels.
| HFMD severity | Cases with complete data | Cases with missing data | ||
|---|---|---|---|---|
| Sensitivity | False-negative | Sensitivity | False-negative | |
| Mild | 521/743 (70%) | 222/743 (30%) | 134/364 (36.8%) | 230/364 (63.2%) |
| Severe | 190/213 (89%) | 23/213 (11%) | 11/37 (29.7%) | 26/37 (71.3%) |
| Critical | 26/29 (90%) | 3/29 (10%) | 9/18 (50%) | 9/18 (50%) |
| Total | 736/984 (75%) | 248/984 (25%) | 154/419 (36.8%) | 265/419 (63.2%) |
Discussion
HFMD is a major public health concern in China, seriously threatening the health of children. Despite the long history of HFMD, China still lacks effective prevention and treatment measures. Therefore, in this study, a simple scoring system was developed to diagnose HFMD quickly, accurately, and reliably at the early stage to prevent disease progression by implementing timely disease management, especially in the medical centers without laboratory-confirmed tests. Our major findings were as follows: (1) This scoring system demonstrated a good discrimination in HFMD cases, with an AUC of 0.918, based on 20 clinical variables collected at the time of admission. (2) All 20 markers of this scoring system can be easily obtained and are comprehensive, including 2 basic markers, 11 markers selected by guidelines of HFMD diagnosis and treatment, and 7 laboratory parameters selected by the chi-square statistic. (3) The score of each marker is determined by calculating its positive constituent ratio, which is simple and objective and can be estimated by its weight. (4) We provided a framework and step-by-step guidelines of the simple scoring system, which can be applied to diagnose other diseases by re-adjusting the markers and scores.
Our validation results showed that the simple scoring system had high diagnostic accuracy for HFMD (sensitivity=0.95, specificity=0.90, AUC=0.918, the positive predictive value=86.4%, the negative predictive value=96.4%). A previous report presented a mortality risk score model comprising 4 laboratory parameters with good discrimination (AUC >0.9) [16]; however, the model can only discriminate children with high mortality risk from the severe HFMD cases. Another study developed a prediction system for identification of the severe HFMD based on 14 variables with an AUC of 0.916 [17], but this system can only discriminate the severe HFMD cases from the mild HFMD cases. Compared to these studies previous [16,17], our study had a different purpose – to discriminate HFMD cases from non-HFMD cases. In addition, the present study exhibited superior performance in sensitivity, specificity, and AUC. One possibility is that our system covers more parameters than other studies, including clinical features, laboratory indicators, and demographic variables.
Our scoring system included a total of 12 biochemical markers: CRP, WBC, CK, CK-MB, GLU, GGT, ALB, GLB, A/G, Na+, Cl−, and Ca2+. The significant differences in severity among different cases suggest that these markers could play a role in the disease progression of HFMD. Addressed by multiple studies, some of the biochemical markers change with the severity of HFMD [18–20], and the results can be inconsistent. Previous studies [21,22] reported that GLU or WBC is the marker for HFMD disease progression, rather than the risk factors for HFMD complications. Moreover, the CRP and CK-MB levels are correlated with HFMD severity [23]. Furthermore, Cl− is an independent risk factor, along with suitable combinations of other risk factors, and can be useful for the detection of severe cases [24]. A recent study [17] found that GLU, platelet, percentage of lymphocytes, LDH, ALP, CK, CK-MB, Cr, uric acid, Cl−, and ALT are important independent risk factors for severe HFMD. In addition to the factors mentioned in previous studies, our study indicated that other factors can also be used to predict HFMD, such as GGT ALB, GLB, A/G, Na+, and Ca2+.
The criterion gold standard diagnosis for HFMD is the serological test. However, this diagnosis requires special equipment, and is time-consuming and difficult to perform in general medical centers or for every patient. In these settings, our study provides a simple and alternative method for HFMD diagnosis. All the markers in our scoring system can be collected within 1 h after hospital admission. In addition, we can get the marker scores as long as the parameter results are within the normal range; if they are outside the normal range, one does not need to know the specific value of the laboratory parameters. This is one of the important differences in our simple scoring system compared to other scoring systems [25,26].
The application of our scoring system can be extended. Although this system was built based on inpatient data, it can also be applied to outpatients. Because the markers included in this system are common and easy to obtain, most outpatients also need to undergo these physical examinations and laboratory texts. Therefore, the scoring system for the outpatients could be developed by refining the assigned score. Since we have established a relatively simple diagnostic system, and because external data validation has proved its high performance for HFMD diagnosis, our diagnostic method can also be applied to other disease diagnostic systems.
In this study, the limitations include: first, this was a single-center study, which could be population-specific. Second, the sensitivity and false-negative results were poor in some HFMD cases with missing data by severity levels. These findings indicate that our scoring system may not be suitable when only some of the 20 parameters are available. Fortunately, these 20 parameters are available in most medical institutions in China. Third, although the results of the comparison of diagnostic values across different levels of severity indicated that the sensitivities of the critical and severe cases were greater than that of the mild cases, it cannot distinguish the severity of HFMD. Due to these limitations, this scoring system should be further improved by refining the parameters in prospective and multi-center clinical samples.
Conclusions
In summary, we developed an alternative simple scoring system for HFMD diagnosis with high accuracy and reliability. This simple scoring system is recommended for use in similar clinical settings for early HFMD diagnosis or can be further updated or developed for use in different clinical settings by using the framework and step-by-step guidance provided in this study.
We acknowledge Jianping Xiong and Suihong Qiu for establishing the database.
Supplementary Table
Supplementary Table 1.
Cut-off scores of ROC curve.
| Cut-off score | Sensiti-vity | 1-Specificity | Specificity | YI | Cut-off score | Sensiti-vity | 1-Specifi-city | Specificity | YI | Cut-off score | Sensiti-vity | 1-Specifi-city | Specificity | YI |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| −.141300 | 1.000 | 1.000 | 0.000 | 0.000 | 2.962550 | 1.000 | .417 | 0.583 | 0.583 | 4.016550 | .688 | .083 | 0.917 | 0.604 |
| .871200 | 1.000 | .992 | 0.008 | 0.008 | 2.984350 | 1.000 | .408 | 0.592 | 0.592 | 4.019000 | .675 | .083 | 0.917 | 0.592 |
| 1.066650 | 1.000 | .983 | 0.017 | 0.017 | 2.989450 | .988 | .408 | 0.592 | 0.579 | 4.021900 | .663 | .083 | 0.917 | 0.579 |
| 1.324550 | 1.000 | .975 | 0.025 | 0.025 | 3.033900 | .988 | .400 | 0.600 | 0.588 | 4.023300 | .650 | .083 | 0.917 | 0.567 |
| 1.480850 | 1.000 | .958 | 0.042 | 0.042 | 3.079450 | .988 | .392 | 0.608 | 0.596 | 4.024000 | .638 | .083 | 0.917 | 0.554 |
| 1.594550 | 1.000 | .950 | 0.050 | 0.050 | 3.085800 | .988 | .383 | 0.617 | 0.604 | 4.026550 | .625 | .083 | 0.917 | 0.542 |
| 1.642600 | 1.000 | .933 | 0.067 | 0.067 | 3.094500 | .988 | .375 | 0.625 | 0.613 | 4.029100 | .600 | .083 | 0.917 | 0.517 |
| 1.699600 | 1.000 | .925 | 0.075 | 0.075 | 3.107200 | .988 | .367 | 0.633 | 0.621 | 4.029750 | .588 | .083 | 0.917 | 0.504 |
| 1.750550 | 1.000 | .917 | 0.083 | 0.083 | 3.118400 | .988 | .358 | 0.642 | 0.629 | 4.031000 | .575 | .083 | 0.917 | 0.492 |
| 1.760550 | 1.000 | .908 | 0.092 | 0.092 | 3.137150 | .988 | .350 | 0.650 | 0.638 | 4.031900 | .563 | .083 | 0.917 | 0.479 |
| 1.770000 | 1.000 | .900 | 0.100 | 0.100 | 3.167550 | .988 | .342 | 0.658 | 0.646 | 4.032650 | .538 | .083 | 0.917 | 0.454 |
| 1.792850 | 1.000 | .883 | 0.117 | 0.117 | 3.194100 | .988 | .333 | 0.667 | 0.654 | 4.034550 | .525 | .083 | 0.917 | 0.442 |
| 1.832000 | 1.000 | .875 | 0.125 | 0.125 | 3.205850 | .988 | .325 | 0.675 | 0.663 | 4.036600 | .513 | .083 | 0.917 | 0.429 |
| 1.876650 | 1.000 | .867 | 0.133 | 0.133 | 3.212300 | .988 | .317 | 0.683 | 0.671 | 4.037700 | .500 | .083 | 0.917 | 0.417 |
| 1.900200 | 1.000 | .858 | 0.142 | 0.142 | 3.226800 | .988 | .308 | 0.692 | 0.679 | 4.038650 | .488 | .083 | 0.917 | 0.404 |
| 1.914050 | 1.000 | .850 | 0.150 | 0.150 | 3.239000 | .988 | .300 | 0.700 | 0.688 | 4.039700 | .475 | .083 | 0.917 | 0.392 |
| 1.938850 | 1.000 | .842 | 0.158 | 0.158 | 3.263950 | .988 | .292 | 0.708 | 0.696 | 4.041300 | .463 | .083 | 0.917 | 0.379 |
| 2.016050 | 1.000 | .833 | 0.167 | 0.167 | 3.290850 | .988 | .283 | 0.717 | 0.704 | 4.043300 | .450 | .083 | 0.917 | 0.367 |
| 2.079350 | 1.000 | .817 | 0.183 | 0.183 | 3.312150 | .988 | .275 | 0.725 | 0.713 | 4.044800 | .438 | .083 | 0.917 | 0.354 |
| 2.082750 | 1.000 | .808 | 0.192 | 0.192 | 3.338600 | .988 | .267 | 0.733 | 0.721 | 4.045800 | .413 | .083 | 0.917 | 0.329 |
| 2.107250 | 1.000 | .800 | 0.200 | 0.200 | 3.353850 | .988 | .258 | 0.742 | 0.729 | 4.046600 | .400 | .083 | 0.917 | 0.317 |
| 2.150550 | 1.000 | .792 | 0.208 | 0.208 | 3.363450 | .988 | .250 | 0.750 | 0.738 | 4.048600 | .388 | .083 | 0.917 | 0.304 |
| 2.217100 | 1.000 | .783 | 0.217 | 0.217 | 3.419400 | .988 | .242 | 0.758 | 0.746 | 4.050150 | .375 | .083 | 0.917 | 0.292 |
| 2.266350 | 1.000 | .775 | 0.225 | 0.225 | 3.502600 | .975 | .242 | 0.758 | 0.733 | 4.050700 | .363 | .083 | 0.917 | 0.279 |
| 2.272700 | 1.000 | .767 | 0.233 | 0.233 | 3.539500 | .975 | .233 | 0.767 | 0.742 | 4.051400 | .350 | .083 | 0.917 | 0.267 |
| 2.283150 | 1.000 | .758 | 0.242 | 0.242 | 3.562550 | .975 | .225 | 0.775 | 0.750 | 4.052100 | .338 | .083 | 0.917 | 0.254 |
| 2.323500 | 1.000 | .750 | 0.250 | 0.250 | 3.598600 | .975 | .217 | 0.783 | 0.758 | 4.053650 | .325 | .083 | 0.917 | 0.242 |
| 2.356650 | 1.000 | .733 | 0.267 | 0.267 | 3.624850 | .975 | .208 | 0.792 | 0.767 | 4.055000 | .313 | .083 | 0.917 | 0.229 |
| 2.369800 | 1.000 | .725 | 0.275 | 0.275 | 3.636000 | .975 | .200 | 0.800 | 0.775 | 4.057450 | .300 | .083 | 0.917 | 0.217 |
| 2.391450 | 1.000 | .717 | 0.283 | 0.283 | 3.646100 | .975 | .192 | 0.808 | 0.783 | 4.060500 | .300 | .075 | 0.925 | 0.225 |
| 2.402950 | 1.000 | .708 | 0.292 | 0.292 | 3.661550 | .975 | .183 | 0.817 | 0.792 | 4.062550 | .288 | .075 | 0.925 | 0.213 |
| 2.412300 | 1.000 | .700 | 0.300 | 0.300 | 3.673300 | .975 | .175 | 0.825 | 0.800 | 4.063800 | .275 | .075 | 0.925 | 0.200 |
| 2.419450 | 1.000 | .692 | 0.308 | 0.308 | 3.686300 | .975 | .167 | 0.833 | 0.808 | 4.064100 | .263 | .075 | 0.925 | 0.188 |
| 2.438700 | 1.000 | .683 | 0.317 | 0.317 | 3.722300 | .975 | .158 | 0.842 | 0.817 | 4.064350 | .250 | .075 | 0.925 | 0.175 |
| 2.469800 | 1.000 | .675 | 0.325 | 0.325 | 3.761350 | .975 | .150 | 0.850 | 0.825 | 4.064950 | .238 | .075 | 0.925 | 0.163 |
| 2.490100 | 1.000 | .667 | 0.333 | 0.333 | 3.776550 | .963 | .150 | 0.850 | 0.813 | 4.065650 | .238 | .067 | 0.933 | 0.171 |
| 2.516850 | 1.000 | .658 | 0.342 | 0.342 | 3.786400 | .963 | .142 | 0.858 | 0.821 | 4.067150 | .225 | .067 | 0.933 | 0.158 |
| 2.540200 | 1.000 | .650 | 0.350 | 0.350 | 3.806900 | .963 | .133 | 0.867 | 0.829 | 4.077100 | .213 | .067 | 0.933 | 0.146 |
| 2.546800 | 1.000 | .642 | 0.358 | 0.358 | 3.835850 | .963 | .125 | 0.875 | 0.838 | 4.091350 | .200 | .067 | 0.933 | 0.133 |
| 2.553800 | 1.000 | .633 | 0.367 | 0.367 | 3.874750 | .950 | .125 | 0.875 | 0.825 | 4.110800 | .188 | .067 | 0.933 | 0.121 |
| 2.559400 | 1.000 | .625 | 0.375 | 0.375 | 3.913650 | .950 | .117 | 0.883 | 0.833 | 4.127150 | .175 | .067 | 0.933 | 0.108 |
| 2.576300 | 1.000 | .617 | 0.383 | 0.383 | 3.945600 | .950 | .108 | 0.892 | 0.842 | 4.132050 | .175 | .058 | 0.942 | 0.117 |
| 2.613000 | 1.000 | .608 | 0.392 | 0.392 | 3.968400 | .950 | .100 | 0.900 | 0.850 | 4.139000 | .163 | .050 | 0.950 | 0.113 |
| 2.634650 | 1.000 | .600 | 0.400 | 0.400 | 3.982850 | .938 | .100 | 0.900 | 0.838 | 4.148700 | .150 | .050 | 0.950 | 0.100 |
| 2.647700 | 1.000 | .592 | 0.408 | 0.408 | 3.992250 | .925 | .100 | 0.900 | 0.825 | 4.154250 | .150 | .042 | 0.958 | 0.108 |
| 2.685250 | 1.000 | .583 | 0.417 | 0.417 | 3.994750 | .913 | .100 | 0.900 | 0.813 | 4.156950 | .138 | .042 | 0.958 | 0.096 |
| 2.714850 | 1.000 | .575 | 0.425 | 0.425 | 3.997250 | .900 | .100 | 0.900 | 0.800 | 4.177450 | .125 | .042 | 0.958 | 0.083 |
| 2.719650 | 1.000 | .567 | 0.433 | 0.433 | 3.998100 | .888 | .100 | 0.900 | 0.788 | 4.210250 | .113 | .042 | 0.958 | 0.071 |
| 2.732650 | 1.000 | .558 | 0.442 | 0.442 | 3.999400 | .875 | .100 | 0.900 | 0.775 | 4.233500 | .100 | .042 | 0.958 | 0.058 |
| 2.758350 | 1.000 | .550 | 0.450 | 0.450 | 4.002150 | .875 | .092 | 0.908 | 0.783 | 4.244150 | .088 | .042 | 0.958 | 0.046 |
| 2.779850 | 1.000 | .542 | 0.458 | 0.458 | 4.004200 | .863 | .092 | 0.908 | 0.771 | 4.247300 | .088 | .033 | 0.967 | 0.054 |
| 2.789450 | 1.000 | .533 | 0.467 | 0.467 | 4.005000 | .850 | .092 | 0.908 | 0.758 | 4.268100 | .075 | .033 | 0.967 | 0.042 |
| 2.791300 | 1.000 | .533 | 0.467 | 0.467 | 4.005700 | .838 | .092 | 0.908 | 0.746 | 4.298400 | .075 | .025 | 0.975 | 0.050 |
| 2.804650 | 1.000 | .517 | 0.483 | 0.483 | 4.006200 | .825 | .092 | 0.908 | 0.733 | 4.369000 | .075 | .017 | 0.983 | 0.058 |
| 2.824500 | 1.000 | .508 | 0.492 | 0.492 | 4.007700 | .813 | .092 | 0.908 | 0.721 | 4.492850 | .063 | .017 | 0.983 | 0.046 |
| 2.840250 | 1.000 | .500 | 0.500 | 0.500 | 4.009250 | .800 | .092 | 0.908 | 0.708 | 4.708050 | .050 | .017 | 0.983 | 0.033 |
| 2.849900 | 1.000 | .475 | 0.525 | 0.525 | 4.009600 | .800 | .092 | 0.908 | 0.708 | 4.899100 | .038 | .017 | 0.983 | 0.021 |
| 2.863100 | 1.000 | .467 | 0.533 | 0.533 | 4.009700 | .775 | .083 | 0.917 | 0.692 | 4.966100 | .038 | .008 | 0.992 | 0.029 |
| 2.879800 | 1.000 | .458 | 0.542 | 0.542 | 4.010200 | .763 | .083 | 0.917 | 0.679 | 5.176100 | .038 | 0.000 | 1.000 | 0.038 |
| 2.898900 | 1.000 | .450 | 0.550 | 0.550 | 4.010750 | .738 | .083 | 0.917 | 0.654 | 5.483750 | .025 | 0.000 | 1.000 | 0.025 |
| 2.916200 | 1.000 | .442 | 0.558 | 0.558 | 4.010900 | .738 | .083 | 0.917 | 0.654 | 5.837050 | .013 | 0.000 | 1.000 | 0.013 |
| 2.931450 | 1.000 | .433 | 0.567 | 0.567 | 4.013500 | .700 | .083 | 0.917 | 0.617 | 7.065500 | 0.000 | 0.000 | 1.000 | 0.000 |
Minimum cut-off score was the result of smallest observed test score subtracting 1; Maximum cut-off score was the result of largest observed test score adding 1. All other cut-off values were the average of 2 adjacent test scores.
Footnotes
Source of support: This work was supported by the National Natural Science Foundation of China (No. 81172776), the Guangdong Science and Technology Planning Project (No. 2016A020216026), and the Department of Education, Guangdong Government under the Top-tier University Development Scheme for Research and Control of Infectious Diseases (No. 2016040)
Conflict of interest
None.
References
- 1.China Ministry of Health. Guideline for diagnosis and treatment of HFMD (2010 edition) of Ministry of Health, P.R.C. Available at: http://www.moh.gov.cn/publicfiles/business/htmlfiles/mohyzs/s3586/201004/46884.htm.
- 2.Xing W, Liao Q, Viboud C, et al. Epidemiological characteristics of hand-foot-and-mouth disease in China, 2008–2012. Lancet Infect Dis. 2014;14:308–18. doi: 10.1016/S1473-3099(13)70342-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sarma N. Hand, foot, and mouth disease: Current scenario and Indian perspective. Indian J Dermatol Venereol Leprol. 2013;79:165–75. doi: 10.4103/0378-6323.107631. [DOI] [PubMed] [Google Scholar]
- 4.Zhu Z, Zhu S, Guo X, et al. Retrospective seroepidemiology indicated that human enterovirus 71 and coxsackievirus A16 circulated wildly in central and southern China before large-scale outbreaks from 2008. Virology J. 2010;7:300. doi: 10.1186/1743-422X-7-300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Zhang Y, Zhu Z, Yang W, et al. An emerging recombinant human enterovirus 71 responsible for the 2008 outbreak of Hand Foot and Mouth Disease in Fuyang city of China. Virology J. 2010;7:94. doi: 10.1186/1743-422X-7-94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dong W, Li X, Yang P, et al. The effects of weather factors on hand, foot and mouth disease in Beijing. Sci Rep. 2016;6:19247. doi: 10.1038/srep19247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Li F, Zeng XY, Wu CH, et al. A new factor influencing pathogen detection by molecular assay in children with both mild and severe hand, foot, and mouth disease. Diagn Microbiol Infect Dis. 2013;76:162–67. doi: 10.1016/j.diagmicrobio.2013.02.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Li W, Yi L, Su J, et al. Seroepidemiology of human enterovirus71 and coxsackievirusA16 among children in Guangdong province, China. BMC Infect Dis. 2013;13:322. doi: 10.1186/1471-2334-13-322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lambert V, Matthews A, MacDonell R, Fitzsimons J. Pediatric early warning systems for detecting and responding to clinical deterioration in children: A systematic review. BMJ Open. 2017;7:e014497. doi: 10.1136/bmjopen-2016-014497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (pediatric risk of mortality) score and PIM (pediatric index of mortality) score in a tertiary care pediatric ICU. Indian J Pediatr. 2010;77:267–71. doi: 10.1007/s12098-010-0031-3. [DOI] [PubMed] [Google Scholar]
- 11.Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16:1110–16. doi: 10.1097/00003246-198811000-00006. [DOI] [PubMed] [Google Scholar]
- 12.Shann F, Pearson G, Slater A, Wilkinson K Pediatric index of mortality (PIM) A mortality prediction model for children in intensive care. Intensive Care Med. 1997;23:201–7. doi: 10.1007/s001340050317. [DOI] [PubMed] [Google Scholar]
- 13.Sankar J, Singh A, Sankar MJ, et al. Pediatric index of mortality and PIM2 scores have good calibration in a large cohort of children from a developing country. Biomed Res Int. 2014;2014 doi: 10.1155/2014/907871. 907871. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lu XL, Qiu J, Zhu YM, et al. Role of pediatric critical illness score in evaluating severity and prognosis of severe hand-foot-mouth disease. Zhongguo Dang Dai Er Ke Za Zhi. 2015;17(9):961–64. [in Chinese] [PubMed] [Google Scholar]
- 15.He F, Liu XX, Zhu LY, et al. Study on the score criteria of severe hand, foot and mouth disease cases. Zhonghua Liu Xing Bing Xue Za Zhi. 2010;31:563–66. [in Chinese] [PubMed] [Google Scholar]
- 16.Qiu J, Lu X, Liu X, et al. Derivation and validation of a mortality risk score for severe hand, foot and mouth disease in China. Sci Rep. 2017;7(1):3371. doi: 10.1038/s41598-017-02658-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Liu G, Xu Y, Wang X, et al. Developing a machine learning system for identification of severe hand, foot, and mouth disease from electronic medical record data. Sci Rep. 2017;7(1):16341. doi: 10.1038/s41598-017-16521-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Li Y, Zhu R, Qian Y, Deng J. The characteristics of blood glucose and WBC counts in peripheral blood of cases of hand foot and mouth disease in China: A systematic review. PLoS One. 2012;7:e29003. doi: 10.1371/journal.pone.0029003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Chang LY, Lin TY, Hsu KH, et al. Clinical features and risk factors of pulmonary edema after enterovirus-71-related hand, foot, and mouth disease. Lancet. 1999;354:1682–88. doi: 10.1016/S0140-6736(99)04434-7. [DOI] [PubMed] [Google Scholar]
- 20.Tian H, Yang QZ, Liang J, et al. Clinical features and management outcomes of severe hand, foot and mouth disease. Med Princ Pract. 2012;21:355–59. doi: 10.1159/000334619. [DOI] [PubMed] [Google Scholar]
- 21.Chen CY, Chang YC, Huang CC, et al. Acute flaccid paralysis in infants and young children with enterovirus 71 infection: MR imaging findings and clinical correlates. Am J Neuroradiol. 2001;22:200–5. [PMC free article] [PubMed] [Google Scholar]
- 22.Kim SJ, Kim JH, Kang JH, et al. Risk factors for neurologic complications of hand, foot and mouth disease in the republic of Korea, 2009. J Korean Med Sci. 2013;28:120–27. doi: 10.3346/jkms.2013.28.1.120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ran F, Wang Y, Zhong L, et al. The relevance of the serum levels of C-reactive protein and creatine kinase-MB to the severity of hand-foot-and-mouth disease patients in China: A meta-analysis. Pediatric Infectious Disease. 2016;8:15–25. [Google Scholar]
- 24.Li W, Teng G, Tong H, et al. Study on risk factors for severe hand, foot and mouth disease in China. PLoS One. 2014;2014:9. doi: 10.1371/journal.pone.0087603. e87603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Dudley JM, Messinezy M, Eridani S, et al. Primary thrombocythaemia: Diagnostic criteria and a simple scoring system for positive diagnosis. Br J Haematol. 1989;71:331–35. doi: 10.1111/j.1365-2141.1989.tb04288.x. [DOI] [PubMed] [Google Scholar]
- 26.Chang CC, Chen TP, Yeh CH, et al. A simple weighted scoring system to guide surgical decision-making in patients with parapneumonic pleural effusion. J Thorac Dis. 2016;8:3168–74. doi: 10.21037/jtd.2016.11.93. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Table 1.
Cut-off scores of ROC curve.
| Cut-off score | Sensiti-vity | 1-Specificity | Specificity | YI | Cut-off score | Sensiti-vity | 1-Specifi-city | Specificity | YI | Cut-off score | Sensiti-vity | 1-Specifi-city | Specificity | YI |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| −.141300 | 1.000 | 1.000 | 0.000 | 0.000 | 2.962550 | 1.000 | .417 | 0.583 | 0.583 | 4.016550 | .688 | .083 | 0.917 | 0.604 |
| .871200 | 1.000 | .992 | 0.008 | 0.008 | 2.984350 | 1.000 | .408 | 0.592 | 0.592 | 4.019000 | .675 | .083 | 0.917 | 0.592 |
| 1.066650 | 1.000 | .983 | 0.017 | 0.017 | 2.989450 | .988 | .408 | 0.592 | 0.579 | 4.021900 | .663 | .083 | 0.917 | 0.579 |
| 1.324550 | 1.000 | .975 | 0.025 | 0.025 | 3.033900 | .988 | .400 | 0.600 | 0.588 | 4.023300 | .650 | .083 | 0.917 | 0.567 |
| 1.480850 | 1.000 | .958 | 0.042 | 0.042 | 3.079450 | .988 | .392 | 0.608 | 0.596 | 4.024000 | .638 | .083 | 0.917 | 0.554 |
| 1.594550 | 1.000 | .950 | 0.050 | 0.050 | 3.085800 | .988 | .383 | 0.617 | 0.604 | 4.026550 | .625 | .083 | 0.917 | 0.542 |
| 1.642600 | 1.000 | .933 | 0.067 | 0.067 | 3.094500 | .988 | .375 | 0.625 | 0.613 | 4.029100 | .600 | .083 | 0.917 | 0.517 |
| 1.699600 | 1.000 | .925 | 0.075 | 0.075 | 3.107200 | .988 | .367 | 0.633 | 0.621 | 4.029750 | .588 | .083 | 0.917 | 0.504 |
| 1.750550 | 1.000 | .917 | 0.083 | 0.083 | 3.118400 | .988 | .358 | 0.642 | 0.629 | 4.031000 | .575 | .083 | 0.917 | 0.492 |
| 1.760550 | 1.000 | .908 | 0.092 | 0.092 | 3.137150 | .988 | .350 | 0.650 | 0.638 | 4.031900 | .563 | .083 | 0.917 | 0.479 |
| 1.770000 | 1.000 | .900 | 0.100 | 0.100 | 3.167550 | .988 | .342 | 0.658 | 0.646 | 4.032650 | .538 | .083 | 0.917 | 0.454 |
| 1.792850 | 1.000 | .883 | 0.117 | 0.117 | 3.194100 | .988 | .333 | 0.667 | 0.654 | 4.034550 | .525 | .083 | 0.917 | 0.442 |
| 1.832000 | 1.000 | .875 | 0.125 | 0.125 | 3.205850 | .988 | .325 | 0.675 | 0.663 | 4.036600 | .513 | .083 | 0.917 | 0.429 |
| 1.876650 | 1.000 | .867 | 0.133 | 0.133 | 3.212300 | .988 | .317 | 0.683 | 0.671 | 4.037700 | .500 | .083 | 0.917 | 0.417 |
| 1.900200 | 1.000 | .858 | 0.142 | 0.142 | 3.226800 | .988 | .308 | 0.692 | 0.679 | 4.038650 | .488 | .083 | 0.917 | 0.404 |
| 1.914050 | 1.000 | .850 | 0.150 | 0.150 | 3.239000 | .988 | .300 | 0.700 | 0.688 | 4.039700 | .475 | .083 | 0.917 | 0.392 |
| 1.938850 | 1.000 | .842 | 0.158 | 0.158 | 3.263950 | .988 | .292 | 0.708 | 0.696 | 4.041300 | .463 | .083 | 0.917 | 0.379 |
| 2.016050 | 1.000 | .833 | 0.167 | 0.167 | 3.290850 | .988 | .283 | 0.717 | 0.704 | 4.043300 | .450 | .083 | 0.917 | 0.367 |
| 2.079350 | 1.000 | .817 | 0.183 | 0.183 | 3.312150 | .988 | .275 | 0.725 | 0.713 | 4.044800 | .438 | .083 | 0.917 | 0.354 |
| 2.082750 | 1.000 | .808 | 0.192 | 0.192 | 3.338600 | .988 | .267 | 0.733 | 0.721 | 4.045800 | .413 | .083 | 0.917 | 0.329 |
| 2.107250 | 1.000 | .800 | 0.200 | 0.200 | 3.353850 | .988 | .258 | 0.742 | 0.729 | 4.046600 | .400 | .083 | 0.917 | 0.317 |
| 2.150550 | 1.000 | .792 | 0.208 | 0.208 | 3.363450 | .988 | .250 | 0.750 | 0.738 | 4.048600 | .388 | .083 | 0.917 | 0.304 |
| 2.217100 | 1.000 | .783 | 0.217 | 0.217 | 3.419400 | .988 | .242 | 0.758 | 0.746 | 4.050150 | .375 | .083 | 0.917 | 0.292 |
| 2.266350 | 1.000 | .775 | 0.225 | 0.225 | 3.502600 | .975 | .242 | 0.758 | 0.733 | 4.050700 | .363 | .083 | 0.917 | 0.279 |
| 2.272700 | 1.000 | .767 | 0.233 | 0.233 | 3.539500 | .975 | .233 | 0.767 | 0.742 | 4.051400 | .350 | .083 | 0.917 | 0.267 |
| 2.283150 | 1.000 | .758 | 0.242 | 0.242 | 3.562550 | .975 | .225 | 0.775 | 0.750 | 4.052100 | .338 | .083 | 0.917 | 0.254 |
| 2.323500 | 1.000 | .750 | 0.250 | 0.250 | 3.598600 | .975 | .217 | 0.783 | 0.758 | 4.053650 | .325 | .083 | 0.917 | 0.242 |
| 2.356650 | 1.000 | .733 | 0.267 | 0.267 | 3.624850 | .975 | .208 | 0.792 | 0.767 | 4.055000 | .313 | .083 | 0.917 | 0.229 |
| 2.369800 | 1.000 | .725 | 0.275 | 0.275 | 3.636000 | .975 | .200 | 0.800 | 0.775 | 4.057450 | .300 | .083 | 0.917 | 0.217 |
| 2.391450 | 1.000 | .717 | 0.283 | 0.283 | 3.646100 | .975 | .192 | 0.808 | 0.783 | 4.060500 | .300 | .075 | 0.925 | 0.225 |
| 2.402950 | 1.000 | .708 | 0.292 | 0.292 | 3.661550 | .975 | .183 | 0.817 | 0.792 | 4.062550 | .288 | .075 | 0.925 | 0.213 |
| 2.412300 | 1.000 | .700 | 0.300 | 0.300 | 3.673300 | .975 | .175 | 0.825 | 0.800 | 4.063800 | .275 | .075 | 0.925 | 0.200 |
| 2.419450 | 1.000 | .692 | 0.308 | 0.308 | 3.686300 | .975 | .167 | 0.833 | 0.808 | 4.064100 | .263 | .075 | 0.925 | 0.188 |
| 2.438700 | 1.000 | .683 | 0.317 | 0.317 | 3.722300 | .975 | .158 | 0.842 | 0.817 | 4.064350 | .250 | .075 | 0.925 | 0.175 |
| 2.469800 | 1.000 | .675 | 0.325 | 0.325 | 3.761350 | .975 | .150 | 0.850 | 0.825 | 4.064950 | .238 | .075 | 0.925 | 0.163 |
| 2.490100 | 1.000 | .667 | 0.333 | 0.333 | 3.776550 | .963 | .150 | 0.850 | 0.813 | 4.065650 | .238 | .067 | 0.933 | 0.171 |
| 2.516850 | 1.000 | .658 | 0.342 | 0.342 | 3.786400 | .963 | .142 | 0.858 | 0.821 | 4.067150 | .225 | .067 | 0.933 | 0.158 |
| 2.540200 | 1.000 | .650 | 0.350 | 0.350 | 3.806900 | .963 | .133 | 0.867 | 0.829 | 4.077100 | .213 | .067 | 0.933 | 0.146 |
| 2.546800 | 1.000 | .642 | 0.358 | 0.358 | 3.835850 | .963 | .125 | 0.875 | 0.838 | 4.091350 | .200 | .067 | 0.933 | 0.133 |
| 2.553800 | 1.000 | .633 | 0.367 | 0.367 | 3.874750 | .950 | .125 | 0.875 | 0.825 | 4.110800 | .188 | .067 | 0.933 | 0.121 |
| 2.559400 | 1.000 | .625 | 0.375 | 0.375 | 3.913650 | .950 | .117 | 0.883 | 0.833 | 4.127150 | .175 | .067 | 0.933 | 0.108 |
| 2.576300 | 1.000 | .617 | 0.383 | 0.383 | 3.945600 | .950 | .108 | 0.892 | 0.842 | 4.132050 | .175 | .058 | 0.942 | 0.117 |
| 2.613000 | 1.000 | .608 | 0.392 | 0.392 | 3.968400 | .950 | .100 | 0.900 | 0.850 | 4.139000 | .163 | .050 | 0.950 | 0.113 |
| 2.634650 | 1.000 | .600 | 0.400 | 0.400 | 3.982850 | .938 | .100 | 0.900 | 0.838 | 4.148700 | .150 | .050 | 0.950 | 0.100 |
| 2.647700 | 1.000 | .592 | 0.408 | 0.408 | 3.992250 | .925 | .100 | 0.900 | 0.825 | 4.154250 | .150 | .042 | 0.958 | 0.108 |
| 2.685250 | 1.000 | .583 | 0.417 | 0.417 | 3.994750 | .913 | .100 | 0.900 | 0.813 | 4.156950 | .138 | .042 | 0.958 | 0.096 |
| 2.714850 | 1.000 | .575 | 0.425 | 0.425 | 3.997250 | .900 | .100 | 0.900 | 0.800 | 4.177450 | .125 | .042 | 0.958 | 0.083 |
| 2.719650 | 1.000 | .567 | 0.433 | 0.433 | 3.998100 | .888 | .100 | 0.900 | 0.788 | 4.210250 | .113 | .042 | 0.958 | 0.071 |
| 2.732650 | 1.000 | .558 | 0.442 | 0.442 | 3.999400 | .875 | .100 | 0.900 | 0.775 | 4.233500 | .100 | .042 | 0.958 | 0.058 |
| 2.758350 | 1.000 | .550 | 0.450 | 0.450 | 4.002150 | .875 | .092 | 0.908 | 0.783 | 4.244150 | .088 | .042 | 0.958 | 0.046 |
| 2.779850 | 1.000 | .542 | 0.458 | 0.458 | 4.004200 | .863 | .092 | 0.908 | 0.771 | 4.247300 | .088 | .033 | 0.967 | 0.054 |
| 2.789450 | 1.000 | .533 | 0.467 | 0.467 | 4.005000 | .850 | .092 | 0.908 | 0.758 | 4.268100 | .075 | .033 | 0.967 | 0.042 |
| 2.791300 | 1.000 | .533 | 0.467 | 0.467 | 4.005700 | .838 | .092 | 0.908 | 0.746 | 4.298400 | .075 | .025 | 0.975 | 0.050 |
| 2.804650 | 1.000 | .517 | 0.483 | 0.483 | 4.006200 | .825 | .092 | 0.908 | 0.733 | 4.369000 | .075 | .017 | 0.983 | 0.058 |
| 2.824500 | 1.000 | .508 | 0.492 | 0.492 | 4.007700 | .813 | .092 | 0.908 | 0.721 | 4.492850 | .063 | .017 | 0.983 | 0.046 |
| 2.840250 | 1.000 | .500 | 0.500 | 0.500 | 4.009250 | .800 | .092 | 0.908 | 0.708 | 4.708050 | .050 | .017 | 0.983 | 0.033 |
| 2.849900 | 1.000 | .475 | 0.525 | 0.525 | 4.009600 | .800 | .092 | 0.908 | 0.708 | 4.899100 | .038 | .017 | 0.983 | 0.021 |
| 2.863100 | 1.000 | .467 | 0.533 | 0.533 | 4.009700 | .775 | .083 | 0.917 | 0.692 | 4.966100 | .038 | .008 | 0.992 | 0.029 |
| 2.879800 | 1.000 | .458 | 0.542 | 0.542 | 4.010200 | .763 | .083 | 0.917 | 0.679 | 5.176100 | .038 | 0.000 | 1.000 | 0.038 |
| 2.898900 | 1.000 | .450 | 0.550 | 0.550 | 4.010750 | .738 | .083 | 0.917 | 0.654 | 5.483750 | .025 | 0.000 | 1.000 | 0.025 |
| 2.916200 | 1.000 | .442 | 0.558 | 0.558 | 4.010900 | .738 | .083 | 0.917 | 0.654 | 5.837050 | .013 | 0.000 | 1.000 | 0.013 |
| 2.931450 | 1.000 | .433 | 0.567 | 0.567 | 4.013500 | .700 | .083 | 0.917 | 0.617 | 7.065500 | 0.000 | 0.000 | 1.000 | 0.000 |
Minimum cut-off score was the result of smallest observed test score subtracting 1; Maximum cut-off score was the result of largest observed test score adding 1. All other cut-off values were the average of 2 adjacent test scores.