Table 1.
Overview of prediction models for diagnosis and prognosis of covid-19
| Study; setting; and outcome | Predictors in final model | Sample size: total No of participants for model development set (No with outcome) | Predictive performance on validation | Overall risk of bias using PROBAST |
||
|---|---|---|---|---|---|---|
| Type of validation* | Sample size: total No of participants for model validation (No with outcome) | Performance* (C index, sensitivity (%), specificity (%), PPV/NPV (%), calibration slope, other (95% CI, if reported)) | ||||
| General population | ||||||
| Decaprio et al8; data from US general population; hospital admission for covid-19 pneumonia (proxy events)† | Age, sex, number of previous hospital admissions, 11 diagnostic features, interactions between age and diagnostic features | 1.5 million (unknown) | Training test split | 369 865 (unknown) | C index 0.73 | High |
| Decaprio et al8; data from US general population; hospital admission for covid-19 pneumonia (proxy events)† | Age and ≥500 features related to diagnosis history | 1.5 million (unknown) | Training test split | 369 865 (unknown) | C index 0.81 | High |
| Decaprio et al8; data from US general population; hospital admission for covid-19 pneumonia (proxy events)† | ≥500 undisclosed features, including age, diagnostic history, social determinants of health, Charlson comorbidity index | 1.5 million (unknown) | Training test split | 369 865 (unknown) | C index 0.81 | High |
| Diagnosis | ||||||
| Original review | ||||||
| Feng et al10; data from China, patients presenting at fever clinic; suspected covid-19 pneumonia | Age, temperature, heart rate, diastolic blood pressure, systolic blood pressure, basophil count, platelet count, mean corpuscular haemoglobin content, eosinophil count, monocyte count, fever, shiver, shortness of breath, headache, fatigue, sore throat, fever classification, interleukin 6 | 132 (26) | Temporal validation | 32 (unclear) | C index 0.94 | High |
| Lopez-Rincon et al35; data from international genome sequencing data repository, target population unclear; covid-19 diagnosis | Specific sequences of base pairs | 553 (66) | 10-fold cross validation | Not applicable | C index 0.98, sensitivity100, specificity 99 | High |
| Meng et al12; data from China, asymptomatic patients with suspected covid-19; covid-19 diagnosis | Age, activated partial thromboplastin time, red blood cell distribution width SD, uric acid, triglyceride, serum potassium, albumin/globulin, 3-hydroxybutyrate, serum calcium | 620 (302) | External validation | 145 (80) | C index 0.87‡ | High |
| Song et al30; data from China, inpatients with suspected covid-19; covid-19 diagnosis | Fever, history of close contact, signs of pneumonia on CT, neutrophil to lymphocyte ratio, highest body temperature, sex, age, meaningful respiratory syndromes | 304 (73) | Training test split | 95 (18) | C index 0.97 (0.93 to 1.00) | High |
| Yu et al24; data from China, paediatric inpatients with confirmed covid-19; severe disease (yes/no) defined based on clinical symptoms | Direct bilirubin; alanine transaminase | 105 (8) | Apparent performance only | Not applicable | F1 score 1.00 | High |
| Update 1 | ||||||
| Martin et al41; simulated patients with suspected covid-19; covid-19 diagnosis | Unknown | Not applicable | External validation only (simulation) | Not applicable | Sensitivity 97, specificity 96 | High |
| Sun et al40; data from Singapore, patients with suspected infection presenting at infectious disease clinic; covid-19 diagnosis | Age, sex, temperature, heart rate, systolic blood pressure, diastolic blood pressure, sore throat | 292 (49) | Leave-one-out cross validation | Not applicable | C index 0.65 (0.57 to 0.73) | High |
| Sun et al40; data from Singapore, patients with suspected infection presenting at infectious disease clinic; covid-19 diagnosis | Sex, temperature, heart rate, respiration rate, diastolic blood pressure, sore throat, sputum production, shortness of breath, gastrointestinal symptoms, lymphocytes, neutrophils, eosinophils, creatinine | 292 (49) | Leave-one-out cross validation | Not applicable | C index 0.88 (0.83 to 0.93) | High |
| Sun et al40; data from Singapore, patients with suspected infection presenting at infectious disease clinic; covid-19 diagnosis | Sex, temperature, heart rate, respiration rate, diastolic blood pressure, sputum production, gastrointestinal symptoms, chest radiograph or CT scan suggestive of pneumonia, neutrophils, eosinophils, creatinine | 292 (49) | Leave-one-out cross validation | Not applicable | C index 0.88 (0.83 to 0.93) | High |
| Sun et al40; data from Singapore, patients with suspected infection presenting at infectious disease clinic; covid-19 diagnosis | Sex, covid-19 case contact, travel to Wuhan, travel to China, temperature, heart rate, respiration rate, diastolic blood pressure, sore throat, sputum production, gastrointestinal symptoms, chest radiograph or CT scan suggestive of pneumonia, neutrophils, eosinophils, creatinine, sodium | 292 (49) | Leave-one-out cross validation | Not applicable | C index 0.91 (0.86 to 0.96) | High |
| Wang et43; data from China, patients with suspected covid-19; covid-19 pneumonia | Epidemiological history, wedge shaped or fan shaped lesion parallel to or near the pleura, bilateral lower lobes, ground glass opacities, crazy paving pattern, white blood cell count | 178 (69) | External validation | 116 (68) | C index 0.85, calibration slope 0.56 | High |
| Wu et al45; data from China, inpatients with suspected covid-19; covid-19 diagnosis | Lactate dehydrogenase, calcium, creatinine, total protein, total bilirubin, basophil, platelet distribution width, kalium, magnesium, creatinine kinase isoenzyme, glucose | 108 (12) | Training test split | 107 (61) | C index 0.99, sensitivity 100, specificity 94 | High |
| Zhou et al46; data from China, inpatients with confirmed covid-19; severe pneumonia | Age, sex, onset-admission time, high blood pressure, diabetes, CHD, COPD, white blood cell counts, lymphocyte, neutrophils, alanine transaminase, aspartate aminotransferase, serum albumin, serum creatinine, blood urea nitrogen, CRP | 250 (79) | Training test split | 127 (38) | C index 0.88 (0.94 to 0.92), sensitivity 89, specificity 74 | High |
| Diagnostic imaging | ||||||
| Original review | ||||||
| Barstugan et al31; data from Italy, patients with suspected covid-19; covid-19 diagnosis | Not applicable | 53 (not applicable) | Cross validation | Not applicable | Sensitivity 93, specificity 100 | High |
| Chen et al26; data from China, people with suspected covid-19 pneumonia; covid-19 pneumonia | Not applicable | 106 (51) | Training test split | 27 (11) | Sensitivity 100, specificity 82 | High |
| Gozes et al25; data from China and US,§ patients with suspected covid-19; covid-19 diagnosis | Not applicable | 50 (unknown) | External validation with Chinese cases and US controls | Unclear | C index 0.996 (0.989 to 1.000) | High |
| Jin et al11; data from China, US, and Switzerland,¶ patients with suspected covid-19; covid-19 diagnosis | Not applicable | 416 (196) | Training test split | 1255 (183) | C index 0.98, sensitivity 94, specificity 95 | High |
| Jin et al33; data from China, patients with suspected covid-19; covid-19 pneumonia | Not applicable | 1136 (723) | Training test split | 282 (154) | C index: 0.99, sensitivity 97, specificity 92 | High |
| Li et al34; data from China, patients with suspected covid-19; covid-19 diagnosis | Not applicable | 2969 (400) | Training test split | 353 (68) | C index 0.96 (0.94 to 0.99), sensitivity 90 (83 to 94), specificity 96 (93 to 98) | High |
| Shan et al28; data from China, people with confirmed covid-19; segmentation and quantification of infection regions in lung from chest CT scans | Not applicable | 249 (not applicable) | Training test split | 300 (not applicable) | Dice similarity coefficient 91.6%** | High |
| Shi et al36; data from China, target population unclear; covid-19 pneumonia | 5 categories of location features from imaging: volume, number, histogram, surface, radiomics | 2685 (1658) | Fivefold cross validation | Not applicable | C index 0.94 | High |
| Wang et al29; data from China, target population unclear; covid-19 diagnosis | Not applicable | 259 (79) | Internal, other images from same people | Not applicable | C index 0.81 (0.71 to 0.84), sensitivity 83, specificity 67 | High |
| Xu et al27; data from China, target population unclear; covid-19 diagnosis | Not applicable | 509 (110) | Training test split | 90 (30) | Sensitivity 87, PPV 81 | High |
| Song et al23; data from China, target population unclear; diagnosis of covid-19 v healthy controls | Not applicable | 123 (61) | Training test split | 51 (27) | C index 0.99 | High |
| Song et al23; data from China, target population unclear; diagnosis of covid-19 v bacterial pneumonia | Not applicable | 131 (61) | Training test split | 57 (27) | C index 0.96 | High |
| Zheng et al38; data from China, target population unclear; covid-19 diagnosis | Not applicable | Unknown | Temporal validation | Unknown | C index 0.96 | High |
| Update 1 | ||||||
| Abbas et al47; data from repositories (origin unspecified), target population unclear; covid-19 diagnosis | Not applicable | 137 (unknown) | Training test split | 59 (unknown) | C index 0.94, sensitivity 98, specificity 92 | High |
| Apostolopoulos et al48; data from repositories (US, Italy); patients with suspected covid-19; covid-19 diagnosis | Not applicable | 1427 (224) | 10-fold cross validation | Not applicable | Sensitivity 99, specificity 97 | High |
| Bukhari et al49; data from Canada and US; patients with suspected covid-19; covid-19 diagnosis | Not applicable | 223 (unknown) | Training test split | 61 (17) | Sensitivity 98, PPV 91 | High |
| Chaganti et al50; data from Canada, US, and European countries; patients with suspected covid-19; percentage lung opacity | Not applicable | 631 (not applicable) | Training test split | 100 (not applicable) | Correlation§§ 0.98 | High |
| Chaganti et al50; data from Canada, US, and European countries; patients with suspected covid-19; percentage high lung opacity | Not applicable | 631 (not applicable) | Training test split | 100 (not applicable) | Correlation§§ 0.98 | High |
| Chaganti et al50; data from Canada, US, and European countries; patients with suspected covid-19; severity score | Not applicable | 631 (not applicable) | Training test split | 100 (not applicable) | Correlation§§ 0.97 | High |
| Chaganti et al50; data from Canada, US, and European countries; patients with suspected covid-19; lung opacity score | Not applicable | 631 (not applicable) | Training test split | 100 (not applicable) | Correlation§§ 0.97 | High |
| Chowdhury et al39; data from repositories (Italy and other unspecified countries), target population unclear; covid-19 v “normal” | Not applicable | Unknown | Fivefold cross validation | Not applicable | C index 0.99 | High |
| Chowdhury et al39; data from repositories (Italy and other unspecified countries), target population unclear; covid-19 v “normal” and viral pneumonia | Not applicable | Unknown | Fivefold cross validation | Not applicable | C index 0.98 | High |
| Chowdhury et al39; data from repositories (Italy and other unspecified countries), target population unclear; covid-19 v “normal” | Not applicable | Unknown | Fivefold cross validation | Not applicable | C index 0.998 | High |
| Chowdhury et al39; data from repositories (Italy and other unspecified countries), target population unclear; covid-19 v “normal” and viral pneumonia | Not applicable | Unknown | Fivefold cross validation | Not applicable | C index 0.99 | High |
| Fu et al51; data from China, target population unclear; covid-19 diagnosis | Not applicable | 610 (100) | External validation | 309 (50) | C index 0.99, sensitivity 97, specificity 99 | High |
| Gozes et al52; data from China, people with suspected covid-19; covid-19 diagnosis | Not applicable | 50 (unknown) | External validation | 199 (109) | C index 0.95 (0.91 to 0.99) | High |
| Imran et al53; data from unspecified source, target population unclear; covid-19 diagnosis | Not applicable | 357 (48) | Twofold cross validation | Not applicable | Sensitivity 90, specificity 81 | High |
| Li et al54; data from China, inpatients with confirmed covid-19; severe and critical covid-19 | Severity score based on CT scans | Not applicable | External validation of existing score | 78 (not applicable) | C index 0.92 (0.84 to 0.99) | High |
| Li et al55; data from unknown origin, patients with suspected covid-19; covid-19 | Not applicable | 360 (120) | Training test split | 135 (45) | C index 0.97 | High |
| Hassanien et al56; data from repositories (origin unspecified), people with suspected covid-19; covid-19 diagnosis | Not applicable | Unknown | Training test split | Unknown | Sensitivity 95, specificity 100 | High |
| Tang et al57; data from China, patients with confirmed covid-19; covid-19 severe v non-severe | Not applicable | 176 (55) | Threefold cross validation | Not applicable | C index 0.91, sensitivity 93, specificity 75 | High |
| Wang et al42; data from China, inpatients with suspected covid-19; covid-19 | Not applicable | 709 (560) | External validation in other centres | 508 (223) | C index (average) 0.87 | High |
| Zhang et al58; data from repositories (origin unspecified), people with suspected covid-19; covid-19 | Not applicable | 1078 (70) | Twofold cross validation | Not applicable | C index 0.95, sensitivity 96, specificity 71 | High |
| Zhou et al59; data from China, patients with suspected covid-19; covid-19 diagnosis | Not applicable | 191 (35) | External validation in other centres | 107 (57) | C index 0.92, sensitivity 83, specificity 86 | High |
| Prognosis | ||||||
| Original review | ||||||
| Bai et al9; data from China, inpatients at admission with mild confirmed covid-19; deterioration into severe/critical disease (period unspecified) | Combination of demographics, signs and symptoms, laboratory results and features derived from CT images | 133 (54) | Unclear | Not applicable | C index 0.95 (0.94 to 0.97) | High |
| Caramelo et al18; data from China, target population unclear; mortality (period unspecified)†† | Age, sex, presence of any comorbidity (hypertension, diabetes, cardiovascular disease, chronic respiratory disease, cancer)†† | Unknown | Not reported | Not applicable | Not reported | High |
| Gong et al32; data from China, inpatients with confirmed covid-19 at admission; severe covid-19 (within minimum 15 days) | Age, serum LDH, CRP, variation of red blood cell distribution width, blood urea nitrogen, albumin, direct bilirubin | 189 (28) | External validation (two centres) | 165 (40) and 18 (4) | Centre 1: C index 0.85 (0.79 to 0.92), sensitivity 78, specificity 78; centre 2: sensitivity 75, specificity 100 | High |
| Lu et al19; data from China, inpatients at admission with suspected or confirmed covid-19; mortality (within 12 days) | Age, CRP | 577 (44) | Not reported | Not applicable | Not reported | High |
| Qi et al20; data from China, inpatients with confirmed covid-19 at admission; hospital stay >10 days | 6 features derived from CT images‡‡ (logistic regression model) | 26 (20) | 5 fold cross validation | Not applicable | C index 0.92 | High |
| Qi et al20; data from China, inpatients with confirmed covid-19 at admission; hospital stay >10 days | 6 features derived from CT images‡‡ (random forest) | 26 (20) | 5 fold cross validation | Not applicable | C index 0.96 | High |
| Shi et al37; data from China, inpatients with confirmed covid-19 at admission; death or severe covid-19 (period unspecified) | Age (dichotomised), sex, hypertension | 478 (49) | Validation in less severe cases | 66 (15) | Not reported | High |
| Xie et al7; data from China, inpatients with confirmed covid-19 at admission; mortality (in hospital) | Age, LDH, lymphocyte count, SPO2 | 299 (155) | External validation (other Chinese centre) | 130 (69) | C index 0.98 (0.96 to 1.00), calibration slope 2.5 (1.7 to 3.7) | High |
| Yan et al21; data from China, inpatients suspected of covid-19; mortality (period unspecified) | LDH, lymphocyte count, high sensitivity CRP | 375 (174) | Temporal validation, selecting only severe cases | 29 (17) | Sensitivity 92, PPV 95 | High |
| Yuan et al22; data from China, inpatients with confirmed covid-19 at admission; mortality (period unspecified) | Clinical scorings of CT images (zone, left/right, location, attenuation, distribution of affected parenchyma) | Not applicable | External validation of existing model | 27 (10) | C index 0.90 (0.87 to 0.93) | High |
| Update 1 | ||||||
| Huang et al60; data from China, inpatients with confirmed covid-19 at admission; severe symptoms three days after admission | Underlying diseases, fast respiratory rate >24/min, elevated CRP level (>10 mg/dL), elevated LDH level (>250 U/L) | 125 (32) | Apparent performance only | Not applicable | C index 0.99 (0.97 to 1.00), sensitivity 0.91, specificity 0.96 | High |
| Pourhomayoun et al61; data from 76 countries, inpatients with confirmed covid-19; in-hospital mortality (period unspecified) | Unknown | Unknown | 10-fold cross validation | Not applicable | C index 0.96, sensitivity 90, specificity 0.97 | High |
| Sarkar et al44; data from several continents (Australia, Asia, Europe, North America), inpatients with covid-19 symptoms; death v recovery (period unspecified) | Age, days from symptom onset to hospitalisation, from Wuhan, sex, visit to Wuhan | 80 (37) | Apparent performance only | Not applicable | C index 0.97 | High |
| Wang et al42; data from China, inpatients with confirmed covid-19; length of hospital stay | Age and CT features | 301 (not applicable) | Not reported | Not applicable | Not reported | High |
| Zeng et al62; data from China, inpatients with confirmed covid-19; severe disease progression (period unspecified) | CT features | 338 (76) | Cross validation (number of folds unclear) | Not applicable | C index 0.88 | High |
| Zeng et al62; data from China, inpatients with confirmed covid-19; severe disease progression (period unspecified) | CT features and laboratory markers | 338 (76) | Cross validation (number of folds unclear) | Not applicable | C index 0.88 | High |
CHD=coronary heart disease; COPD=chronic obstructive pulmonary disease; covid-19=coronavirus disease 2019; CRP=C reactive protein; CT=computed tomography; LDH=lactate dehydrogenase; NPV=negative predictive value; PPV=positive predictive value; PROBAST=prediction model risk of bias assessment tool; SPO2=oxygen saturation.
*
Performance is given for the strongest form of validation reported. This is indicated in the column “type of validation.” When a training test split was used, performance on the test set is reported. Apparent performance is the performance observed in the development data.
†
Proxy events used: pneumonia (except from tuberculosis), influenza, acute bronchitis, or other specified upper respiratory tract infections (no patients with covid-19 pneumonia in data).
‡
Calibration plot presented, but unclear which data were used.
§
The development set contains scans from Chinese patients, the testing set contains scans from Chinese cases and controls, and US controls.
¶
Data contain mixed cases and controls. Chinese data and controls from US and Switzerland.
**
Describes similarity between segmentation of the CT scan by a medical doctor and automated segmentation.
††
Outcome and predictor data were simulated.
‡‡
Wavelet-HLH_gldm_SmallDependenceLowGrayLevelEmphasis, wavelet-LHH_glcm_Correlation, wavelet-LHL_glszm_GrayLevelVariance, wavelet-LLH_glszm_SizeZoneNonUniformityNormalized, wavelet-LLH_glszm_SmallAreaEmphasis, wavelet-LLH_glcm_Correlation. §§Pearson correlation between the predicted and ground truth scores for patients with lung abnormalities.