Skip to main content
Journal of Personalized Medicine logoLink to Journal of Personalized Medicine
. 2021 Jun 3;11(6):501. doi: 10.3390/jpm11060501

Machine Learning to Predict In-Hospital Mortality in COVID-19 Patients Using Computed Tomography-Derived Pulmonary and Vascular Features

Simone Schiaffino 1,, Marina Codari 2,, Andrea Cozzi 3,*,, Domenico Albano 4,5, Marco Alì 6, Roberto Arioli 7, Emanuele Avola 8, Claudio Bnà 9, Maurizio Cariati 10, Serena Carriero 8, Massimo Cressoni 1, Pietro S C Danna 7, Gianmarco Della Pepa 8, Giovanni Di Leo 1, Francesco Dolci 11, Zeno Falaschi 7, Nicola Flor 12, Riccardo A Foà 10,13, Salvatore Gitto 3, Giovanni Leati 13, Veronica Magni 3, Alexis E Malavazos 14, Giovanni Mauri 15,16, Carmelo Messina 4, Lorenzo Monfardini 9, Alessio Paschè 7, Filippo Pesapane 17, Luca M Sconfienza 3,4, Francesco Secchi 1,3, Edoardo Segalini 18, Angelo Spinazzola 13, Valeria Tombini 19, Silvia Tresoldi 10, Angelo Vanzulli 15,19, Ilaria Vicentin 19, Domenico Zagaria 7, Dominik Fleischmann 2,20, Francesco Sardanelli 1,3
Editor: Jorge Luis Espinoza
PMCID: PMC8230339  PMID: 34204911

Abstract

Pulmonary parenchymal and vascular damage are frequently reported in COVID-19 patients and can be assessed with unenhanced chest computed tomography (CT), widely used as a triaging exam. Integrating clinical data, chest CT features, and CT-derived vascular metrics, we aimed to build a predictive model of in-hospital mortality using univariate analysis (Mann–Whitney U test) and machine learning models (support vectors machines (SVM) and multilayer perceptrons (MLP)). Patients with RT-PCR-confirmed SARS-CoV-2 infection and unenhanced chest CT performed on emergency department admission were included after retrieving their outcome (discharge or death), with an 85/15% training/test dataset split. Out of 897 patients, the 229 (26%) patients who died during hospitalization had higher median pulmonary artery diameter (29.0 mm) than patients who survived (27.0 mm, p < 0.001) and higher median ascending aortic diameter (36.6 mm versus 34.0 mm, p < 0.001). SVM and MLP best models considered the same ten input features, yielding a 0.747 (precision 0.522, recall 0.800) and 0.844 (precision 0.680, recall 0.567) area under the curve, respectively. In this model integrating clinical and radiological data, pulmonary artery diameter was the third most important predictor after age and parenchymal involvement extent, contributing to reliable in-hospital mortality prediction, highlighting the value of vascular metrics in improving patient stratification.

Keywords: COVID-19, lung, pulmonary artery, tomography, X-ray computed, machine learning, support vector machine, neural networks, computer, prognosis

1. Introduction

Since the inception of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, chest computed tomography (CT) has been widely used both as a triaging test on emergency department admission [1,2,3]—especially in the case of unavailability of immediate result of reverse transcriptase-polymerase chain reaction (RT-PCR)—and as a diagnostic tool to monitor Coronavirus Disease 2019 (COVID-19) pneumonia [1,2].

The widely recognized systemic prothrombotic profile of COVID-19, swiftly linked with worse outcomes [4,5,6,7,8,9,10,11], directly affects pulmonary vasculature [11,12,13,14,15,16,17,18,19,20]. In hospitalized COVID-19 patients with moderate and severe disease, CT-detected lung parenchymal involvement [21,22] has been associated with varying degrees of pulmonary arterial vascular damage [10,11,12,13,14,15], which disrupts vascular dynamics and increases pulmonary artery (PA) pressure [23,24,25,26,27,28,29]. Of note, bedside ultrasound examinations performed during hospitalization have hinted the prognostic value of indirect signs of impaired pulmonary circulation, such as right ventricular longitudinal strain and increased end-diastolic chamber size [29,30]. However, these parameters are encumbered by operator dependency and potential overlap with other pre-existing causes of right ventricular dysfunction [31]; moreover, the routine performance of echocardiography in the triage of COVID-19 patients during pandemic peaks could prove challenging, due to the high volume of patients referred to emergency departments.

Rather than identifying thrombotic phenomena as a byproduct of pulmonary embolism, ever-increasing evidence points to a direct origin of thrombosis in arterial lung vasculature [14,32,33], highlighting the need for a more accurate characterization of these phenomena in their true location [34].

Diagnosis of pulmonary arterial thrombosis requires contrast-enhanced CT angiography, which implies the intravenous administration of iodinated contrast agents [35]. However, alterations of pulmonary vascular metrics also detectable with unenhanced CT, such as the enlargement of the PA and an increased ratio between diameters of the PA and of the ascending aorta (AA), are known indirect signs of pulmonary hypertension caused by fibrotic or thromboembolic processes [36,37,38,39,40,41,42]. Notably, these pulmonary vascular metrics have been demonstrated to be altered also in COVID-19 patients—when compared to previous values measured on CT scans acquired before the SARS-CoV-2 pandemic—and to carry prognostic implications [25,26,27]. Since unenhanced chest CT is still widely performed in COVID-19 patients, the integration of CT-derived vascular features with other readily available clinical and imaging parameters could improve the stratification of COVID-19 patients on emergency department admission, potentially improving patient management and prognosis.

This multicenter study, conducted in six different hospitals in northern Italy during the first pandemic peak of 2020, aims to evaluate the prognostic power of a machine learning model integrating imaging features of lung parenchyma and vasculature with clinical data of COVID-19 patients routinely retrieved on emergency department admission.

2. Materials and Methods

2.1. Study Population

This multicenter observational study was conducted in six different institutions in northern Italy: Azienda Ospedaliero-Universitaria Maggiore della Carità, Novara (Center 1); ASST Grande Ospedale Metropolitano Niguarda, Milano (Center 2); Fondazione Poliambulanza Istituto Ospedaliero, Brescia (Center 3); ASST Crema—Ospedale Maggiore, Crema (Center 4); ASST Santi Paolo e Carlo, Milano (Center 5); and IRCCS Istituto Ortopedico Galeazzi, Milano (Center 6). Approval for this retrospective study was obtained from the Ethics Committee of the coordinating center IRCCS Policlinico San Donato (Comitato Etico IRCCS Ospedale San Raffaele, protocol code COVID19-TCretro, protocol number 77/INT/2020, approved 5 May 2020). Study-specific informed consent was waived due to the retrospective nature of the study.

We included patients hospitalized in each of these institutions during the study period (21 February to 30 April 2020) with RT-PCR-confirmed SARS-CoV-2 infection and unenhanced chest CT performed on emergency department admission. Authors from each center reviewed their own institutional electronic databases to retrieve patients’ clinical data, including: demographics, symptoms, partial pressure of oxygen in arterial blood (PaO2), comorbidities, smoking history, body mass index, and white blood cells, lymphocytes, platelets, and lactate dehydrogenase values. For outcome assessment, censoring was applied on 1 June 2020, when all patients had either been discharged or had died during hospitalization.

2.2. Image Acquisition and Analysis

Unenhanced chest CT scans were performed in all six centers in supine position, during a single inspiratory breath-hold condition when possible. Table 1 shows the technical characteristics of the scanners and acquisition parameters.

Table 1.

Center-specific technical characteristics of the CT scanners and acquisition parameters.

Center Location Vendor Model Acquired Slices Slice Thickness (mm) Tube Voltage (kVp)
Azienda Ospedaliero-Universitaria Maggiore della Carità Novara Philips Healthcare Ingenuity Core 128 1 120
ASST Grande Ospedale Metropolitano Niguarda Milano Siemens Healthineers Somatom Definition Edge 128 1.5 120
Fondazione Poliambulanza Istituto Ospedaliero Brescia General Electric Healthcare LightSpeed RT 16 16 1.25 120
ASST Crema—Ospedale Maggiore Crema Canon Aquilion CXL 64 1.5 135
General Electric Healthcare Revolution EVO 64 1.25 120
ASST Santi Paolo e Carlo Milano General Electric Healthcare LightSpeed RT 16 16 1.25 120
IRCCS Istituto Ortopedico Galeazzi Milano Siemens Healthineers Somatom Definition AS 64 64 1.5 120

Radiologists with 7 to 32 years of experience blindly reviewed CT images from their own institution, assessing lung parenchyma and the presence of pleural effusion and mediastinum lymph nodes with short-axis diameters larger than 1 cm [43]. Lung parenchymal involvement was qualitatively assessed by radiologists on their own institutional digital imaging and communications in medicine (DICOM) viewer, considering the presence of parenchymal involvement signs, (i.e., ground-glass opacities (GGOs), consolidations, and crazy paving pattern) and the volumetric extent of parenchymal involvement, assessed according to Chung et al. [21]: 0% (absent, 0); 1–25% (minimal, 1); 26–50% (mild, 2); 51–75% (moderate, 3); and over 75% (severe, 4). As described by Wells et al. [44], PA and AA maximum diameters were measured on a single axial slice selected at the level of pulmonary arterial main trunk bifurcation (Figure 1).

Figure 1.

Figure 1

Example of axial slice selected at the level of pulmonary arterial main trunk bifurcation. The solid and dashed lines identify the maximum diameters of the pulmonary artery and ascending aorta, respectively.

2.3. Data Preprocessing and Initial Feature Selection

To build a predictive model of in-hospital mortality, we selected among available features those with no more than 25% missing data. In these selected features, residual missing data were imputed with the mean value of the respective feature. Data were randomly split into a training/validation set (85%) and a testing set (15%). In each set, all features were rescaled to have null mean and unitary variance.

To prevent overfitting, we used training/validation data to perform a two-step feature selection process. First, we removed highly correlated features by thresholding the feature correlation matrix, setting the absolute Pearson’s correlation threshold value at 0.7. Then, we used the least absolute shrinkage and selection operator (LASSO) to compute feature importance, i.e., the absolute value of the LASSO regression coefficients [45,46]. The best α value was determined using a ten-fold cross-validation process. Selected features were used to develop several machine learning models using both support vector machines (SVM) and multilayer perceptrons (MLP).

2.4. Support Vector Machines

We initialized the SVM classifier with linear kernel, C value = 1, and balanced class-weights using all selected features. Then, we performed a systematic hyperparameter grid-search to find: the optimal SVM kernel among linear, radial basis function, polynomial, and sigmoid kernels; the best C values among 100 values linearly sampled in the range 1–8; and the best γ value (for nonlinear kernels) among ten logarithmically-sampled (base 10) values in the range 10−10–10−1. Each candidate model (n = 310) was validated using a ten-fold cross-validation process, accounting for 31,000 fits. The best model was defined as the one that maximized the average F1 score in validation data. F1 score was selected as a scoring metric to consider class imbalance. Finally, the best model was refitted on the entire training/validation set.

Given this initial model and its optimal C (Copt) and γ (γopt) hyperparameters, we performed an importance-based backward feature selection. We optimized SVM hyperparameters for each feature subset using a second systematic grid search strategy, further fine-tuning C and γ values in narrower ranges. The new search-grid was defined by selecting 25 C values linearly sampled in the range (Copt ± 2), while 50 γ values were logarithmically sampled in the range (10log(γopt1), 10log(γopt+1)) Each one of the 1250 candidate models was validated using a ten-fold cross-validation process. Again, the best model (i.e., the one that maximized the average F1 score in validation data) was finally refitted over the entire training/validation set.

2.5. Multilayer Perceptrons

After selecting all features with a non-zero LASSO coefficient, we further selected features by keeping the most important ones that allowed us to meet the rule of thumb of at least ten events in our training/validation set for each feature included in our model [47,48].

The general MLP architecture is summarized in Table 2. We initialized the hidden unit numbers N and M values to 7 and 5, respectively. To prevent dying rectified linear unit (ReLU) issues, we used LeakyReLU [49] as activation function (α = 0.03), while weights were initialized using the method proposed by He et al. [50]. To improve model generalizability and prevent overfitting, we added a dropout layer after each fully connected layer with a dropout rate equal to 0.2. The last layer of the perceptron comprised a single unit with a sigmoid activation function to encode patient mortality. The initial learning rate was set to 0.01, being then decreased during training using an exponential decay schedule of 1000 steps and a base of 0.5. We set binary cross-entropy as the loss function, adopted the Adam optimizer [51], and set a default batch-size value of 32. The F1 score, the area under the curve (AUC) at receiving operator characteristic (ROC) analysis, accuracy, precision, and recall were selected as the performance metrics. The training stopped if the loss function did not decrease for 25 epochs, then the best weights corresponding to the loss function minimum were restored. The maximum number of epochs was set to 1000.

Table 2.

Multilayer perceptron summary architecture.

Layer Number of Hidden Units Trainable Parameters
Dense_1 Number of selected features (f) f × (f + 1)
Dropout_1 Number of selected features (f) 0
Dense_2 N N × (f + 1)
Dropout_2 N 0
Dense_3 N N × (N + 1)
Dropout_3 N 0
Dense_4 M M × (N + 1)
Dropout_4 M 0
Dense_5 M M × (M + 1)
Dropout_5 M 0
Dense_6 1 M + 1

N: number of hidden units in the second and third layer of the multilayer perceptron; M: number of hidden units in the fourth and fifth layer of the multilayer perceptron.

Given this model and its hyperparameters, we performed a systematic grid-search ten-fold cross-validation to optimize the following hyperparameters: the batch size in (4, 16, 32, 64); the dropout rate in (0.1, 0.2, 0.4); the starting learning rate in (0.1, 0.01, 0.001); the number of hidden units in the second and third layers (N) in (5, 7, 10); and the number of hidden units in the fourth and fifth layers (M) in (3, 5, 7). Each one of the 324 candidate MLPs was validated using a ten-fold cross-validation process (3240 fits). The best model (i.e., the one that maximized the average F1 score in validation data) was finally refitted over the entire training/validation set.

2.6. Statistical Analysis

Due to their nonparametric distribution, assessed with the Shapiro–Wilk test, continuous variables were reported as median with interquartile range (IQR), while categorical variables as total number and percentage. The Mann–Whitney U test was used to compare means from different groups for continuous variables during univariate explorative analysis. Statistical analyses were performed using SPSS v.26.0 (IBM SPSS Inc., Chicago, IL, USA), and p values < 0.05 were considered statistically significant. All SVMs and MLPs were developed using python v.3.7.7 [52], sklearn v.0.23.1 [53], and tensorflow-gpu v.2.2.0 [54] on a laptop with an Intel Core i7-6500 CPU (2.50 GHz), 8 GB of RAM, and a Nvidia GeForce 940MX GPU.

3. Results

3.1. Population Characteristics

The study population included 897 COVID-19 patients (608 males, 68%; median age 66 years, IQR 55–77 years), hospitalized in six different centers from 21 February to 30 April 2020. A total of 270 patients came from Center 1, 197 from Center 2, 194 from Center 3, 144 from Center 4, 80 from Center 5, and 12 from Center 6.

Symptoms, comorbidities, height, weight, and laboratory parameters were assessed on emergency department admission and are reported in Table 3, alongside CT-derived lung and vascular features. The main symptoms observed on emergency department admission were fever (500/897 patients, 56%), cough (277/897 patients, 31%), and dyspnea (229/897 patients, 26%). During hospitalization, which lasted a median of 7 days (IQR 4–13 days), 133/897 patients (15%) were admitted to intensive care units (ICU). Low molecular weight heparin (LMWH) was administered to 296/897 patients (33%): to 88/296 (30%) at therapeutic dosage, to 177/296 (60%) at prophylactic dosage, while for the remaining 31/296 (10%) dosage data was not available. At censoring for outcome assessment (June 1, 2020), we found that 229/897 (26%) patients had died during hospitalization, while the remaining 668/897 (74%) had been discharged.

Table 3.

Demographic, clinical, and imaging characteristics of the two sets of patients.

Variable Variable Type Overall
(897 Patients)
Training/Validation Set
(572 Patients)
Test Set
(102 Patients)
Demographics
Sex Categorial 608 M/289 F 389 M/183 F 66 M/36 F
Age (years) Continuous 66.2 (55.1–76.5) 66.9 (55.9–77.0) 66.7 (52.7–79.2)
Comorbidities
Cardiovascular diseases Dichotomic 433 (48%) 320 (56%) 53 (52%)
Diabetes Dichotomic 151 (17%) 113 (20%) 19 (19%)
Oncological history Dichotomic 76 (8%) 52 (9%) 10 (10%)
Chronic kidney insufficiency Dichotomic 52 (6%) 45 (8%) 3 (3%)
Outcome
Deceased patients Dichotomic 229 (26%) 160 (28%) 30 (29%)
CT findings and features
Lung parenchyma Ground-glass opacities Dichotomic 681 (76%) 504 (93%) 90 (92%)
Consolidations Dichotomic 434 (48%) 271 (50%) 53 (54%)
Crazy paving pattern Dichotomic 194 (22%) 162 (30%) 36 (37%)
Extent of parenchymal involvement * Discrete 2 (1–3) 2 (1–3) 2 (1–3)
Bilateral parenchymal involvement Dichotomic 631 (70%) 502 (93%) 86 (88%)
Vascular features PA diameter (mm) Continuous 28.0 (25.0–30.0) 28.0 (25.0–30.1) 28.0 (25.0–30.3)
AA diameter (mm) Continuous 34.0 (32.0–37.0) 35.0 (32.0–37.0) 34.0 (31.0–37.4)
PA/AA ratio Continuous 0.81 (0.73–0.89) 0.81 (0.73–0.90) 0.81 (0.74–0.89)

All descriptive statistics are reported as median and interquartile ranges or frequencies for continuous and dichotomic/discrete variables, respectively. M: males; F: females; PA: pulmonary artery; AA: ascending aorta. * Semi-quantitatively assessed from 0 to 4, according to Chung et al. [21], as follows: 0% = 0 (absent); 1–25% = 1 (minimal); 26–50% = 2 (mild); 51–75% = 3 (moderate); and over 75% = 4 (severe).

3.2. Explorative Univariate Analysis of Pulmonary Vascular Features

The overall median PA maximum diameter was 28 mm (25–30 mm) and median AA 34 mm (32–37 mm). Patients who died during hospitalization showed significantly higher median PA maximum diameter (29.0 mm, IQR 26.0–32.0 mm) compared to patients who survived (27.0 mm, IQR 25.0–30.0 mm, p < 0.001), and significantly higher median AA maximum diameter (36.6 mm, IQR 34.0–39.0 mm, versus 34.0 mm, IQR 31.0–36.0 mm, p < 0.001).

3.3. Support Vector Machines and Multilayer Perceptrons

After discarding features with a percentage of missing data higher than 25%, 14 features were selected for further processing (Table 3). After this selection step, residual missing data were clustered in 223 patients, who had almost only input data and were therefore also removed. Only 674 out of the initial 897 patients (75%) were therefore used to develop all machine learning models. The percentage of missing data ranged from 0% to 5% in the remaining features (median 0.4%, IQR 0.1–3.2%). After the initial random database split, our training/validation dataset was composed of 572 patients, 160 of whom (28%) died during hospitalization, while the testing set accounted for 102 patients, 30 of whom died during hospitalization (29%). No features were excluded after Pearson’s correlation matrix thresholding (Figure 2). Obtained LASSO coefficients for all initial features are reported in Figure 3.

Figure 2.

Figure 2

Feature correlation matrix, the color gradient representing the Pearson’s correlation coefficient value. GGOs: ground glass opacities; PA: pulmonary artery; AA: ascending aorta.

Figure 3.

Figure 3

Coefficient of the least absolute shrinkage and selection operator (LASSO) regression in training/validation data. PA: pulmonary artery; AA: ascending aorta; GGOs: ground glass opacities.

Fourteen SVM models were developed using different feature subsets. The best SVM model took 10 features as inputs (Table S1). The best model had a radial basis function kernel, with C equal to 5.924 and γ equal to 6.06 × 10−5. After the model was refitted on the entire training/validation dataset, the final model performances on testing data were: an F1 score of 0.632, an AUC of 0.747, a precision of 0.522, and a recall of 0.800. All SVMs had radial basis function kernels, their other hyperparameters and performance being reported in Table S1, while the ROC curve of the best SVM model is depicted in panel a of Figure 4.

Figure 4.

Figure 4

Receiver operator characteristic curves of the best support vector machine (a) and multilayer perceptron (b) models.

Due to the high number of events in the training/validation set, MLPs were built using all features with a non-null LASSO coefficient (n = 10). Therefore, all developed MLPs have the same input features as the best SVM model. Among the 324 models resulting from the systematic hyperparameter grid search, the best MLP had the following hyperparameters: N and M equal to 7 and 3 respectively; a batch-size of 4; a dropout rate equal to 0.1; and a starting learning rate equal to 0.01. After refitting the best model on the entire training/validation dataset, the final performance on testing data were: an F1 score of 0.618, an AUC of 0.844, a precision of 0.680, and a recall of 0.567. The ROC curve of the best MLP is depicted in panel b of Figure 4.

4. Discussion

In this retrospective multicenter study, we developed a predictive model of in-hospital mortality for COVID-19 patients, using clinical and radiological data acquired on emergency department admission. The CT data of 674 patients from six hospitals in northern Italy were used to extract pulmonary parenchymal and vascular features. While machine learning models centered on clinical and imaging data have been proposed to aid both COVID-19 diagnosis and severity stratification [55,56,57,58,59], the inclusion of vascular features stems from an ever-larger corpus of observations that link vascular (particularly endothelial) impairment in COVID-19 [4,5,6,7,8,9] to pulmonary thromboembolism [11,12,13,16,17,18,19,20,23] and gross damage to pulmonary arterial vessels [25,26,27].

In a preliminary study on a limited sample size [26] we already observed that COVID-19 patients have a PA maximum diameter greater than that measured in previous CTs performed for non-cardiovascular reasons. In that study, enlarged PA diameter was also associated to death during hospitalization [26]. These observations were confirmed by the present study already through the explorative univariate analysis of pulmonary vascular features, where we found that patients who died during hospitalization had a significantly higher PA maximum diameter. Further, as shown in Figure 3, the LASSO regression coefficient for the PA maximum diameter was the third highest among investigated predictors of in-hospital death, after age and the extent of lung parenchymal involvement. It is worth noting that lung consolidation, appearing in the later stages of COVID-19 pneumonia, seems to be associated with better prognosis, as highlighted by the negative LASSO regression coefficient. This apparently counterintuitive result, already hinted at by a few studies [60,61], can be explained by considering that extensive parenchymal involvement in COVID-19 patients most frequently leads to in-hospital death, leaving no time for pulmonary damage progression to consolidation. Indeed, consolidation was present in almost 50% of patients from our cohort but was sparsely distributed and was associated with GGOs in 91% of cases, pointing out how hospital admission occurred at relatively early stages of COVID-19 pneumonia.

The prognostic role of the PA maximum diameter also shows how the early microvascular damage caused by SARS-CoV-2 infection may represent one of the most insidious aspects of severe COVID-19 presentations [10]. Notably, SARS-CoV-2-induced microthrombosis in distal branches of the PA can start the disruption of the already fragile coagulation profile of these patients [5,8,9], but can be difficult to detect even on contrast-enhanced CT scans [23]. Conversely, the PA diameter is an indirect metric of the thromboembolic profile of COVID-19 patients that can be easily obtained from unenhanced CT scans. Its integration into a reliable prognostic model further highlights how features extracted from CT scans performed without the administration of a contrast agent carry a substantial prognostic potential in COVID-19 patients, as already hinted at by other studies [62,63].

This study was performed at the early stage of the SARS-CoV-2 pandemic, when prophylactic LMWH was not yet routinely administered. This represents the main limitation of our study, preventing the investigation of the impact of appropriate treatment—targeting vascular damage—on COVID-19 patients’ prognosis. Moreover, shortages in ICU beds affecting all hospitals in northern Italy during the first pandemic peak [64] frequently restricted ICU admission to relatively younger patients without comorbidities, preventing us from assessing ICU admission predictive models, since these would have been biased by such patient selection. Of note, this logistical stress on healthcare systems also precluded the inclusion of young patients (which were prevalently treated at home), hindered data collection on admission, and likely impacted the prognosis of the included patients: all these aspects must be carefully considered when evaluating the generalizability of our results. From a technical point of view, the main limitation of our study is the lack of a longitudinal and independent test set, as we randomly sampled test patients from the original multicenter cohort. Since we aimed first to maximize the heterogeneity of the training set as much as possible, using retrospective data collected from all centers, a robust generalizability assessment will need to be conducted prospectively on an independent test set. Although commonly reported in published literature [25,37,38,39,44], another technical limitation is represented by the manual measurement of the diameters of great vessels on axial scans: a better approach could be represented by acquiring vessel area measurements on reformatted oblique reconstructions considering the vessel axis [42]. Moreover, we focused on a single feature selection technique (i.e., the LASSO) and on few machine learning models (i.e., SVMs and MLPs) to develop the in-hospital mortality predictor, with a comparably small batch size in MLPs, although supported by results of systematic hyperparameter search. Finally, the use of the mean feature value as an imputation technique may also lead to suboptimal performance. Indeed, the use of multiple imputations or machine learning based imputation techniques could improve the predictive power of developed classifiers.

In conclusion, our study shows how a predictive model integrating simple clinical and radiological data acquired on admission, such as the PA/AA ratio on unenhanced CT scans, allowed to predict in-hospital death in COVID-19 patients, highlighting the major role of pulmonary vascular involvement in the stratification of COVID-19 patients.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jpm11060501/s1, Table S1: Input features, hyperparameters, and performance of the support vector machines developed as a result of the LASSO-importance-based backward feature selection.

Author Contributions

Conceptualization, F.S. (Francesco Sardanelli), S.S., M.C. (Marina Codari), and A.C.; methodology, S.S., A.C., M.C. (Marina Codari), G.D.L., and F.S. (Francesco Sardanelli); formal analysis, M.C. (Marina Codari), A.C., and S.S.; investigation, S.S., M.C. (Marina Codari), A.C., D.A., M.A., R.A., E.A., C.B., M.C. (Maurizio Cariati), S.C., M.C. (Massimo Cressoni), P.S.C.D., G.D.P., G.D.L., F.D., Z.F., N.F., R.A.F., S.G., G.L., V.M., A.E.M., G.M., C.M., L.M., A.P., F.P., L.M.S., F.S. (Francesco Secchi), E.S., A.S., V.T., S.T., A.V., I.V., D.Z., D.F., and F.S. (Francesco Sardanelli); resources, C.B., M.C. (Maurizio Cariati), L.M.S., A.S., A.V., D.F., and F.S. (Francesco Sardanelli); data curation, S.S., M.C. (Marina Codari), A.C., D.A., M.A., R.A., E.A., S.C., M.C. (Massimo Cressoni), P.S.C.D., G.D.P., G.D.L., F.D., Z.F., N.F., R.A.F., S.G., G.L., V.M., A.E.M., G.M., C.M., L.M., A.P., F.P., F.S. (Francesco Secchi), E.S., S.T., and D.Z.; writing—original draft preparation, S.S., M.C. (Marina Codari), and A.C.; writing—review and editing, S.S., M.C. (Marina Codari), A.C., V.M., D.F., M.A., M.C. (Massimo Cressoni), N.F., F.P., L.M.S., F.S. (Francesco Secchi), and F.S. (Francesco Sardanelli); visualization, M.C. (Marina Codari) and A.C.; supervision, F.S. (Francesco Sardanelli), D.F., C.B., M.C. (Maurizio Cariati), L.M.S., A.S., and A.V.; project administration, A.C.; funding acquisition, F.S. (Francesco Secchi). All authors have read and agreed to the published version of the manuscript.

Funding

This study was partially supported by Ricerca Corrente funding from Italian Ministry of Health to IRCCS Policlinico San Donato.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of IRCCS Policlinico San Donato (Comitato Etico IRCCS Ospedale San Raffaele, protocol code COVID19-TCretro, protocol number 77/INT/2020, approved 5 May 2020).

Informed Consent Statement

Patient consent was waived due to the retrospective nature of the study.

Data Availability Statement

We will be able to make available deidentified individual participant data that underlie the results reported in this article, beginning 12 months and ending 24 months following article publication. We will consider requests from researchers who will provide a methodologically sound proposal and will share data needed to achieve the aims declared in the approved proposal. Proposals may be submitted up to 36 months following article publication. After 24 months the data will be available in the data warehouse of the coordinating center but without investigator support other than deposited metadata. Information regarding submitting proposals and accessing data may be obtained by email inquiry to the first/corresponding author at andrea.cozzi1@unimi.it.

Conflicts of Interest

S. Schiaffino declares to have received travel support from Bracco Imaging and to be member of speakers’ bureau for General Electric Healthcare. D. Fleischmann declares to have received a research grant from Siemens AG, to be a shareholder of Segmed, Inc., and to have ownership interests in Ischemia View, Inc. F. Sardanelli declares to have received grants from or to be a member of the speakers’ bureau/advisory board for Bayer Healthcare, Bracco Group, and General Electric Healthcare. All other authors declare that they have no conflicts of interest and that they have nothing to disclose.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Akl E.A., Blažić I., Yaacoub S., Frija G., Chou R., Appiah J.A., Fatehi M., Flor N., Hitti E., Jafri H., et al. Use of Chest Imaging in the Diagnosis and Management of COVID-19: A WHO Rapid Advice Guide. Radiology. 2021;298:E63–E69. doi: 10.1148/radiol.2020203173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Rubin G.D., Ryerson C.J., Haramati L.B., Sverzellati N., Kanne J.P., Raoof S., Schluger N.W., Volpi A., Yim J.-J., Martin I.B.K., et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology. 2020;296:172–180. doi: 10.1148/radiol.2020201365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Sverzellati N., Milanese G., Milone F., Balbi M., Ledda R.E., Silva M. Integrated Radiologic Algorithm for COVID-19 Pandemic. J. Thorac. Imaging. 2020;35:228–233. doi: 10.1097/RTI.0000000000000516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Tang N., Li D., Wang X., Sun Z. Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia. J. Thromb. Haemost. 2020;18:844–847. doi: 10.1111/jth.14768. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Nadkarni G.N., Lala A., Bagiella E., Chang H.L., Moreno P.R., Pujadas E., Arvind V., Bose S., Charney A.W., Chen M.D., et al. Anticoagulation, Bleeding, Mortality, and Pathology in Hospitalized Patients with COVID-19. J. Am. Coll. Cardiol. 2020;76:1815–1826. doi: 10.1016/j.jacc.2020.08.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Fogarty H., Townsend L., Ni Cheallaigh C., Bergin C., Martin-Loeches I., Browne P., Bacon C.L., Gaule R., Gillett A., Byrne M., et al. COVID-19 coagulopathy in Caucasian patients. Br. J. Haematol. 2020;189:1044–1049. doi: 10.1111/bjh.16749. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Fauvel C., Weizman O., Trimaille A., Mika D., Pommier T., Pace N., Douair A., Barbin E., Fraix A., Bouchot O., et al. Pulmonary embolism in COVID-19 patients: A French multicentre cohort study. Eur. Heart J. 2020;41:3058–3068. doi: 10.1093/eurheartj/ehaa500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Masi P., Hékimian G., Lejeune M., Chommeloux J., Desnos C., Pineton De Chambrun M., Martin-Toutain I., Nieszkowska A., Lebreton G., Bréchot N., et al. Systemic Inflammatory Response Syndrome Is a Major Contributor to COVID-19–Associated Coagulopathy. Circulation. 2020;142:611–614. doi: 10.1161/CIRCULATIONAHA.120.048925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Gu S.X., Tyagi T., Jain K., Gu V.W., Lee S.H., Hwa J.M., Kwan J.M., Krause D.S., Lee A.I., Halene S., et al. Thrombocytopathy and endotheliopathy: Crucial contributors to COVID-19 thromboinflammation. Nat. Rev. Cardiol. 2021;18:194–209. doi: 10.1038/s41569-020-00469-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Teuwen L.-A., Geldhof V., Pasut A., Carmeliet P. COVID-19: The vasculature unleashed. Nat. Rev. Immunol. 2020;20:389–391. doi: 10.1038/s41577-020-0343-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Lowenstein C.J., Solomon S.D. Severe COVID-19 Is a Microvascular Disease. Circulation. 2020;142:1609–1611. doi: 10.1161/CIRCULATIONAHA.120.050354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Varga Z., Flammer A.J., Steiger P., Haberecker M., Andermatt R., Zinkernagel A.S., Mehra M.R., Schuepbach R.A., Ruschitzka F., Moch H. Endothelial cell infection and endotheliitis in COVID-19. Lancet. 2020;395:1417–1418. doi: 10.1016/S0140-6736(20)30937-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Ackermann M., Verleden S.E., Kuehnel M., Haverich A., Welte T., Laenger F., Vanstapel A., Werlein C., Stark H., Tzankov A., et al. Pulmonary Vascular Endothelialitis, Thrombosis, and Angiogenesis in Covid-19. N. Engl. J. Med. 2020;383:120–128. doi: 10.1056/NEJMoa2015432. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Lax S.F., Skok K., Zechner P., Kessler H.H., Kaufmann N., Koelblinger C., Vander K., Bargfrieder U., Trauner M. Pulmonary Arterial Thrombosis in COVID-19 with Fatal Outcome. Ann. Intern. Med. 2020;173:350–361. doi: 10.7326/M20-2566. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Carsana L., Sonzogni A., Nasr A., Rossi R.S., Pellegrinelli A., Zerbi P., Rech R., Colombo R., Antinori S., Corbellino M., et al. Pulmonary post-mortem findings in a series of COVID-19 cases from northern Italy: A two-centre descriptive study. Lancet Infect. Dis. 2020;20:1135–1140. doi: 10.1016/S1473-3099(20)30434-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Léonard-Lorant I., Delabranche X., Séverac F., Helms J., Pauzet C., Collange O., Schneider F.F., Labani A., Bilbault P., Molière S., et al. Acute Pulmonary Embolism in Patients with COVID-19 at CT Angiography and Relationship to d-Dimer Levels. Radiology. 2020;296:E189–E191. doi: 10.1148/radiol.2020201561. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Grillet F., Behr J., Calame P., Aubry S., Delabrousse E. Acute Pulmonary Embolism Associated with COVID-19 Pneumonia Detected with Pulmonary CT Angiography. Radiology. 2020;296:E186–E188. doi: 10.1148/radiol.2020201544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Monfardini L., Morassi M., Botti P., Stellini R., Bettari L., Pezzotti S., Alì M., Monaco C.G., Magni V., Cozzi A., et al. Pulmonary thromboembolism in hospitalised COVID-19 patients at moderate to high risk by Wells score: A report from Lombardy, Italy. Br. J. Radiol. 2020;93:20200407. doi: 10.1259/bjr.20200407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Schiaffino S., Giacomazzi F., Esseridou A., Cozzi A., Carriero S., Mazzaccaro D.P., Nano G., Di Leo G., Spagnolo P., Sardanelli F. Pulmonary thromboembolism in coronavirus disease 2019 patients undergoing thromboprophylaxis. Medicine. 2021;100:e24002. doi: 10.1097/MD.0000000000024002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.De Cobelli F., Palumbo D., Ciceri F., Landoni G., Ruggeri A., Rovere-Querini P., D’Angelo A., Steidler S., Galli L., Poli A., et al. Pulmonary Vascular Thrombosis in COVID-19 Pneumonia. J. Cardiothorac. Vasc. Anesth. 2021 doi: 10.1053/j.jvca.2021.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Chung M., Bernheim A., Mei X., Zhang N., Huang M., Zeng X., Cui J., Xu W., Yang Y., Fayad Z.A., et al. CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) Radiology. 2020;295:202–207. doi: 10.1148/radiol.2020200230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Bernheim A., Mei X., Huang M., Yang Y., Fayad Z.A., Zhang N., Diao K., Lin B., Zhu X., Li K., et al. Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection. Radiology. 2020;295:685–691. doi: 10.1148/radiol.2020200463. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Grillet F., Busse-Coté A., Calame P., Behr J., Delabrousse E., Aubry S. COVID-19 pneumonia: Microvascular disease revealed on pulmonary dual-energy computed tomography angiography. Quant. Imaging Med. Surg. 2020;10:1852–1862. doi: 10.21037/qims-20-708. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lins M., Vandevenne J., Thillai M., Lavon B.R., Lanclus M., Bonte S., Godon R., Kendall I., De Backer J., De Backer W. Assessment of Small Pulmonary Blood Vessels in COVID-19 Patients Using HRCT. Acad. Radiol. 2020;27:1449–1455. doi: 10.1016/j.acra.2020.07.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Esposito A., Palmisano A., Toselli M., Vignale D., Cereda A., Rancoita P.M.V., Leone R., Nicoletti V., Gnasso C., Monello A., et al. Chest CT-derived pulmonary artery enlargement at the admission predicts overall survival in COVID-19 patients: Insight from 1461 consecutive patients in Italy. Eur. Radiol. 2021;31:4031–4041. doi: 10.1007/s00330-020-07622-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Spagnolo P., Cozzi A., Foà R.A., Spinazzola A., Monfardini L., Bnà C., Alì M., Schiaffino S., Sardanelli F. CT-derived pulmonary vascular metrics and clinical outcome in COVID-19 patients. Quant. Imaging Med. Surg. 2020;10:1325–1333. doi: 10.21037/qims-20-546. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Aoki R., Iwasawa T., Hagiwara E., Komatsu S., Utsunomiya D., Ogura T. Pulmonary vascular enlargement and lesion extent on computed tomography are correlated with COVID-19 disease severity. Jpn. J. Radiol. 2021;39:451–458. doi: 10.1007/s11604-020-01085-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Eslami V., Abrishami A., Zarei E., Khalili N., Baharvand Z., Sanei-Taheri M. The Association of CT-measured Cardiac Indices with Lung Involvement and Clinical Outcome in Patients with COVID-19. Acad. Radiol. 2021;28:8–17. doi: 10.1016/j.acra.2020.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.D’Andrea A., Scarafile R., Riegler L., Liccardo B., Crescibene F., Cocchia R., Bossone E. Right Ventricular Function and Pulmonary Pressures as Independent Predictors of Survival in Patients with COVID-19 Pneumonia. JACC Cardiovasc. Imaging. 2020;13:2467–2468. doi: 10.1016/j.jcmg.2020.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Li Y., Li H., Zhu S., Xie Y., Wang B., He L., Zhang D., Zhang Y., Yuan H., Wu C., et al. Prognostic Value of Right Ventricular Longitudinal Strain in Patients With COVID-19. JACC Cardiovasc. Imaging. 2020;13:2287–2299. doi: 10.1016/j.jcmg.2020.04.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Fukui M., Cavalcante J.L. Relation among Right Ventricular Dysfunction, Lung Damage, and Mortality in Patients With COVID-19. JACC Cardiovasc. Imaging. 2020;13:1858–1859. doi: 10.1016/j.jcmg.2020.05.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Saba L., Sverzellati N. Is COVID Evolution Due to Occurrence of Pulmonary Vascular Thrombosis? J. Thorac. Imaging. 2020;35:344–345. doi: 10.1097/RTI.0000000000000530. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Wichmann D., Sperhake J.-P., Lütgehetmann M., Steurer S., Edler C., Heinemann A., Heinrich F., Mushumba H., Kniep I., Schröder A.S., et al. Autopsy Findings and Venous Thromboembolism in Patients With COVID-19. Ann. Intern. Med. 2020;173:268–277. doi: 10.7326/M20-2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Zhou H., Zhang G., Deng X., Jin B., Qiu Q., Yan M., Wang X., Zheng X. Understanding the current status of patients with pulmonary hypertension during COVID-19 outbreak: A small-scale national survey from China. Pulm. Circ. 2020;10 doi: 10.1177/2045894020924566. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Rouhezamin M.R., Haseli S. Diagnosing Pulmonary Thromboembolism in COVID-19: A Stepwise Clinical and Imaging Approach. Acad. Radiol. 2020;27:896–897. doi: 10.1016/j.acra.2020.04.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Matsushita S., Matsuoka S., Yamashiro T., Fujikawa A., Yagihashi K., Kurihara Y., Nakajima Y. Pulmonary arterial enlargement in patients with acute exacerbation of interstitial pneumonia. Clin. Imaging. 2014;38:454–457. doi: 10.1016/j.clinimag.2014.02.004. [DOI] [PubMed] [Google Scholar]
  • 37.Shen Y., Wan C., Tian P., Wu Y., Li X., Yang T., An J., Wang T., Chen L., Wen F. CT-Base Pulmonary Artery Measurement in the Detection of Pulmonary Hypertension. Medicine. 2014;93:e256. doi: 10.1097/MD.0000000000000256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Yagi M., Taniguchi H., Kondoh Y., Ando M., Kimura T., Kataoka K., Furukawa T., Suzuki A., Johkoh T., Hasegawa Y. CT-determined pulmonary artery to aorta ratio as a predictor of elevated pulmonary artery pressure and survival in idiopathic pulmonary fibrosis. Respirology. 2017;22:1393–1399. doi: 10.1111/resp.13066. [DOI] [PubMed] [Google Scholar]
  • 39.Terzikhan N., Bos D., Lahousse L., Wolff L., Verhamme K.M.C., Leening M.J.G., Felix J.F., Gall H., Ghofrani H.A., Franco O.H., et al. Pulmonary artery to aorta ratio and risk of all-cause mortality in the general population: The Rotterdam Study. Eur. Respir. J. 2017;49:1602168. doi: 10.1183/13993003.02168-2016. [DOI] [PubMed] [Google Scholar]
  • 40.Iyer A.S., Wells J.M., Vishin S., Bhatt S.P., Wille K.M., Dransfield M.T. CT Scan-Measured Pulmonary Artery to Aorta Ratio and Echocardiography for Detecting Pulmonary Hypertension in Severe COPD. Chest. 2014;145:824–832. doi: 10.1378/chest.13-1422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Remy-Jardin M., Ryerson C.J., Schiebler M.L., Leung A.N.C., Wild J.M., Hoeper M.M., Alderson P.O., Goodman L.R., Mayo J., Haramati L.B., et al. Imaging of Pulmonary Hypertension in Adults: A Position Paper from the Fleischner Society. Radiology. 2021;298:531–549. doi: 10.1148/radiol.2020203108. [DOI] [PubMed] [Google Scholar]
  • 42.Turner V.L., Jubran A., Kim J.B., Maret E., Moneghetti K.J., Haddad F., Amsallem M., Codari M., Hinostroza V., Mastrodicasa D., et al. CTA pulmonary artery enlargement in patients with severe aortic stenosis: Prognostic impact after TAVR. J. Cardiovasc. Comput. Tomogr. 2021 doi: 10.1016/j.jcct.2021.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Sardanelli F., Cozzi A., Monfardini L., Bnà C., Foà R.A., Spinazzola A., Tresoldi S., Cariati M., Secchi F., Schiaffino S. Association of mediastinal lymphadenopathy with COVID-19 prognosis. Lancet Infect. Dis. 2020;20:1230–1231. doi: 10.1016/S1473-3099(20)30521-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Wells J.M., Washko G.R., Han M.K., Abbas N., Nath H., Mamary A.J., Regan E., Bailey W.C., Martinez F.J., Westfall E., et al. Pulmonary Arterial Enlargement and Acute Exacerbations of COPD. N. Engl. J. Med. 2012;367:913–921. doi: 10.1056/NEJMoa1203830. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Tibshirani R. Regression Shrinkage and Selection via the Lasso. J. R. Stat. Soc. Ser. B. 1996;58:267–288. doi: 10.1111/j.2517-6161.1996.tb02080.x. [DOI] [Google Scholar]
  • 46.Muthukrishnan R., Rohini R. LASSO: A feature selection technique in predictive modeling for machine learning; Proceedings of the 2016 IEEE International Conference on Advances in Computer Applications (ICACA); Coimbatore, India. 24 October 2016; pp. 18–20. [Google Scholar]
  • 47.Peduzzi P., Concato J., Feinstein A.R., Holford T.R. Importance of events per independent variable in proportional hazards regression analysis II. Accuracy and precision of regression estimates. J. Clin. Epidemiol. 1995;48:1503–1510. doi: 10.1016/0895-4356(95)00048-8. [DOI] [PubMed] [Google Scholar]
  • 48.Peduzzi P., Concato J., Kemper E., Holford T.R., Feinstein A.R. A simulation study of the number of events per variable in logistic regression analysis. J. Clin. Epidemiol. 1996;49:1373–1379. doi: 10.1016/S0895-4356(96)00236-3. [DOI] [PubMed] [Google Scholar]
  • 49.Maas A.L., Hannun A.Y., Ng A.Y. Rectifier Nonlinearities Improve Neural Network Acoustic Models; Proceedings of the 30th International Conference on Machine Learning; Atlanta, GA, USA. 16–21 June 2013. [Google Scholar]
  • 50.He K., Zhang X., Ren S., Sun J. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification; Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV); Santiago, Chile. 7–13 December 2015; pp. 1026–1034. [Google Scholar]
  • 51.Kingma D.P., Ba J. Adam: A Method for Stochastic Optimization. arXiv. 20141412.6980 [Google Scholar]
  • 52.Van Rossum G., Drake F.L. Python 3 Reference Manual. CreateSpace; Scotts Valley, CA, USA: 2009. [Google Scholar]
  • 53.Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011;12:2825–2830. [Google Scholar]
  • 54.Abadi M., Agarwal A., Barham P., Brevdo E., Chen Z., Citro C., Corrado G.S., Davis A., Dean J., Devin M., et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv. 20161603.04467 [Google Scholar]
  • 55.Cury R.C., Megyeri I., Lindsey T., Macedo R., Batlle J., Kim S., Baker B., Harris R., Clark R.H. Natural Language Processing and Machine Learning for Detection of Respiratory Illness by Chest CT Imaging and Tracking of COVID-19 Pandemic in the US. Radiol. Cardiothorac. Imaging. 2021;3:e200596. doi: 10.1148/ryct.2021200596. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Tezza F., Lorenzoni G., Azzolina D., Barbar S., Leone L.A.C., Gregori D. Predicting in-Hospital Mortality of Patients with COVID-19 Using Machine Learning Techniques. J. Pers. Med. 2021;11:343. doi: 10.3390/jpm11050343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Roberts M., Driggs D., Thorpe M., Gilbey J., Yeung M., Ursprung S., Aviles-Rivero A.I., Etmann C., McCague C., Beer L., et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat. Mach. Intell. 2021;3:199–217. doi: 10.1038/s42256-021-00307-0. [DOI] [Google Scholar]
  • 58.Driggs D., Selby I., Roberts M., Gkrania-Klotsas E., Rudd J.H.F., Yang G., Babar J., Sala E., Schönlieb C.-B. Machine Learning for COVID-19 Diagnosis and Prognostication: Lessons for Amplifying the Signal While Reducing the Noise. Radiol. Artif. Intell. 2021:e210011. doi: 10.1148/ryai.2021210011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Signoroni A., Savardi M., Benini S., Adami N., Leonardi R., Gibellini P., Vaccher F., Ravanelli M., Borghesi A., Maroldi R., et al. BS-Net: Learning COVID-19 pneumonia severity on a large chest X-ray dataset. Med. Image Anal. 2021;71:102046. doi: 10.1016/j.media.2021.102046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Hu Q., Guan H., Sun Z., Huang L., Chen C., Ai T., Pan Y., Xia L. Early CT features and temporal lung changes in COVID-19 pneumonia in Wuhan, China. Eur. J. Radiol. 2020;128:109017. doi: 10.1016/j.ejrad.2020.109017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Xiong Y., Sun D., Liu Y., Fan Y., Zhao L., Li X., Zhu W. Clinical and High-Resolution CT Features of the COVID-19 Infection. Investig. Radiol. 2020;55:332–339. doi: 10.1097/RLI.0000000000000674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Schiaffino S., Albano D., Cozzi A., Messina C., Arioli R., Bnà C., Bruno A., Carbonaro L.A., Carriero A., Carriero S., et al. CT-derived Chest Muscle Metrics for Outcome Prediction in Patients with COVID-19. Radiology. 2021 doi: 10.1148/radiol.2021204141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Pediconi F., Rizzo V., Schiaffino S., Cozzi A., Della Pepa G., Galati F., Catalano C., Sardanelli F. Visceral adipose tissue area predicts intensive care unit admission in COVID-19 patients. Obes. Res. Clin. Pract. 2021;15:89–92. doi: 10.1016/j.orcp.2020.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Grasselli G., Pesenti A., Cecconi M. Critical Care Utilization for the COVID-19 Outbreak in Lombardy, Italy. JAMA. 2020;323:1545–1546. doi: 10.1001/jama.2020.4031. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Data Availability Statement

We will be able to make available deidentified individual participant data that underlie the results reported in this article, beginning 12 months and ending 24 months following article publication. We will consider requests from researchers who will provide a methodologically sound proposal and will share data needed to achieve the aims declared in the approved proposal. Proposals may be submitted up to 36 months following article publication. After 24 months the data will be available in the data warehouse of the coordinating center but without investigator support other than deposited metadata. Information regarding submitting proposals and accessing data may be obtained by email inquiry to the first/corresponding author at andrea.cozzi1@unimi.it.


Articles from Journal of Personalized Medicine are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES