Highlights
-
•
Chest CT helps identify patients with severe COVID-19 using only three qualitative features.
-
•
A qualitative model based on three qualitative variables can avoid calculating semi-quantitative total CT score.
-
•
New Early Warning Score 2 is comparable to the CT score for identification of severe forms of COVID-19.
Keywords: Severe acute respiratory syndrome coronavirus 2, Tomography, X-ray computed (CT), COVID-19, Risk factors, Severity of illness index
Abbreviations: AUC, area under the curve; COPD, chronic obstructive pulmonary disease; COVID-19, Coronavirus disease 2019; CT, computed tomography; EWS, Early Warning Score; GGO, ground glass opacity; HU, Hounsfield units; ICU, intensive care unit; LUL, left upper lobe; ML, middle lobe; NEWS2, New Early Warning Score 2; OR, odd ratio; ROC, receiver operating characteristic; RT-PCR, reverse transcriptase-polymerase chain reaction; RUL, right upper lobe; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; SD, standard deviation
Abstract
Purpose
The purpose of this study was to identify clinical and chest computed tomography (CT) features associated with a severe form of coronavirus disease 2019 (COVID-19) and to propose a quick and easy to use model to identify patients at risk of a severe form.
Materials and methods
A total of 158 patients with biologically confirmed COVID-19 who underwent a chest CT after the onset of the symptoms were included. There were 84 men and 74 women with a mean age of 68 ± 14 (SD) years (range: 24–96 years). There were 100 non-severe and 58 severe cases. Their clinical data were recorded and the first chest CT examination was reviewed using a computerized standardized report. Univariate and multivariate analyses were performed in order to identify the risk factors associated with disease severity. Two models were built: one was based only on qualitative CT features and the other one included a semi-quantitative total CT score to replace the variable representing the extent of the disease. Areas under the ROC curves (AUC) of the two models were compared with DeLong's method.
Results
Central involvement of lung parenchyma (P < 0.001), area of consolidation (P < 0.008), air bronchogram sign (P < 0.001), bronchiectasis (P < 0.001), traction bronchiectasis (P < 0.011), pleural effusion (P < 0.026), large involvement of either one of the upper lobes or of the middle lobe (P < 0.001) and total CT score ≥ 15 (P < 0.001) were more often observed in the severe group than in the non-severe group. No significant differences were found between the qualitative model (large involvement of either upper lobes or middle lobe [odd ratio (OR) = 2.473], central involvement [OR = 2.760], pleural effusion [OR = 2.699]) and the semi-quantitative model (total CT score ≥ 15 [OR = 3.342], central involvement [OR = 2.344], pleural effusion [OR = 2.754]) with AUC of 0.722 (95% CI: 0.638–0.806) vs. 0.739 (95% CI: 0.656–0.823), respectively (P = 0.209).
Conclusion
We have developed a new qualitative chest CT-based multivariate model that provides independent risk factors associated with severe form of COVID-19.
1. Introduction
By the end of 2019, coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) broke out in Wuhan City, China, and rapidly spread worldwide [1]. Although the majority of symptomatic patients have a mild form with fever, dry cough, dyspnea, myalgia and fatigue [1], some patients can develop a severe form of the disease. The latter may present with rapid worsening, sometimes resulting in acute respiratory distress syndrome that may require mechanical ventilation or intensive care unit (ICU) monitoring, and even in death [2]. Severe disease has been reported to occur in around 15% of the patients and mortality is around 2% of COVID-19 cases [3]. After a massive “first wave”, which has been particularly contagious and deadly and put health systems under strain in the spring of 2020 [4], [5], [6], a period of relative transitional lull appeared during the summer. In Europe, however, since September 2020, the number of new cases of COVID-19 has started to rise again. The need to early identify patients at risk of a severe form of COVID-19, for an early and appropriate management, is rendered critical due to the large number of affected patients, the limited number of beds in health centers and the lack of specific or effective treatment to date.
The gold standard for establishing the diagnosis of SARS-CoV-2 infection is based on the biological analysis of respiratory secretions by real-time reverse transcriptase-polymerase chain reaction (RT-PCR) assay. Computed tomography (CT) not only can help make the diagnosis of COVID-19 with greater sensitivity than RT-PCR test [7], [8], [9], [10], [11], but has also been used to suggest disease severity or outcome of the disease [12], [13], [14]. Indeed, some clinical, biological and radiological features are associated with the severity of the disease [15]. Regarding CT, the most widely proposed CT tool to assess severity is a semi-quantitative CT score, aimed at reflecting the extent of the lung abnormalities related to the COVID-19 [16], [17]. However, the way to calculate these “total CT scores” varies throughout the literature, along with their thresholds used to optimize differentiation between non-severe and severe disease, thus limiting their implementation in clinical practice [17], [18], [19], [20], [21]. We hypothesized that a model based only on qualitative features should be more robust and less cumbersome to use, particularly in a context of increased clinical workload.
The purpose of this study was to investigate the clinical and CT features associated with severe COVID-19 patients, and to propose a model based on qualitative CT features able to rapidly identify these patients.
2. Materials and methods
2.1. Study population
In accordance with French law, this retrospective study on medical records was authorized by the “Commission Nationale de l’Informatique et des Libertés”. The participants were informed of the possibility of using the information concerning them, for biomedical research purposes, and had a right of opposition. Among a cohort of 499 patients over 18 years of age hospitalized from February 25th to April 30th 2020 in Reims University Hospital for COVID-19, those with confirmation of the diagnosis of COVID-19 using RT-PCR assay of SARS-CoV-2 and with a chest CT examination performed after the onset of the symptoms were retrospectively included. Patients with missing clinical data (n = 1), patients with prior lobectomy (n = 2) and those for whom CT examination was not available (n = 3), not interpretable (n = 2) or performed more than 15 days after the onset of the symptoms (n = 22) were excluded. Fig. 1 shows the flow chart of the study population.
Fig. 1.
Study flow chart. COVID-19: Coronavirus disease 2019; CT: computed tomography; RT-PCR: reverse transcriptase-polymerase chain reaction.
Patients were assigned to two groups, including “non-severe cases” and “severe cases” according to the worst clinical situation during hospitalization, assessed at the end of hospitalization. The “severe cases” patients were hospitalized patients who required non-invasive ventilation or humidified nasal high-flow oxygen system (Optiflow™) or invasive ventilation or extracorporeal membrane oxygenation as well as those who died prior to discharge. “Non-severe cases” patients corresponded to the other patients, regardless of the need for oxygen therapy.
The clinical parameters including age, sex, body mass index, New Early Warning Score 2 (NEWS2), hospital length of stay, time between the onset of the symptoms and the first CT examination (divided into two groups, 0 to 7 days and 8 to 15 days), admission or not into ICU during hospitalization stay, worst clinical situation during hospitalization, status (dead or alive) at day 28 after the onset of the symptoms, comorbidities (diabetes mellitus, cardiovascular disease, systemic hypertension, asthma and chronic obstructive pulmonary disease), symptoms and clinical signs, as well as laboratory findings, were collected and evaluated. We used an Early Warning Score (EWS) based on an adapted version of the NEWS2 score with age ≥ 65 (score 3 points) developed by Liao et al. [22].
2.2. CT examinations and imaging evaluation
2.2.1. CT protocol
All patients underwent chest CT examinations covering from the apex to the lung base, in the supine position, using a 64-row helical CT scanner (GE Revolution EVO CT Scanner, GE Medical Systems). The scanning parameters were as follows: reconstruction matrix of 512 mm × 512 mm and 1.375:1 helical pitch. Lung window setting was with a window level of −600 Hounsfield units (HU) and window width of 1600 HU. One hundred and seven (107/158; 67.7%) CT examinations were acquired without intravenous administration of iodinated contrast material, with the following parameters: tube voltage, 120 kV; automatic tube current, 100 to 320 mA; and slice thickness, 1.25 mm with 1.25 mm increment. Fifty-one (51/158; 32.3%) CT examinations were acquired after intravenous administration of iodinated contrast material at a pulmonary arterial phase, with the following parameters: tube voltage, 100 kV; automatic tube current, 100 to 450 mA; slice thickness, 0.625 mm with 0.625 mm increment.
2.2.2. Image analysis
A radiologist, blinded to clinical data and laboratory findings reviewed all CT examinations (the first CT when the patients had multiple CT examinations), using a computerized structured report for COVID-19 available on the web platform (“Keydiag”, Keymaging, Grenoble, France, https://keydiag.org). This structured report is based on the guidelines for COVID-19, issued by the French Thoracic Imaging Society. It includes the following imaging features, defined according to the Fleischner Society Glossary of Terms for Thoracic Imaging [23], as follows: ground glass opacity (GGO) (nodular or area of GGO), consolidation (rounded or subpleural band or area of consolidation), crazy-paving pattern, axial distribution of the disease (presence of central involvement or not), sagittal distribution (inferior and/or posterior involvement), number of lobes affected, lobar distribution, degree of involvement of each lung lobe, overall extent of lung involvement measured as detailed below. Other abnormalities including intralobular reticulation, vascular enlargement (dilatation of pulmonary vessels around and within the lesions), air bronchogram sign within GGO or consolidation, bronchiectasis, traction bronchiectasis, halo sign, reversed halo sign, pleural thickening, intrathoracic lymph node enlargement (short-axis diameter ≥ 10 mm), pleural effusion and pericardial effusion, were noted. Presence of pulmonary embolism, pulmonary infarction and enlargement of the pulmonary trunk (diameter of main pulmonary artery > 29 mm) was reported too.
Each of the five lung lobes was assessed for degree of involvement and scored as “0” (0%), “1” (1%–9%), “2” (10%–24%), “3” (25%–49%), “4” (50%–74%) or “5” (75%–100%). A “total CT score” was established by summing up the degree of involvement of each lobe (quoted from 0 to 5) so as to obtain a score ranging from 0 to 25. Overall extent of lung involvement was also classified according to the French guidelines [24] as follows: none (0%), minimal (1%–9%), moderate (10%–24%), extensive (25%–49%), severe (50%–74%) and critical (75%–100%), and was obtained in two ways: 1. visually and 2. using Keydiag software. The overall extent, assessed visually, was dichotomized into two groups: “none, minimal, moderate and extensive” overall extent (0%–49%) and “severe and critical” extent (50%–100%). The involvement of all five lobes was also reported. Area of GGO and area of consolidation were defined as non-nodular opacities, concerning multiple adjacent pulmonary lobules.
For the purpose of this study, we created a new qualitative imaging criterion reflecting the extent of the disease in a lobe. A visual involvement of more than half of a lobe was then called “large involvement”.
2.3. Statistical analysis
Categorical variables were expressed as raw numbers, proportions and percentages. Continuous variables were expressed as means ± standard deviations (SD) or medians (interquartile ranges) and ranges. Differences between the non-severe group and the severe group were analyzed using Chi2 or Fisher exact test for categorical variables and Student t-test or Wilcoxon–Mann–Whitney U-test for continuous variables. The calculation of the weighted kappa coefficient was performed to assess the agreement between the visual analysis and the analysis with Keydiag software [25]. NEWS2 was analyzed both as a continuous variable (ranging from 0 to 15) and as a dichotomized variable: “0 to 6” and “7 to 15”.
Multivariate logistic regression with backward selection, including variables with a P-value < 0.05 by univariate analyses, was used to identify the independent risk factors associated with the development of severe COVID-19. Systematic adjustment was performed on age and on time between the first symptoms and the first CT examination. The most relevant clinically and radiologically qualitative variables were chosen in building the multivariate model. Goodness of fit of the final model was assessed using the Hosmer–Lemeshow test. Receiver operating characteristic (ROC) analysis was used to determine the clinical value of total CT scores in distinguishing the non-severe and severe forms, and find the corresponding cutoff value. Then, another multivariate model was built replacing the qualitative variable representing the degree of spatial extent by the total CT score as a categorical variable, using the determined cutoff. The discriminatory ability of the two multivariate models was evaluated by using the area under the ROC curve (AUC) and its 95% CI, and compared using the method of DeLong et al. [26]. Differences in diagnostic performances between the total CT score and the NEWS2 as continuous variables were also analyzed by comparing the ROC curves. All analyses were performed by using SAS statistical software (version 9.4, SAS Institute, Cary) and R software program (version 3.6.1, R Foundation for Statistical Computing). The level of statistical significance was set at P < 0.05.
3. Results
3.1. Patient demographics and clinical manifestations
One hundred and fifty eight patients constituted the final population: 84 (84/158; 53.2%) men and 74 (74/158; 46.8%) women. Mean age was 68 ± 14 (SD) years (range: 24–96 years). Among the 158 patients, 100 (100/158; 63.3%) were non-severe cases and 58 (58/158; 36.7%) were severe cases. Table 1 summarizes demographics and clinical baseline characteristics between non-severe group and severe group. There was no significant age or sex difference between the two groups.
Table 1.
Demographics and clinical baseline characteristics of non-severe group and severe group.
| Parameter | All patients (n = 158) |
Non-severe group (n = 100) |
Severe group (n = 58) |
P-value |
|---|---|---|---|---|
| Age (years) | 68 ± 14 [24–96] | 68 ± 14 [35–93] | 68 ± 16 [24–96] | 0.918 |
| Sex | 0.088 | |||
| Male | 84 (84/158; 53.2%) | 48 (48/100; 48.0%) | 36 (36/58; 62.1%) | |
| Female | 74 (74/158; 46.8%) | 52 (52/100; 52.0%) | 22 (22/58; 37.9%) | |
| Body mass index (n = 115) | 29 (24, 33) [16–63] | 29 (24, 33) [19–63] | 28 (24, 32) [16–42] | 0.723 |
| NEWS2 ≥ 7 (n = 143) | 78 (78/143; 54.6%) | 42 (42/96; 43.8%) | 36 (36/47; 76.6%) | < 0.001 |
| Hospital stay (days) (n = 157) | 10 (6, 18) [0–137] | 10 (6, 15) [0–69] | 11 (6, 26) [1–137] | 0.070 |
| Time between symptoms and first CT | 7 (4, 10) [0–15] | 7 (3, 10) [0–15] | 7 (5, 10) [0–15] | 0.532 |
| Death at day 28 | 33 (33/158; 20.9%) | 0 (0/100; 0%) | 33 (33/58; 56.9%) | … |
| Hypertension (n = 140) | 91 (91/140; 65.0%) | 60 (60/87; 69.0%) | 31 (31/53; 58.5%) | 0.208 |
| Cardiovascular disease (n = 140) | 69 (69/140; 49.3%) | 43 (43/87; 49.4%) | 26 (26/53; 49.1%) | 0.966 |
| Diabetes mellitus (n = 140) | 55 (55/140; 39.3%) | 30 (30/87; 34.5%) | 25 (25/53; 47.2%) | 0.136 |
| Asthma (n = 32) | 11 (11/32; 34.4%) | 9 (9/19; 47.4%) | 2 (2/13; 15.4%) | 0.128 |
| COPD (n = 32) | 12 (12/32; 37.5%) | 5 (5/19; 26.3%) | 7 (7/13; 53.9%) | 0.150 |
| Wrestling respiratory signs (n = 157) | 5 (5/157; 3.2%) | 1 (1/100; 1.0%) | 4 (4/57; 7.0%) | 0.059 |
| Chest pain (n = 156) | 15 (15/156; 9.6%) | 7 (7/100; 7.0%) | 8 (8/56; 14.3%) | 0.139 |
| Anosmia (n = 152) | 19 (19/152; 12.5%) | 14 (14/97; 14.4%) | 5 (5/55; 9.1%) | 0.339 |
| Fever (n = 157) | 127 (127/157; 80.9%) | 80 (80/99; 80.8%) | 47 (47/58; 81.0%) | 0.972 |
| Ageusia (n = 152) | 17 (17/152; 11.2%) | 12 (12/97; 12.4%) | 5 (5/55; 9.1%) | 0.536 |
| Cough (n = 156) | 124 (124/156; 79.5%) | 78 (78/98; 79.6%) | 46 (46/58; 79.3%) | 0.966 |
| Dyspnea (n = 155) | 103 (103/155; 66.5%) | 60 (60/98; 61.2%) | 43 (43/57; 75.4%) | 0.071 |
| Myalgia (n = 155) | 39 (39/155; 25.2%) | 23 (23/98; 23.5%) | 16 (16/57; 28.1%) | 0.525 |
| Headache (n = 155) | 17 (17/155; 11.0%) | 10 (10/98; 10.2%) | 7 (7/57; 12.3%) | 0.690 |
| Diarrhea (n = 154) | 51 (51/154; 33.1%) | 31 (31/98; 31.6%) | 20 (20/56; 35.7%) | 0.605 |
| Abdominal pain (n = 154) | 21 (21/154; 13.6%) | 13 (13/98; 13.3%) | 8 (8/56; 14.3%) | 0.860 |
| Respiratory rate (n = 143) | 22 (18, 28) [12–55] | 20 (17, 28) [12–44] (n = 94) |
26 (21, 32) [14–55] (n = 49) |
< 0.001 |
| Temperature, °C (n = 150) | 37.7 ± 1.0 [35.0–40.6] | 37.6 ± 1.0 [35.5–40.6] (n = 97) |
37.8 ± 1.0 [35.0–40.3] (n = 53) |
0.108 |
| Mean blood pressure (mmHg) (n = 148) | 92.8 ± 14.9 [57.7–150.0] | 93.1 ± 15.0 [57.7–150.0] (n = 96) |
92.2 ± 14.9 [64.0–119.3] (n = 52) |
0.720 |
| Heart rate (bpm) (n = 151) | 88 ± 17 [47–141] | 86 ± 15 [56–122] (n = 97) |
92 ± 20 [47–141] (n = 54) |
0.092 |
Quantitative data are expressed as means ± standard deviations or medians (Q1, Q3); numbers in brackets are ranges. Qualitative variables are expressed as raw numbers; numbers in parentheses are proportions followed by percentages.
Bold indicates significant P-value.
NEWS2: New Early Warning Score 2; COPD: chronic obstructive pulmonary disease.
Number of patients with missing data: weight (n = 24), BMI (n = 43), NEWS2 (n = 15), hospital length of stay (n = 1), hypertension (n = 18), cardiovascular disease (n = 18), diabetes mellitus (n= 18), asthma (n = 126), COPD (n = 126), wrestling respiratory signs (n = 1), chest pain (n = 2), fever (n = 1), anosmia (n = 6), ageusia (n = 6), cough (n = 2), dyspnea (n = 3), myalgia (n = 3), headache (n = 3), diarrhea (n = 4), abdominal pain (n = 4), respiratory rate (n = 15), temperature (n = 8), median arterial pressure (n = 10), heart rate (n = 7).
In the non-severe group, 67 (67/100; 67%) patients required oxygen therapy. In the severe group, according to the worst clinical situation during hospitalization, 14 (14/58; 24.1%) patients required non-invasive ventilation or humidified nasal high-flow oxygen system (Optiflow™), 11 (11/58; 19.0%) invasive ventilation or extracorporeal membrane oxygenation and 33 (33/58; 56.9%) died prior to discharge.
Forty-two (42/158; 26.6%) patients were admitted in ICU during their hospitalization and 33 (33/158; 20.9%) died within the 28 days after the first symptoms. Median time from the onset of the symptoms to the first CT examination was 7 days (IQR; 4, 10; range: 0–15 days); 86 (86/158; 54.4%) patients underwent CT within the first 7 days after the onset of the symptoms and 72 (72/158; 45.6%) between 8 and 15 days.
Patients in the severe group had a significantly greater NEWS2 than those in the non-severe group (9.3 ± 2.9 [range: 3–15] vs. 6.1 ± 3.4 [range: 0–15]; P < 0.001). Furthermore, a NEWS2 ≥ 7, corresponding to a “high clinical risk requiring urgent or emergency response” [21], was also significantly more often found in the severe group (P < 0.001). There were no significant differences of symptoms and clinical signs between the two groups except for the respiratory rate that was significantly greater in the severe group (P < 0.001).
3.2. Laboratory findings
Table 2 summarizes laboratory findings between non-severe group and severe group. Patients in the severe group had a lower lymphocyte count and greater levels of serum inflammatory biomarkers.
Table 2.
Laboratory findings between non-severe group and severe group.
| Parameter | Total (n = 158) |
Non-severe group (n = 100) |
Severe group (n = 58) |
P-value |
|---|---|---|---|---|
| White blood cell count, × 109/L | 6.90 (5.20, 9.30) | 6.50 (5.00, 7.90) | 8.60 (6.00, 12.10) | < 0.001 |
| (n = 150) | [0.40–28.20] | [0.40–28.20] | [0.40–27.00] | |
| (n = 97) | (n = 53) | |||
| Neutrophil count, × 109/L | 5.05 (3.40, 7.60) | 4.70 (3.15, 5.95) | 6.80 (4.50, 10.50) | < 0.001 |
| (n = 146) | [0.90–25.50] | [1.20–25.50] | [0.90–20.0] | |
| (n = 96) | (n = 50) | |||
| Lymphocyte count, × 109/L | 0.90 (0.70, 1.40) | 1.05 (0.80, 1.55) | 0.85 (0.60, 1.20) | 0.025 |
| (n = 146) | [0.10–11.20] | [0.10–4.40] | [0.30–11.20] | |
| (n = 96) | (n = 50) | |||
| C-reactive protein, mg/L | 71.9 (34.4, 147.0) | 53.8 (20.0, 100.5) | 136.0 (70.5, 206.0) | < 0.001 |
| (n = 146) | [1.6–370.0] | [1.6–345.0] | [6.3–370.0] | |
| (n = 96) | (n = 50) | |||
| D-dimer level, mg/L | 0.82 (0.43, 3.52) | 0.70 (0.43, 1.32) | 1.76 (0.70, 4.13) | 0.060 |
| (n = 51) | [0.11–20.00] | [0.11–18.98] | [0.27–20.00] | |
| (n = 31) | (n = 20) | |||
| NT-proBNP, pg/mL | 424.50 (157.50, 1694.00) | 343.00 (119.00, 1338.00) | 839.00 (221.00, 4175.00) | 0.047 |
| (n = 124) | [7.00–81888.00] | [9.00–81888.00] | [7.00–67152.00] | |
| (n = 81) | (n = 43) |
All data are expressed as medians (Q1, Q3); numbers in brackets are ranges.
Number of patients with missing data: white blood cell count (n = 8), neutrophil count (n = 12), lymphocyte count (n = 12), C-reactive protein (n = 12), D-dimer level (n = 107), NT-ProBNP (n = 34). Bold indicates significant P-value.
3.3. Chest CT findings
On CT, COVID-19-related abnormalities were present in 142/158 patients (89.9%) (Table 3 ). The most common CT findings in both groups included GGO (139/158; 88.0%), consolidation (116/158; 73.4%), intralobular reticulation (120/158; 76.0%), vascular enlargement (113/158; 71.5%), pleural thickening (107/158; 67.7%), crazy-paving pattern (96/158; 60.8%) and air bronchogram sign (93/158; 58.9%).
Table 3.
CT features in non-severe group and severe group.
| Parameter | Total (n = 158) |
Non-severe group (n = 100) |
Severe group (n = 58) |
P-value |
|---|---|---|---|---|
| COVID-19 lesions | 142 (142/158; 89.9%) | 91 (91/100; 91.0%) | 51 (51/58; 87.9%) | 0.538 |
| Central involvement | 78 (78/158; 49.4%) | 38 (38/100; 38.0%) | 40 (40/58; 69.0%) | < 0.001 |
| Posterior and/or inferior distribution | 73 (73/158; 46.2%) | 50 (50/100; 50.0%) | 23 (23/58; 39.7%) | 0.441 |
| GGO | 139 (139/158; 88.0%) | 89 (89/100; 89.0%) | 50 (50/58; 86.2%) | 0.603 |
| Nodular GGO | 84 (84/158; 53.2%) | 59 (59/100; 59.0%) | 25 (25/58; 43.1%) | 0.054 |
| Area of GGO | 126 (126/158; 79.8%) | 76 (76/100; 76.0%) | 50 (50/58; 86.2%) | 0.124 |
| Crazy-paving pattern | 96 (96/158; 60.8%) | 57 (57/100; 57.0%) | 39 (39/58; 67.2%) | 0.204 |
| Consolidation | 116 (116/158; 73.4%) | 69 (69/100; 69.0%) | 47 (47/58; 81.0%) | 0.099 |
| Rounded consolidation | 41 (41/158; 26.0%) | 23 (23/100; 23.0%) | 18 (18/58; 31.0%) | 0.267 |
| Subpleural band of consolidation | 92 (92/158; 58.2%) | 60 (60/100; 60.0%) | 32 (32/58; 55.2%) | 0.553 |
| Area of consolidation | 48 (48/158; 30.4%) | 23 (23/100; 23.0%) | 25 (25/58; 43.1%) | 0.008 |
| Intralobular reticulation | 120 (120/158; 76.0%) | 75 (75/100; 75.0%) | 45 (45/58; 77.6%) | 0.714 |
| Vascular enlargement | 113 (113/158; 71.5%) | 67 (67/100; 67.0%) | 46 (46/58; 79.3%) | 0.098 |
| Air bronchogram sign | 93 (93/158; 58.9%) | 47 (47/100; 47.0%) | 46 (46/58; 79.3%) | < 0.001 |
| Bronchiectasis | 40 (40/158; 25.3%) | 16 (16/100; 16.0%) | 24 (24/58; 41.4%) | < 0.001 |
| Traction bronchiectasis | 15 (15/158; 9.5%) | 5 (5/100; 5.0%) | 10 (10/58; 17.2%) | 0.011 |
| Reversed halo sign | 5 (5/158; 3.2%) | 5 (5/100; 5.0%) | 0 (0/58; 0.0%) | 0.084 |
| Halo sign | 4 (4/158; 2.5%) | 4 (4/100; 4.0%) | 0 (0/58; 0.0%) | 0.123 |
| Pleural thickening | 107 (107/158; 67.7%) | 66 (66/100; 66.0%) | 41 (41/58; 70.7%) | 0.543 |
| Pleural effusion | 27 (27/158; 17.1%) | 12 (12/100; 12.0%) | 15 (15/58; 25.9%) | 0.026 |
| Pericardial effusion | 14 (14/158; 8.9%) | 10 (10/100; 10.0%) | 4 (4/58; 6.9%) | 0.508 |
| Intrathoracic lymph node enlargement | 79 (79/158; 50.0%) | 47 (47/100; 47.0%) | 32 (32/58; 55.2%) | 0.322 |
| Total CT score (cutoff ≥ 15) | 50 (50/158; 31.7%) | 19 (19/100; 19.0%) | 31 (31/58; 53.5%) | < 0.001 |
| Visual overall extent | < 0.001 | |||
| 0–49% | 120 (120/158; 75.9%) | 88 (88/100; 88.0%) | 32 (32/58; 55.2%) | |
| 50–100% | 38 (38/158; 24.1%) | 12 (12/100; 12.0%) | 26 (26/58; 44.8%) | |
| Large involvement of at least RUL or LUL or ML | 47 (47/158; 29.8%) | 19 (19/100; 19.0%) | 28 (24/58; 48.3%) | < 0.001 |
| Involvement of 5 lobes | 105 (105/158; 66.5%) | 61 (61/100; 61.0%) | 44 (44/58; 75.9%) | 0.057 |
| Frequency of lobe involvement | ||||
| Right upper lobe | 128 (128/158; 81.0%) | 79 (79/100; 79.0%) | 49 (49/58; 84.5%) | 0.397 |
| Right middle lobe | 115 (115/158; 72.8%) | 69 (69/100; 69.0%) | 46 (46/58; 79.3%) | 0.160 |
| Right lower lobe | 134 (134/158; 84.8%) | 85 (85/100; 85%) | 49 (49/58; 84.5%) | 0.930 |
| Left upper lobe | 127 (127/158; 80.4%) | 78 (78/100; 78%) | 49 (49/58; 84.5%) | 0.323 |
| Left lower lobe | 133 (133/158; 84.2%) | 85 (85/100; 85%) | 48 (18/58; 82.8%) | 0.710 |
| No. of lobes involved | 5 (4, 5) [0–5] | 5 (3, 5) [0–5] | 5 (5, 5) [0–5] | 0.153 |
| Enlarged pulmonary trunk | 42 (42/158; 26.6%) | 23 (23/100; 23.0%) | 19 (19/58; 32.8%) | 0.181 |
Qualitative variables are expressed as raw numbers; numbers in parentheses are proportions followed by percentages. Quantitative data are expressed as medians (Q1, Q3); numbers in brackets are ranges.
GGO: ground glass opacity; RUL: right upper lobe; LUL: left upper lobe; ML: middle lobe. Bold indicates significant P-value.
Central involvement of lung parenchyma by the disease, area of consolidation, air bronchogram sign, bronchiectasis, traction bronchiectasis and pleural effusion were significantly more often seen in the severe group (Table 3). So was “large involvement of at least one of the three following lobes including LUL, RUL and ML” (P < 0.001). Regarding the visual overall extent of lung involvement, an involvement ≥ 50% was also significantly more often seen in the severe group (P < 0.001).
Weighted kappa coefficient for correlation between visual analysis and Keydiag software to determine the overall extent of lung involvement was excellent (0.929; CI: 95%: 0.895–0.962).
Median total CT score was 11.5 (IQR: 6, 16; range: 0–24). AUC of ROC analysis of the total CT score, taken as a categorical variable with a cutoff value of 15 and performed to identify patients at risk of severe form, was 0.672. With this cutoff value, sensitivity and specificity were 53.4% and 81.0%, respectively (Fig. 2 ). Thus, 19 (19/100; 19.0%) patients had a score ≥ 15 in the non-severe group vs. 31 (31/58; 53.5%) in the severe group. The total CT scores of severe patients were then significantly greater than those of the non-severe patients with this cutoff (P < 0.001).
Fig. 2.
Receiver operating characteristic (ROC) analysis of the total CT score taken as a categorical variable with a cutoff value of 15: the area under the curve to identify patients at risk of a severe form of Coronavirus disease 2019 (COVID-19) was 0.672 (95% CI: 0.597–0.748). The cutoff value of 15 yielded 53.4% sensitivity and 81.0% specificity. AUC: area under the curve; CI: confidence interval.
Among the 51 (51/158; 32.3%) patients in whom pulmonary arterial phase contrast-enhanced CT examination was available, only 3 (3/51; 5.9%) had CT signs of pulmonary embolism. There was only one pulmonary infarction, in the severe group. Among the 27 (27/158; 17.1%) patients who presented with a pleural effusion, none was of great abundance. At the end of the CT analysis, 29 (29/158; 18.4%) cases had a condition described as “compatible” with COVID-19 and 113 (113/158; 71.5%) as “very suggestive” or “typical”.
The AUC of the total CT score as a continuous variable was 0.713 (95% CI: 0.612–0.813) and the AUC of the NEWS2 was 0.768 (95% CI: 0.690–0.845). No significant difference in diagnostic performances between the total CT score and the NEWS2 as continuous variables was found (P = 0.327) (Fig. 3 ).
Fig. 3.
Graph shows receiver operating characteristic (ROC) curves of the total CT score and the New Early Warning Score 2 (NEWS2) as continuous variables for diagnosing severe Coronavirus disease 2019 (COVID-19) patients. No significant difference in diagnostic performances between the total CT score and the NEWS2 as continuous variables was found (P = 0.327).
3.4. Multivariate analysis and models associated with a severe form of COVID-19
Table 4 shows the most relevant clinically and radiologically qualitative multivariate model in relationship to the severe form of COVID-19. The qualitative model included central involvement (OR = 2.760), pleural effusion (OR = 2.699) and large involvement of LUL or RUL or ML (OR = 2.473), and had an AUC of 0.722 (95% CI: 0.638–0.806).
Table 4.
Qualitative multivariate model associated with a severe form of COVID-19.
| Variablea | OR | 95% confidence interval |
|---|---|---|
| Central involvement | 2.760 | 1.239–6.148 |
| Pleural effusion | 2.699 | 1.092–6.669 |
| Large involvement of at least RUL or LUL or ML | 2.473 | 1.103–5.546 |
AUC: 0.722 (95% CI: 0.638–0.806).
RUL: right upper lobe; LUL: left upper lobe; ML: middle lobe; OR: odd ratio.
Adjusted on age and on time between the first symptoms and the first CT examination
Table 5 shows the second multivariate model that was built replacing the qualitative variable “large involvement of at least RUL or LUL or ML”, representing the degree of spatial extent, by the total CT score as a categorical variable using the cutoff of 15. This semi-quantitative model included central involvement (OR = 2.344), pleural effusion (OR = 2.754) and total CT score ≥ 15 (OR = 3.342), and had an AUC of 0.739 (95% CI: 0.656–0.823).
Table 5.
Semi-quantitative multivariate model associated with a severe form of COVID-19.
| Variablea | OR | 95% confidence interval |
|---|---|---|
| Central involvement | 2.344 | 1.032–5.325 |
| Pleural effusion | 2.754 | 1.099–6.901 |
| Total CT score ≥ 15 | 3.342 | 1.466–7.619 |
AUC: 0.739 (95% CI: 0.656–0.823).
CT: computed tomography; OR: odd ratio.
Adjusted on age and on time between the first symptoms and the first CT examination.
No significant difference in diagnostic performances between these two models was found (P = 0.209) (Fig. 4 ).
Fig. 4.
Graph shows receiver operating characteristic (ROC) curves of the qualitative and the semi-quantitative multivariate models for diagnosing severe Coronavirus disease 2019 (COVID-19) patients. No significant difference in diagnostic performances between these two models was found (P = 0.209).
4. Discussion
We propose a new qualitative model allowing a simple and quick evaluation of first chest CT examination performed in patients with COVID-19, in order to identify patients at risk of developing a severe form of the disease. This model is based on the assessment of central involvement of lung parenchyma by the disease, of pleural effusion and of large involvement of at least one upper lobe or of the middle lobe.
Our qualitative model does not differ in terms of performances from a model integrating the most commonly used and previously validated total semi-quantitative CT score, based on the overall extent of the disease [16]. Although the time needed to assess this total CT score has never been calculated, to our knowledge, it is highly likely that the time required to sum up the scores of each five lobes is higher than just rapidly checking three lobes for the presence of any large COVID-19-related abnormality. Of note, calculation of well-aerated lung in COVID-19 patients using automatic software has been reported as taking a median time of as much as 270 seconds, even though software-assisted calculations are meant to be quicker than human quantifications [27]. Moreover, the use of a total semi-quantitative CT score seems difficult to translate to daily practice as the way to calculate the score widely varies throughout the different studies [12], [13], [19], [20], [28]. Besides, the cutoffs used to differentiate non-severe and severe forms of COVID-19 also vary, depending on the type of score used, the outcome of the study, the type and size of the population, the number of severe cases included [15], [17], [18], [21], [29].
In our study, the performances of the qualitative model and of the model with integration of the total semi-quantitative CT score, as well as the performances of the semi-quantitative CT score on its own, were all lower than those reported in the literature. Notably, the area under the ROC curve for the total semi-quantitative CT score on its own was 0.672 using an optimal threshold of 15, whereas other authors have reported values of AUC in the range of 0.85–0.90 using a total CT score with the same range (0–25), albeit not with the same cutoffs [15], [29]. Differences in criteria at inclusion as well as in the clinical definition of severe forms may explain these discrepancies.
Area of consolidation, bronchiectasis and traction bronchiectasis were significantly more often found in the severe group of patients. In line with some other studies, air bronchogram sign [13], [17] and pleural effusion [17] were also significantly more often found. Unlike in some other studies, pericardial effusion was not found as a criteria associated with a severe form, nor was lymph node enlargement, nor was crazy-paving pattern [13], [17]. The most relevant CT features were all related to the extent of the disease. The total CT scores of the patients in the severe group were significantly higher than those of the non-severe group, which was consistent with most studies [15], [17], [18], [29].
As far as clinical criteria are concerned, Early Warning Scores have been reported to be among the most accurate tools for predicting patient deterioration outside ICU [30] and have been designed to assess infected patients at their arrival to the emergency department. Some of the variants of the EWS, such as the one used in our study, which is an adapted version of the NEWS2, were suggested to evaluate physiological parameters in acute COVID-19 patients and thus to be able to early and rapidly identify patients requiring ICU admission [31], [32]. In our study, NEWS2 had performances similar to those of the total semi-quantitative CT score, thus emphasizing the value of this easy to perform and quick clinical tool in identifying patients at risk of a severe form of COVID-19. To our knowledge, so far, our study is the first to compare both EWS and CT in this context.
In line with other studies, in our study patients with severe disease had higher neutrophil and lower lymphocyte counts, as well as higher C-reactive protein serum level [15], [18], [19], [21]. Conversely, and unlike in other studies [15], [19], [33], D-dimer level was not significantly associated with disease severity. However, this result has to be considered in light of the low number of patients with pulmonary embolism in our study compared to what has been reported so far in other studies [34], [35], [36], [37].
Limitations of our study include its retrospective nature. Patients thus underwent CT examinations based on clinical need and not based on a predesigned study protocol. This may have caused a selection bias of the more clinically ill patients. Moreover, patients may have received treatments before CT examination with a potential impact on the extent and nature of the lesions. Another limitation is the absence of interobserver reproducibility assessment. However, evaluation of the CT examinations was thorough and standardized through the use of a computerized standardized report illustrating each CT feature with a typical image. Lastly, the time between the onset of the symptoms and the CT examination was not standardized. Yet, we included patients whose CT examinations were performed within the first 15 days and the median time wound up being of seven days in both non-severe and severe groups.
In conclusion, we suggest the use of a qualitative model associating three “easy to assess” qualitative CT features including: (i) central involvement of lung parenchyma by the disease; (ii) pleural effusion; and (iii) large involvement of at least one upper lobe or of the middle lobe. Using this qualitative model may facilitate the assessment of the chest CT by the radiologist by avoiding the cumbersome calculation of the total semi-quantitative CT score. Our results should be confirmed by further larger scale-studies with a more balanced data between the two groups.
Human rights
The authors declare that the work described has been carried out in accordance with the Declaration of Helsinki of the World Medical Association revised in 2013 for experiments involving humans.
Informed consent and patient details
The authors declare that this report does not contain any personal information that could lead to the identification of the patients.
The authors declare that they obtained an informed consent from the patients included in the article. The authors also confirm that the personal details of the patients have been removed.
Disclosure of interest
The authors declare that they have no competing interest.
Funding
This work did not receive any grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author contributions
All authors attest that they meet the current International Committee of Medical Journal Editors (ICMJE) criteria for Authorship.
References
- 1.Huang C., Wang Y., Li X., Ren L., Zhao J., Hu Y. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395:497–506. doi: 10.1016/S0140-6736(20)30183-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Guan W.-J., Ni Z.-Y., Hu Y., Liang W.-H., Ou C.-Q., He J.-X. Clinical characteristics of Coronavirus Disease 2019 in China. N Engl J Med. 2020;382:1708–1720. doi: 10.1056/NEJMoa2002032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wu Z., McGoogan J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72,314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323:1239–1242. doi: 10.1001/jama.2020.2648. [DOI] [PubMed] [Google Scholar]
- 4.Denys A., Guiu B., Chevallier P., Digklia A., de Kerviler E., de Baere T. Interventional oncology at the time of COVID-19 pandemic: problems and solutions. Diagn Interv Imaging. 2020;101:347–353. doi: 10.1016/j.diii.2020.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Soyer P. Lessons learned from chest CT in COVID-19. Diagn Interv Imaging. 2020;101:261–262. doi: 10.1016/j.diii.2020.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Barral M., Dohan A., Marcelin C., Carteret T., Zurlinden O., Pialat J.-B. COVID-19 pandemic: a stress test for interventional radiology. Diagn Interv Imaging. 2020;101:333–334. doi: 10.1016/j.diii.2020.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ai T., Yang Z., Hou H., Zhan C., Chen C., Lv W. Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology. 2020;296:E32–E40. doi: 10.1148/radiol.2020200642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Fang Y., Zhang H., Xie J., Lin M., Ying L., Pang P. Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Radiology. 2020;296:E115–E117. doi: 10.1148/radiol.2020200432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hani C., Trieu N.H., Saab I., Dangeard S., Bennani S., Chassagnon G. COVID-19 pneumonia: a review of typical CT findings and differential diagnosis. Diagn Interv Imaging. 2020;101:263–268. doi: 10.1016/j.diii.2020.03.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kim H., Hong H., Yoon S.H. Diagnostic performance of CT and reverse transcriptase-polymerase chain reaction for Coronavirus disease 2019: a meta-analysis. Radiology. 2020;296:E145–E155. doi: 10.1148/radiol.2020201343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Li J., Xi L., Wang X., Fang F., Lv X., Zhang D. Radiology indispensable for tracking COVID-19. Diagn Interv Imaging. 2020 doi: 10.1016/j.diii.2020.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Feng Z., Yu Q., Yao S., Luo L., Zhou W., Mao X. Early prediction of disease progression in COVID-19 pneumonia patients with chest CT and clinical characteristics. Nat Commun. 2020;11:4968. doi: 10.1038/s41467-020-18786-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Yu Y., Wang X., Li M., Gu L., Xie Z., Gu W. Nomogram to identify severe coronavirus disease 2019 (COVID-19) based on initial clinical and CT characteristics: a multi-center study. BMC Med Imaging. 2020;20:111. doi: 10.1186/s12880-020-00513-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Liu F., Zhang Q., Huang C., Shi C., Wang L., Shi N. CT quantification of pneumonia lesions in early days predicts progression to severe illness in a cohort of COVID-19 patients. Theranostics. 2020;10(12):5613–5622. doi: 10.7150/thno.45985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Durhan G., Ardalı Düzgün S., Başaran Demirkazık F., Irmak İ., İdilman İ., Gülsün Akpınar M. Visual and software-based quantitative chest CT assessment of COVID-19: correlation with clinical findings. Diagn Interv Radiol. 2020;26:557–564. doi: 10.5152/dir.2020.20407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pan F., Ye T., Sun P., Gui S., Liang B., Li L. Time course of lung changes at chest CT during recovery from coronavirus disease 2019 (COVID-19) Radiology. 2020;295:715–721. doi: 10.1148/radiol.2020200370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lyu P., Liu X., Zhang R., Shi L., Gao J. The performance of chest CT in evaluating the clinical severity of COVID-19 pneumonia: identifying critical cases based on CT characteristics. Invest Radiol. 2020;55:412–421. doi: 10.1097/RLI.0000000000000689. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Francone M., Iafrate F., Masci G.M., Coco S., Cilia F., Manganaro L. Chest CT score in COVID-19 patients: correlation with disease severity and short-term prognosis. Eur Radiol. 2020;30:6808–6817. doi: 10.1007/s00330-020-07033-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wang X., Hu X., Tan W., Mazzone P., Mireles-Cabodevila E., Han X.-Z. Multi-center study of temporal changes and prognostic value of a CT visual severity score in hospitalized patients with COVID-19. AJR Am J Roentgenol. 2020 doi: 10.2214/AJR.20.24044. [DOI] [PubMed] [Google Scholar]
- 20.Bernheim A., Mei X., Huang M., Yang Y., Fayad Z.A., Zhang N. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology. 2020;295:200463. doi: 10.1148/radiol.2020200463. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Khosravi B., Aghaghazvini L., Sorouri M., Naybandi Atashi S., Abdollahi M., Mojtabavi H. Predictive value of initial CT scan for various adverse outcomes in patients with COVID-19 pneumonia. Heart Lung J Crit Care. 2020 doi: 10.1016/j.hrtlng.2020.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Liao X., Wang B., Kang Y. Novel coronavirus infection during the 2019–2020 epidemic: preparing intensive care units-the experience in Sichuan Province, China. Intensive Care Med. 2020;46:357–360. doi: 10.1007/s00134-020-05954-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hansell D.M., Bankier A.A., MacMahon H., McLoud T.C., Müller N.L., Remy J. Fleischner Society: glossary of terms for thoracic imaging. Radiology. 2008;246:697–722. doi: 10.1148/radiol.2462070712. [DOI] [PubMed] [Google Scholar]
- 24.Jalaber C., Lapotre T., Morcet-Delattre T., Ribet F., Jouneau S., Lederlin M. Chest CT in COVID-19 pneumonia: a review of current knowledge. Diagn Interv Imaging. 2020;101:431–437. doi: 10.1016/j.diii.2020.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Soyer P. Agreement and observer variability. Diagn Interv Imaging. 2018;99:53–54. doi: 10.1016/j.diii.2018.01.009. [DOI] [PubMed] [Google Scholar]
- 26.DeLong E.R., DeLong D.M., Clarke-Pearson D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845. [PubMed] [Google Scholar]
- 27.Colombi D., Bodini F.C., Petrini M., Maffi G., Morelli N., Milanese G. Well-aerated lung on admitting chest CT to predict adverse outcome in COVID-19 Pneumonia. Radiology. 2020;296:E86–E96. doi: 10.1148/radiol.2020201433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Zhao W., Zhong Z., Xie X., Yu Q., Liu J. Relation between chest CT findings and clinical conditions of coronavirus disease (COVID-19) pneumonia: a multi-center study. AJR Am J Roentgenol. 2020;214:1072–1077. doi: 10.2214/AJR.20.22976. [DOI] [PubMed] [Google Scholar]
- 29.Mahdjoub E., Mohammad W., Lefevre T., Debray M.P., Khalil A. Admission chest CT score predicts 5-day outcome in patients with COVID-19. Intensive Care Med. 2020;46:1648–1650. doi: 10.1007/s00134-020-06118-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Nannan Panday R.S., Minderhoud T.C., Alam N., Nanayakkara P.W.B. Prognostic value of early warning scores in the emergency department and acute medical unit: a narrative review. Eur J Intern Med. 2017;45:20–31. doi: 10.1016/j.ejim.2017.09.027. [DOI] [PubMed] [Google Scholar]
- 31.Gidari A., De Socio G.V., Sabbatini S., Francisci D. Predictive value of National Early Warning Score 2 (NEWS2) for intensive care unit admission in patients with SARS-CoV-2 infection. Infect Dis. 2020;52:698–704. doi: 10.1080/23744235.2020.1784457. [DOI] [PubMed] [Google Scholar]
- 32.Kostakis I., Smith G.B., Prytherch D., Meredith P., Price C., Chauhan A. The performance of the National Early Warning Score and National Early Warning Score 2 in hospitalised patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Resuscitation. 2020 doi: 10.1016/j.resuscitation.2020.10.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Zhang J.-J., Dong X., Cao Y.-Y., Yuan Y.-D., Yang Y.-B., Yan Y.-Q. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy. 2020;75:1730–1741. doi: 10.1111/all.14238. [DOI] [PubMed] [Google Scholar]
- 34.Grillet F., Behr J., Calame P., Aubry S., Delabrousse E. Acute pulmonary embolism associated with COVID-19 pneumonia detected with pulmonary CT angiography. Radiology. 2020;296:E186–E188. doi: 10.1148/radiol.2020201544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Léonard-Lorant I., Delabranche X., Séverac F., Helms J., Pauzet C., Collange O. Acute pulmonary embolism in patients with COVID-19 at CT angiography and relationship to D-dimer levels. Radiology. 2020;296:E189–E191. doi: 10.1148/radiol.2020201561. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Le Berre A., Marteau V., Emmerich J., Zins M. Concomitant acute aortic thrombosis and pulmonary embolism complicating COVID-19 pneumonia. Diagn Interv Imaging. 2020;101:321. doi: 10.1016/j.diii.2020.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cellina M., Oliva G. Acute pulmonary embolism in a patient with COVID-19 pneumonia. Diagn Interv Imaging. 2020;101:325–326. doi: 10.1016/j.diii.2020.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]