Abstract
Purpose
Critically ill COVID-19 patients have an increased risk of developing pulmonary embolism (PE). Diagnosis of PE by point-of-care ultrasound (POCUS) might reduce the need for computed tomography pulmonary angiography (CTPA), while decreasing time-to-diagnosis.
Materials & methods
This prospective, observational study included adult ICU patients with COVID-19. Multi-organ (lungs, deep vein, cardiac) POCUS was performed within 24 h of CTPA, looking for subpleural consolidations, deep venous thrombosis (DVT), and right ventricular strain (RVS). We reported the scan time, and calculated diagnostic accuracy measures for these signs separately and in combination.
Results
70 consecutive patients were included. 23 patients (32.8%) had a PE. Median scan time was 14 min (IQR 11–17). Subpleural consolidations' diagnostic accuracy was: 42.9% (95%CI [34.1–52.0]). DVT's and RVS' diagnostic accuracy was: 75.6% (95%CI [67.1–82.9]) and 74.4% (95%CI [65.8–81.8]). Their sensitivity was: 24.0% (95%CI [9.4–45.1]), and 40.0% (95%CI [21.3–61.3]), while their specificity was: 88.8% (95%CI [80.8–94.3]), and: 83.0% (95%CI [74.2–89.8]), respectively. Multi-organ POCUS sensitivity was: 87.5% (95%CI [67.6–97.3]), and specificity was: 25% (95%CI [16.9–34.7]).
Conclusions
Multi-organ rather than single-organ POCUS can be of aid in ruling out PE in critically ill COVID-19 and help select patients for CTPA. In addition, finding RVS can make PE more likely, while a DVT would preclude the need for a CTPA.
Registration: www.trialregister.nl: NL8540.
Keywords: Point-of-care ultrasound, POCUS, COVID-19, ICU, Pulmonary embolism, Diagnostics
Abbreviations: APACHE II, Acute Physiology and chronic Health Evaluation II; aPTT, activated prothrombin time; ARDS, acute respiratory distress syndrome; BMI, Body Mass Index; CI, confidence interval; COVID-19, coronavirus Disease 2019; CRP, C-reactive protein; CT, computed tomography; CTPA, computed tomography pulmonary angiography; CON1, 1 subpleural consolidation of ≥1 cm (probable criterium); CON2, ≥2 subpleural consolidations of ≥1 cm (high likelihood criterium); CVC, central venous catheter; DVT, deep venous thrombosis; EtCO2, end-tidal carbon dioxide; FiO2, Fraction of inspired oxygen; Hs, high sensitivity; ICU, intensive care unit; IQR, inter-quartile range; IU, international units; kPa, kilopascal; L, liter; LDH, lactate dehydrogenase; LV, left ventricle; NLR, negative likelihood ratio; NT-pro BNP, N-terminal pro b-type natriuretic peptide; NPV, negative predictive value; PaO2, partial pressure of arterial oxygen; PaO2/FiO2 (P/F) ratio, ratio of arterial oxygen partial pressure to fractional inspired oxygen); PE, pulmonary embolism; PEEP, positive end-expiratory pressure; PLR, positive likelihood ratio; POCUS, point-of-care ultrasound; PPV, positive predictive value; SD, standard deviation; SOFA, Sequential Organ Failure Assessment; RV, right ventricle; RVS, right ventricular strain; US-, ultrasound negative; US+, ultrasound positive
1. Introduction
Clinical manifestations of Coronavirus Disease 2019 (COVID-19) extend beyond that of a pneumonia [1]. Critically ill COVID-19 patients have an increased risk of developing pulmonary embolism (PE), compared to hospitalized and intensive care unit (ICU) patients with other (respiratory) infections [[2], [3], [4], [5]]. Computed tomography pulmonary angiography (CTPA) is the gold standard to diagnose PE. However, transport outside of the controlled ICU environment carries substantial risks for patients, while necessitating post-transport decontamination [6,7] It also requires more personnel and reduces time that can be spent on patient care.
Point-of-care ultrasound (POCUS) might reduce the need for CTPAs in these patients in already overwhelmed acute and critical health care systems worldwide, particularly in low- and middle-income [[8], [9], [10]]. Recent consensus statements recommend the use of POCUS as an initial test in critically ill COVID-19 patients [3,5,11,12]. However literature on POCUS in the detection of PE in these patients is still limited.
Studies in non-COVID-19 settings have shown that lung POCUS – by demonstrating subpleural consolidations – has good diagnostic accuracy for detecting PE [[13], [14], [15]]. However, since subpleural consolidations are also prominent sonographic features in COVID-19 and acute respiratory distress syndrome (ARDS), lung POCUS might be less accurate in detecting PE in critically ill COVID-19 patients [16,17]. To date, studies in COVID-19 patients are scarce, although a small case series showed promising results [18].
Compression ultrasound of the deep veins of the lower extremities (deep vein POCUS) has demonstrated excellent accuracy in detecting deep venous thrombosis (DVT) – a proxy for PE – across a myriad of settings, including this pandemic [[19], [20], [21], [22]]. Furthermore, cardiac POCUS may help identify pulmonary embolism (PE) indirectly by the detection of new right ventricular strain (RVS), which has high specificity for PE. Cardiac POCUS has been reported to be as reliable as formal echocardiography in detecting RVS [23]. However, RVS is found in 22–50% of patients with ARDS, which could limit its use in COVID-19 [19,24].
Combining different POCUS modalities (lung, deep veins, and cardiac) to detect pulmonary embolism has been advocated as well [3,5,11,12,25]. In the emergency department (ED) setting there are promising results of multi-organ POCUS in the detection of PE [7,10,26], while others found no benefit [27]. However, there have been no studies in critically ill and/or COVID-19 patients.
Therefore, our aim was to investigate the diagnostic accuracy of multi-organ POCUS – lung, deep vein and cardiac – separately, and in combination in the detection of PE in critically ill COVID-19 patients.
2. Materials & methods
2.1. Study design and setting
This is a prospective, observational study conducted at the academic adult ICU of the Amsterdam UMC, location VUmc, Amsterdam, The Netherlands, between October 20th 2020 and February 20th, 2021. Patients were followed up until May 20th, 2021. The study, and use of data gathered during routine ultrasound and CTPA, was approved by the Medical Ethical Committee of the VUmc, along with a waiver of informed consent (2020.011). The trial was registered in the Dutch trial registry (NL8540).
2.2. Patients
Consecutive adult patients (≥18 years) with a laboratory confirmed diagnosis of COVID-19 were eligible for inclusion upon admission to ICU. Patients were included when the multi-organ POCUS examination (index test) was performed within 24 h of a CTPA (reference test). They could be included again if a CTPA was repeated. Patients were excluded if they were already on therapeutic anticoagulation, transferred to another hospital within 24 h of CTPA, or died within 24 h of CTPA. Baseline characteristics along with ventilator settings and laboratory values were collected from the electronic patient record closest to the time of CTPA. Please see the Supplemental Material for the diagnostic and treatment protocol on our ICU.
2.3. Medical work up
As part of the departmental protocol, patients received a CT(PA) when their clinical condition did not improve or deteriorated approximately every 7 days to diagnose PE, to determine the extent of pulmonary involvement, and possible superinfection. If a PE was diagnosed, a CTPA was typically not repeated. A normal CT could still be repeated though to monitor the degree of pulmonary involvement or detect a possible superinfection. As commonly used clinical decision rules (i.e., WELLS, GENEVA or YEARS-criteria) are not validated in the ICU nor in the COVID-19 setting [[28], [29], [30], [31]], the decision to perform a CTPA was made by the treating ICU-consultant in consultation with a daily multidisciplinary team consisting of a consultant microbiologist, a consultant pulmonologist and at least three other ICU consultants. Please see the Supplemental Material for the full diagnostic and treatment protocol on our ICU.
2.4. Index test: Point-of-care ultrasound (POCUS)
POCUS is part of the standard care on our ICU. All examinations were performed or supervised by experienced ultrasound physicians using a COVID-19 unit-restricted SonoSite-Edge II ultrasound machine [32]. They were blinded for CTPA result, but not for the clinical data. Lastly, the scan time was recorded for all investigations.
2.5. Pulmonary embolism
2.5.1. Lung POCUS: subpleural consolidations
Pulmonary embolism has been associated with subpleural consolidations on lung ultrasound [13,33]. A 10–5 MHz linear transducer or 5–3 MHz curvilinear transducer with lung examination setting was used. The lung ultrasound protocol consisted of a structured assessment of 6 zones of each hemithorax [34]. The recently modified criteria from Mathis et al. were used to diagnose PE in COVID-19 patients [14,18]. Likelihood of PE was determined to be:
-
•
High: when two or more subpleural consolidations (≥1 cm) were detected (Fig. 1, Video 1);
-
•
Probable: when only one subpleural consolidation (≥1 cm) was detected;
-
•
Possible: when two (or more) subpleural consolidations (<1 cm) were detected (Fig. 2, Video 2);
-
•
Low: when no consolidations were detected.
Fig. 1.
Subpleural consolidation ≥1 cm.
Fig. 2.
Subpleural consolidation <1 cm.
2.5.2. Deep vein POCUS: Deep venous thrombosis (DVT)
A two-point compression ultrasound of the common femoral and popliteal vein was performed, using a 10–5 MHz linear transducer on venous setting following a previously described protocol (Supplementary Video 3, Supplementary Video 4) [35,36]. If an indwelling central venous catheter (CVC) was present, it was also examined as far as possible after skin insertion. Deep venous thrombosis was defined as a non-compressible vein (Supplementary Video 5, Supplementary Video 6, Supplementary Video 7).
2.5.3. Cardiac POCUS: Right ventricular strain (RVS)
Cardiac POCUS was performed using a 5–2 MHz phased array transducer on cardiac setting from as many cardiac windows as obtainable (i.e., ≥1 parasternal short axis, subcostal short axis, or 4 chamber apical). Right ventricular strain was determined by ‘eye-balling’ interventricular septum flattening/bowing into the left ventricle (‘D-sign’) (Video 8), assessing the right ventricle (RV) to left ventricle (LV) basal end diastolic diameter ratio, and/or McConnell's sign. A RV:LV ratio of ≥1 was deemed abnormal (Video 9) [19,33,35,37]. A visible RV thrombus was also deemed to be diagnostic.
2.5.4. Computed tomography pulmonary angiography (CTPA)
CTPAs were evaluated by our local radiologists with varying degrees of experience, who did have access to the clinical information – but not the POCUS examinations. Pulmonary embolism was defined as a constant intravascular filling defect on CTPA (see Supplemental Material for the scan protocol). Location of a filling defect was registered for each lobe until most distal subsegmental levels. Right ventricular strain (RV/LV ratio ≥ 1) was also registered.
2.6. Statistical analysis
Baseline characteristics and outcome variables were presented as means ± standard deviations (±SD), medians and interquartile range [IQR], or numbers (percentages %) as appropriate.
We determined diagnostic accuracy measures; sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy for subpleural consolidations, right ventricular strain, and deep venous thrombosis separately and in different combinations. In addition, we determined a receiver operating characteristic (ROC) curve with corresponding area under the curve (AUC) for subpleural consolidations ≥1 cm. We established the optimal cut-off with the Youden index. Missing data were handled by pairwise deletion. McNemar's tests were used to compare sensitivities and specificities of each single-organ modality with multi-organ POCUS. Missing data were handled by pairwise deletion. Statistical analyses were performed using SPSS IBM version 24.0 (SPSS Inc., Chicago, IL, USA).
3. Results
140 CTPAs were made in 70 consecutive patients, with a median of 2 scans per patient [IQR 1–3]. In 11 instances multi-organ POCUS examinations data was missing because the scan was not conducted withing 24 h of the CT scan. In 2 patients all of the POCUS scans were incomplete or could not performed, this was also the case for 1 additional lung POCUS, 4 deep vein POCUS and 2 cardiac POCUS (Fig. 3 ). This was mainly due to the positioning of the patient (i.e., prone or lateral position). Patient baseline characteristics and outcomes are shown in Table 1 . 62 Patients (88.6%) required mechanical ventilation during admission. 66 Patients (94.2%) developed moderate or severe ARDS. No patients had a previous history of RVS. 23 Patients (32.8%) had a PE, of which 12 (52.2%) were subsegmental. Median scan time of a full multi-organ POCUS exam was 14 min [IQR 11–17].
Fig. 3.
Study participant flow chart.
Table 1.
Baseline characteristics at admission by patient.
Patients | |
---|---|
Demographics | |
Age (years) | 67.5 [60–75] |
Sex (male) | 56 (80) |
BMI (kg/m2) | 28.6 (5.0) |
APACHE II | 12 [10−13] |
SOFA score | 7.6 (2.9) |
Charlson Comorbidity Index | 3 (1.8) |
Laboratory parameters | |
Creatinine (umol/L) | 79 [64–104] |
Leucocytes (x109/L) | 10.5 (4.7) |
CRP (mg/L) | 128 [83–188] |
Procalcitonin (ug/L) | 0.35 [0.15–1.05] |
LDH (U/L) | 505 [399–648] |
Hs Troponin T (ng/L) | 23 [12.5–54] |
NT-pro BNP (ng/L) | 642 [272–903] |
aPTT (sec) | 25 [23–28] |
D-dimer (ng/ml) | 2.21 [1.18–6.30] |
Ventilation parameters | |
PEEP (cm H2O) | 11 (2.3) |
PaO2/FiO2 ratio | 106.3 [75.8–147.5] |
etCO2 gap (kPa) | 1.76 (0.92) |
Outcomes | |
ICU length of stay (days) | 14 [6–31.5] |
Mechanical ventilation (days) | 14.5 [6–31] |
28-day Mortality | 25 (35.7) |
90-day Mortality | 28 (40.0) |
Values are n (%), mean(±SD), or median [IQR] as appropriate.
APACHE II: Acute Physiology and chronic Health Evaluation II; aPTT: activated prothrombin time; BMI: Body Mass Index; CRP: C-reactive protein; EtCO2: end-tidal carbon dioxide; FiO2: Fraction of inspired oxygen; Hs: high sensitivity; IQR: inter-quartile range; ICU: intensive care unit; IU: international units; kPa: kilopascal; L: liter; LDH: lactate dehydrogenase; NT-pro BNP: N-terminal pro b-type natriuretic peptide; PE: pulmonary embolism; PEEP: positive end-expiratory pressure; paO2: partial pressure of arterial oxygen; PaO2/FiO2 (P/F) ratio: ratio of arterial oxygen partial pressure to fractional inspired oxygen); SD: standard deviation; SOFA: Sequential Organ Failure Assessment.
3.1. Detection of pulmonary embolism
3.1.1. Single-organ POCUS
In Table 2 diagnostic accuracy measures are shown for the different POCUS signs of PE. For the high likelihood criterium for PE (≥2 subpleural consolidations of ≥1 cm) sensitivity was 70.8% (95%CI 48.9–87.4) and specificity 35.0% (95%CI 25.7–45.2). 82 Patients (62.6%) had high PE likelihood criterium, of which 65 (79.2%) were falsely positive.
Table 2.
2x2 contingency tables & diagnostic accuracy measures for POCUS signs of pulmonary embolism.
POCUS | CT PE + |
PE - | Sensitivity % (95% CI) | Specificity % (95% CI) | NLR (95% CI) | PLR (95% CI) | NPV % (95% CI) | PPV % (95% CI) | Diagnostic Accuracy % (95% CI) | |
---|---|---|---|---|---|---|---|---|---|---|
CON2 | US+ | 17 | 65 | 70.8 (48.9–87.4) | 36.3 (25.7–45.2) | 0.80 (0.41–1.58) | 1.11 (0.83–1.49) | 84.1 (72.9–91.2) | 20.7 (16.3–26.0) | 42.9 (34.1–52.0) |
US- | 7 | 37 | ||||||||
CON1 | US+ | 2 | 22 | 8.7 (1.1–28.4) | 78.6 (69.5–86.1) | 1.16 (0.99–1.36) | 0.41 (0.10–1.61) | 79.4 (76.7–81.9) | 8.3 (2.3–26.5) | 65.9 (56.9–74.1) |
US- | 21 | 81 | ||||||||
DVT | US+ | 6 | 11 | 24.0 (9.4–45.1) | 88.8 (80.8–94.3) | 0.86 (0.68–1.08) | 2.14 (0.88–5.22) | 82.1 (78.4–85.2) | 35.3 (18.3–57.1) | 75.6 (67.1–82.9) |
US- | 19 | 87 | ||||||||
RVS | US+ | 10 | 17 | 40.0 (21.3–61.3) | 83.0 (74.2–89.8) | 0.72 (0.52–1.01) | 2.35 (1.23–4.49) | 84.7 (79.9–88.5) | 37.0 (23.6–52.9) | 74.4 (65.8–81.8) |
US- | 15 | 83 | ||||||||
RVS or DVT | US+ | 14 | 26 | 56.0 (34.9–75.6) | 73.5 (63.6–81.9) | 0.60 (0.38–0.95) | 2.11 (1.31–3.41) | 86.8 (80.6–91.2) | 35.0 (25.0–46.5) | 69.9 (61.0–77.9) |
US- | 11 | 72 | ||||||||
RVS or DVT or CON2 | US+ | 21 | 75 | 87.5 (67.6–97.3) | 25.0 (16.9–34.7) | 0.5 (0.16–1.52) | 1.17 (0.97–1.41) | 89.3 (73.3–96.2) | 21.9 (18.8–25.3) | 37.1 (28.6–46.2) |
US- | 3 | 25 | ||||||||
[RVS or DVT] & CON2 | US+ | 10 | 16 | 43.5 (23.2–65.5) | 83.3 (74.4–90.2) | 0.68 (0.47–0.98) | 2.62 (1.37–4.98) | 86.0 (81.0–89.9) | 38.5 (24.7–54.4) | 75.6 (66.9–83.0) |
US- | 13 | 80 |
CI: confidence interval; CT: computed tomography; CON1: 1 subpleural consolidation of ≥ 1 cm (probable criterium); CON2: ≥2 subpleural consolidations of ≥ 1 cm (high likelihood criterium); DVT: deep venous thrombosis; NLR: negative likelihood ratio; NPV: negative predictive value; PLR: positive likelihood ratio; PE: pulmonary embolism; POCUS: point-of-care ultrasound; PPV: positive predictive value; RVS: right ventricular strain; US-: ultrasound negative; US+: ultrasound positive.
For the probable PE criterium (1 subpleural consolidation of ≥1 cm) sensitivity was 8.7% (95%CI 1.1–28.4), and specificity 78.6% (95%CI 69.5–86.1). Employing the possible or low likelihood criteria did not result in improved diagnostic accuracy measures.
The ROC curve for subpleural consolidations ≥1 cm showed an AUC of 0.58 (95%CI [0.45–0.72]) with an optimal cut-off of 4 subpleural consolidations (Youden 0.29) (Fig. 4 ). At this cut-off sensitivity was 60.9% and specificity 66.0%, with a PLR of 1.79 and NLR of 0.59.
Fig. 4.
ROC-curve of subpleural consolidations and pulmonary embolism.
DVT found by POCUS had a sensitivity of 24.0% (95%CI [9.4–45.1]), but the highest specificity of all findings: 88.8% (95%CI [80.8–94.3]). For RVS on cardiac POCUS sensitivity was 40.0% (95%CI [21.3–61.3]) and a specificity 83.0% (95%CI [74.2–89.8]). A visible RV thrombus was detected in 0% of cases.
3.1.2. Multi-organ POCUS
Signs of different POCUS modalities were combined to determine if this would increase diagnostic accuracy. For a positive multi-organ POCUS – so either a positive high PE likelihood criterium (CON2), DVT, or RVS – sensitivity was highest 87.5% (95%CI [67.6–97.3]), but specificity was lowest 25% (95%CI [16.9–34.7]). A high PE likelihood criterium (CON2) combined with either a DVT or RVS resulted in the highest PPV and diagnostic accuracy. McNemar's tests for sensitivity and specificity of all modalities compared to multi-organ POCUS are shown in Supplemental Table 1. No combination of DVT or RVS with the probable likelihood criterium (CON1) improved diagnostic accuracy measures.
4. Discussion
The main findings of this prospective, observational study in adult critically ill COVID-19 patients are that: 1) Single-organ POCUS modalities alone have low sensitivity and therefore are not helpful in ruling out PE; 2) Subpleural consolidations are hardly useful in the detection of PE; 3) DVT has good specificity for PE and its detection alone obviates the need for CTPA; 4) RVS has similar diagnostic accuracy to other settings, despite concomitant ARDS or mechanical ventilation; 5) Multi-organ POCUS however has good sensitivity for PE and may have a role in ruling out PE.
4.1.1. Single-organ POCUS
4.1.1.1. Subpleural consolidations
We found rather poor diagnostic accuracy for subpleural consolidations in the detection of PE in critically ill COVID-19 patients, regardless of the criterium used or the number of subpleural consolidations. In contrast, the latest systematic reviews in the non-COVID-19 setting showed diagnostic accuracy of lung ultrasound was good in studies where the diagnostic threshold was at least one subpleural consolidation; with a sensitivity of 81.4–87%, and a specificity of 81.8–87.4% [13,33]. Studies using a diagnostic threshold of at least two subpleural consolidations had a worse sensitivity 44.2%, but better specificity 96.5% [33]. Most included studies were conducted in ambulatory patients [[14], [15],33,[39], [40]], and the authors noted a high potential of selection bias, limiting generalization of those results to critically ill COVID-19 patients.
Our results might be less remarkable when we revisit the pathophysiology of subpleural consolidations. Subpleural consolidations originate from the depletion of air in the lung. This may be caused by infarction, atelectasis or inflammation, and as such can be found in PE, COVID-19 pneumonia and ARDS [15,16,41,42]. This overlap is probably what caused the poor diagnostic accuracy in this particular setting. In addition, 52.2% of PEs found in our study were subsegmental. Outside the critically ill COVID-19 setting sensitivity of lung POCUS also drops from 89% – in central, lobar, and segmental PE – to 21% in subsegmental PE [13].
Contrarily, in early reports, one group have argued that since thrombi are often located distally, and obscured by consolidation they often cannot be reliably visualized by CTPA, while lung POCUS actually does pick-up those PEs [18,43]. However, there is debate about what the clinical significance of such small emboli is, and if they should warrant detection or treatment [44,45]. PE – found by CTPA – has been associated with a higher mortality in COVID-19 [4]. The REMAP-CAP trial showed no benefit of therapeutic anticoagulation in critically ill patients with COVID-19, while the majority of those patients will probably have subpleural consolidations [46].
4.1.1.2. DVT
The prevalence of PE (32.8%) and DVT (21.4%) found in our study was comparable with the prevalence found in other screening studies in COVID-19 patients [4,47,48].The pathophysiology of PE in COVID-19 is still not entirely clear. In addition to ‘classic’ PE, it is probably multifactorial as COVID-19 patients exhibit all components of Virchow's triad: an immune-mediated hypercoagulable state, endothelial injury, and stasis/turbulence of blood flow (due to immobilization, hypoxemia induced vasoconstriction, and further exacerbated by the insertion of CVCs) [3,4,22,[47], [48], [49]]. Indeed, PEs are associated with lower limb DVT in only 10–13.6% of COVID-19 patients, while this is 56–61% in undifferentiated cohorts, explaining the lower sensitivity [7,50,51]. Specificity is good though, indicating that finding a DVT can make the diagnosis of PE a little more likely, but its absence cannot rule it out.
On the other hand, detecting a DVT by POCUS would preclude the necessity of a CTPA, since therapeutic anticoagulation should be started regardless according to international guidelines [33,52]. Systematic DVT scanning by POCUS could save costs, time, and potentially lives given the increased mortality risk with concomitant venous thromboembolism [4]. Meanwhile it would circumvent the downsides of having to take a potentially contagious and unstable patient to and from radiology for a CTPA or formal DVT scan. In line with current guidelines, we therefore advocate its use, as a DVT scan takes less than 2 min to perform, and results are just as reliable as a formal radiology exam [3,4,19,21].
4.1.1.3. RVS
As is the case with subpleural consolidations, RVS is not a direct marker of PE. It is a proxy for clot burden and cardio-pulmonary reserve [19,23]. The sensitivity and specificity found for RVS in our study are comparable to what is found for different features of RVS (i.e., RV:LV ratio ≥ 1, ‘D-sign’, and McConnell's sign) in the non-COVID-19 literature; 40% versus 24–54%, and 83% versus 87–98.6% respectively [19,23,33]. This supports the premise that RVS might help rule PE in, but cannot rule it out [19,23,33]. This could be explained by the fact that (COVID-19 related) ARDS itself can cause RVS [19,24,53]. Mechanical ventilation can also be of influence, but this was deemed less likely as we adhered to stringent lung protective mechanical ventilation protocols [24,54]. In order to discriminate between different causes of RVS – like PE or (COVID-19 related) ARDS – it might be prudent to perform scans at baseline and regular intervals. We hypothesize that a sudden increase in RV:LV ratio without changes in respiratory mechanics might be associated with PE, but further research is required before drawing definitive conclusions.
4.1.2. Multi-organ POCUS
The sensitivity found for multi-organ POCUS: 87.5% is similar to that in the non-COVID ED setting: 90% [7,33,55], and cardiac-deep vein POCUS alone: 91% [10]. Our results suggest that multi-organ POCUS has a role in ruling out PE, or at least have the potential to reduce the amount of CTPA's. The main driver of the good sensitivity seems to be subpleural consolidations, which as mentioned before seem to be ubiquitous in critically ill COVID-19 patients. The high likelihood PE criterium (CON2) was found in almost two-thirds of patients and falsely positive in >80% of them. This causes specificity to drop well below the values found in a non-COVID-19 ED setting: 25% versus 86.2%. The specificity for solely cardiac-deep vein POCUS was comparable though 73.5% and 81% [7,10].
4.2. Global perspective
COVID-19 has laid bare health disparities along socio-economic, racial, cultural, and ethnic lines within and across regions and nations. CTPA is not always readily available in low- and middle-income countries, especially when resources are overwhelmed. POCUS is an affordable and easy-to-use tool, which could reduce barriers to timely adequate care [10]. Two descriptive studies suggest that POCUS could reduce >50% of CTPAs [33]. The emergence of handheld devices has made POCUS even more accessible. A complete multi-organ POCUS scan takes less than 15 min. Moreover, it is easy to learn for (para)medical personnel [56,57]. In our study, all POCUS scans were performed by ICU residents. Considering that non-specialist physicians see and treat the bulk of COVID-19 patients, this underscores the applicability of POCUS in a real-life setting.
4.3. Limitations and strengths
First, this was a single center study with a relatively limited sample size. Still, this is the first and largest prospective study of consecutive COVID-19 patients to investigate the use of multi-organ POCUS in the detection of PE. Furthermore, our sample size was similar to those of most included studies in the aforementioned systematic reviews on VTE [4,47,48]. Second, although we are a tertiary center, our case mix was reflective of the Dutch COVID-19 ICU population since ICU patients were divided across the country according to a fair share principle. As such selection bias was minimal. Third, our way of assessing RVS combines the features most commonly reported in the literature [33]. We decided to use parameters that interpreters would be able to ‘eye-ball’ as we believed this is simpler, quicker and requires less skill than performing exact measurements (i.e., tricuspid annular plane systolic excursion [TAPSE], or measuring RV:LV ratio), while having similar diagnostic traits [19,23,33]. We thought our results would be applicable to a larger group of operators this way. Fourth, diagnostic accuracy studies on multi-organ POCUS in PE often suffer from suboptimal reporting [33]. Mainly information on the blinding of POCUS operators and the time interval between POCUS and the reference test are rarely reported. In our study, POCUS operators were always blinded from the CTPA results, which minimizes information bias. Time between POCUS and CT was always below 24 h, so we believe any imprecision of the diagnostic accuracy measures was therefore curtailed. Additionally, our reference standard is the current gold standard in diagnosing PE. Our strict diagnostic protocol minimized the chance of missing any PEs, thereby reducing the risk of negatively influencing the diagnostic accuracy measures.
4.4. Future research
Future prospective investigations or randomized controlled trials should be conducted to analyze the impact of multi-organ POCUS in cost-effectiveness, reduced radiation exposure, and time to definitive diagnosis of PE. Further research should also focus on aspects that can help distinguish the different etiologies of consolidations like doppler and contrast enhanced ultrasonography [18,[58], [59], [60], [61]]. Additionally, it would be interesting to see if POCUS signs could be used in concert with other parameters (i.e. et-CO2 gap or rapid D-dimer rise), or clinical decision rules [62,63].
5. Conclusion
Multi-organ rather than single-organ POCUS can be of aid in ruling out PE in critically ill COVID-19 and help select patients for CTPA. In addition, finding RVS can make PE more likely in this setting, while a DVT would preclude the need for a CTPA.
The following are the supplementary data related to this article.
Supplementary material
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
CRediT authorship contribution statement
A.W.E. Lieveld: Conceptualization, Methodology, Formal analysis, Data curation, Investigation, Writing – original draft, Writing – review & editing, Visualization. M.L.A. Heldeweg: Investigation, Writing – original draft, Writing – review & editing. J.M. Smit: Methodology, Formal analysis, Investigation, Writing – original draft, Writing – review & editing. M.E. Haaksma: Conceptualization, Methodology, Formal analysis, Data curation, Investigation, Writing – original draft, Writing – review & editing, Visualization. L. Veldhuis: Investigation, Writing – original draft, Writing – review & editing. R.S. Walburgh-Schmidt: Investigation, Writing – original draft, Writing – review & editing. J. Twisk: Methodology, Formal analysis, Data curation, Writing – original draft, Writing – review & editing. P.W.B. Nanayakkara: Supervision, Project administration, Resources, Writing – original draft, Writing – review & editing. L. Heunks: Supervision, Project administration, Resources, Writing – original draft, Writing – review & editing. P.R. Tuinman: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Supervision, Project administration, Resources.
Declaration of Competing Interest
The authors have no conflict of interest nor any financial disclosures.
References
- 1.Wadman M., Couzin-Frankel J., Kaiser J., Matacic C. A rampage through the body. Science. 2020;368:356–360. doi: 10.1126/science.368.6489.356. [DOI] [PubMed] [Google Scholar]
- 2.Bikdeli B., Madhavan M.V., Jimenez D., Chuich T., Dreyfus I., Driggin E., et al. COVID-19 and thrombotic or thromboembolic disease: implications for prevention, antithrombotic therapy, and follow-up: JACC state-of-the-art review. J Am Coll Cardiol. 2020;75:2950–2973. doi: 10.1016/j.jacc.2020.04.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Moores L.K., Tritschler T., Brosnahan S., Carrier M., Collen J.F., Doerschug K., et al. Prevention, diagnosis, and treatment of VTE in patients with coronavirus disease 2019: CHEST guideline and expert panel report. Chest. 2020;158:1143–1163. doi: 10.1016/j.chest.2020.05.559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Malas M.B., Naazie I.N., Elsayed N., Mathlouthi A., Marmor R., Clary B. Thromboembolism risk of COVID-19 is high and associated with a higher risk of mortality: a systematic review and meta-analysis. EClinicalMedicine. 2020;29–30 doi: 10.1016/j.eclinm.2020.100639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.COVID-19 and Pulmonary Embolism - Hematology.org 2021. https://www.hematology.org:443/covid-19/covid-19-and-pulmonary-embolism (accessed April 9, 2021)
- 6.Liew M.F., Siow W.T., Yau Y.W., See K.C. Safe patient transport for COVID-19. Crit Care. 2020;24:94. doi: 10.1186/s13054-020-2828-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Nazerian P., Vanni S., Volpicelli G., Gigli C., Zanobetti M., Bartolucci M., et al. Accuracy of point-of-care multiorgan ultrasonography for the diagnosis of pulmonary embolism. Chest. 2014;145:950–957. doi: 10.1378/chest.13-1087. [DOI] [PubMed] [Google Scholar]
- 8.COVID-19 Map Johns Hopkins Coronavirus Resource Center. 2021. https://coronavirus.jhu.edu/map.html (accessed December 31, 2021).
- 9.New COVID-19 Cases Worldwide. Johns Hopkins Coronavirus Resource Center 2021https://coronavirus.jhu.edu/data/new-cases (accessed October 18, 2021).
- 10.Kagima J., Stolbrink M., Masheti S., Mbaiyani C., Munubi A., Joekes E., et al. Diagnostic accuracy of combined thoracic and cardiac sonography for the diagnosis of pulmonary embolism: a systematic review and meta-analysis. PLoS ONE. 2020;15 doi: 10.1371/journal.pone.0235940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mucha S.R., Dugar S., McCrae K., Joseph D., Bartholomew J., Sacha G.L., et al. Update to coagulopathy in COVID-19: manifestations and management. CCJM. 2020 doi: 10.3949/ccjm.87a.ccc024-up. [DOI] [PubMed] [Google Scholar]
- 12.Hussain A., Via G., Melniker L., Goffi A., Tavazzi G., Neri L., et al. Multi-organ point-of-care ultrasound for COVID-19 (PoCUS4COVID): international expert consensus. Crit Care. 2020;24:702. doi: 10.1186/s13054-020-03369-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Squizzato A., Rancan E., Dentali F., Bonzini M., Guasti L., Steidl L., et al. Diagnostic accuracy of lung ultrasound for pulmonary embolism: a systematic review and meta-analysis. J Thromb Haemost. 2013;11:1269–1278. doi: 10.1111/jth.12232. [DOI] [PubMed] [Google Scholar]
- 14.Mathis G., Blank W., Reissig A., Lechleitner P., Reuss J., Schuler A., et al. Thoracic ultrasound for diagnosing pulmonary embolism: a prospective multicenter study of 352 patients. Chest. 2005;128:1531–1538. doi: 10.1378/chest.128.3.1531. [DOI] [PubMed] [Google Scholar]
- 15.Reissig A., Heyne J.-P., Kroegel C. Sonography of lung and pleura in pulmonary embolism. Chest. 2001;120:1977–1983. doi: 10.1378/chest.120.6.1977. [DOI] [PubMed] [Google Scholar]
- 16.Mojoli F., Bouhemad B., Mongodi S., Lichtenstein D. Lung ultrasound for critically ill patients. Am J Respir Crit Care Med. 2019;199:701–714. doi: 10.1164/rccm.201802-0236CI. [DOI] [PubMed] [Google Scholar]
- 17.Lieveld A., Kok B., Schuit F., Azijli K., Heijmans J., van Laarhoven A., et al. Diagnosing COVID-19 pneumonia in a pandemic setting: lung ultrasound versus CT (LUVCT) A multi-centre, prospective, observational study. ERJ Open. 2020;22 doi: 10.1183/23120541.00539-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Zotzmann V., Lang C.N., Wengenmayer T., Bemtgen X., Schmid B., Mueller-Peltzer K., et al. Combining lung ultrasound and Wells score for diagnosing pulmonary embolism in critically ill COVID-19 patients. J Thromb Thrombolysis. 2020 doi: 10.1007/s11239-020-02323-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Schrift D, Barron K, Arya R, Choe C. The use of POCUS to manage ICU patients with COVID-19. Journal of Ultrasound in Medicine n.d.;n/a. doi: 10.1002/jum.15566. [DOI] [PubMed]
- 20.Kapoor S., Chand S., Dieiev V., Fazzari M., Tanner T., Lewandowski D.C., et al. Thromboembolic events and role of point of care ultrasound in hospitalized Covid-19 patients needing intensive care unit admission. J Intensive Care Med. 2020 doi: 10.1177/0885066620964392. 0885066620964392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Elsenga H.E., Collée A., Rosendaal A.V. Agreement between emergency physicians and radiologists for the diagnosis of deep venous thrombosis with compression ultrasound: a prospective study. Eur J Emerg Med. 2021;28:25–28. doi: 10.1097/MEJ.0000000000000728. [DOI] [PubMed] [Google Scholar]
- 22.Smit J.M., Matta J.E.L., Vink R., Müller M.C.A., Choi K.F., FEHP Baarle, et al. Coronavirus disease 2019 is associated with catheter-related thrombosis in critically ill patients: a multicenter case-control study. Thromb Res. 2021;200:87–90. doi: 10.1016/j.thromres.2021.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fields J.M., Davis J., Girson L., Au A., Potts J., Morgan C.J., et al. Transthoracic echocardiography for diagnosing pulmonary embolism: a systematic review and Meta-analysis. J Am Soc Echocardiogr. 2017;30:714–723.e4. doi: 10.1016/j.echo.2017.03.004. [DOI] [PubMed] [Google Scholar]
- 24.Zochios V., Parhar K., Tunnicliffe W., Roscoe A., Gao F. The right ventricle in ARDS. Chest. 2017;152:181–193. doi: 10.1016/j.chest.2017.02.019. [DOI] [PubMed] [Google Scholar]
- 25.Laursen C.B., Clive A., Hallifax R., Pietersen P.I., Asciak R., Davidsen J.R., et al. European Respiratory Society statement on thoracic ultrasound. Eur Respir J. 2021;57 doi: 10.1183/13993003.01519-2020. [DOI] [PubMed] [Google Scholar]
- 26.Buhumaid R.E., St-Cyr Bourque J., Shokoohi H., Ma I.W.Y., Longacre M., Liteplo A.S. Integrating point-of-care ultrasound in the ED evaluation of patients presenting with chest pain and shortness of breath. Am J Emerg Med. 2019;37:298–303. doi: 10.1016/j.ajem.2018.10.059. [DOI] [PubMed] [Google Scholar]
- 27.Zanobetti M., Scorpiniti M., Gigli C., Nazerian P., Vanni S., Innocenti F., et al. Point-of-care ultrasonography for evaluation of acute dyspnea in the ED. Chest. 2017;151:1295–1301. doi: 10.1016/j.chest.2017.02.003. [DOI] [PubMed] [Google Scholar]
- 28.Stals M.A.M., Kaptein F., Kroft L., Klok F.A., Huisman M.V. Challenges in the diagnostic approach of suspected pulmonary embolism in COVID-19 patients. Postgraduate Medicine. 2021 doi: 10.1080/00325481.2021.1920723. 0:null. [DOI] [PubMed] [Google Scholar]
- 29.Katsios C., Donadini M., Meade M., Mehta S., Hall R., Granton J., et al. Prediction scores do not correlate with clinically adjudicated categories of pulmonary embolism in critically ill patients. Can Respir J. 2014;21:36–42. doi: 10.1155/2014/296161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Girardi A.M., Bettiol R.S., Garcia T.S., Ribeiro G.L.H., Rodrigues É.M., Gazzana M.B., et al. Wells and Geneva scores are not reliable predictors of pulmonary embolism in critically ill patients: a retrospective study. J Intensive Care Med. 2020;35:1112–1117. doi: 10.1177/0885066618816280. [DOI] [PubMed] [Google Scholar]
- 31.Silveira P.C., Ip I.K., Goldhaber S.Z., Piazza G., Benson C.B., Khorasani R. Performance of Wells score for deep vein thrombosis in the inpatient setting. JAMA Intern Med. 2015;175:1112. doi: 10.1001/jamainternmed.2015.1687. [DOI] [PubMed] [Google Scholar]
- 32.Touw H.R.W., Tuinman P.R., Gelissen H.P.M.M., Lust E., Elbers P.W.G. Lung ultrasound: routine practice for the next generation of internists. Neth J Med. 2015;73:100–107. [PubMed] [Google Scholar]
- 33.Falster C., Jacobsen N., Coman K.E., Højlund M., Gaist T.A., Posth S., et al. Diagnostic accuracy of focused deep venous, lung, cardiac and multiorgan ultrasound in suspected pulmonary embolism: a systematic review and meta-analysis. Thorax. 2021 doi: 10.1136/thoraxjnl-2021-216838. [DOI] [PubMed] [Google Scholar]
- 34.Lieveld A.W.E., Kok B., Azijli K., Schuit F.H., Ven P.M., Korte C.L., et al. Assessing COVID-19 pneumonia—clinical extension and risk with point-of-care ultrasound: a multicenter, prospective, observational study. Journal of the American College of Emergency Physicians Open. 2021;2 doi: 10.1002/emp2.12429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Soni. Elsevier; 2015. Point of Care Ultrasound. [Google Scholar]
- 36.Lee J.H., Lee S.H., Yun S.J. Comparison of 2-point and 3-point point-of-care ultrasound techniques for deep vein thrombosis at the emergency department: a meta-analysis. Medicine. 2019;98 doi: 10.1097/MD.0000000000015791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Matthews J.C., McLaughlin V. Acute right ventricular failure in the setting of acute pulmonary embolism or chronic pulmonary hypertension: a detailed review of the pathophysiology, diagnosis, and management. Curr Cardiol Rev. 2008;4:49–59. doi: 10.2174/157340308783565384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Mathis G., Metzler J., Fussenegger D., Sutterlütti G., Feurstein M., Fritzsche H. Sonographic observation of pulmonary infarction and early infarctions by pulmonary embolism. Eur Heart J. 1993;14:804–808. doi: 10.1093/eurheartj/14.6.804. [DOI] [PubMed] [Google Scholar]
- 40.Abootalebi A., Golshani K., Karami M., Masoumi B., Aliasgharlou M. Diagnostic validity of ultrasonography in evaluation of pulmonary thromboembolism. Adv Biomed Res. 2016;5:4. doi: 10.4103/2277-9175.174975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Lieveld A.W.E., Kok B., Schuit F.H., Azijli K., Heijmans J., van Laarhoven A., et al. Diagnosing COVID-19 pneumonia in a pandemic setting: Lung Ultrasound versus CT (LUVCT) A multi-centre, prospective, observational study. ERJ Open Research. 2020:00539–02020. doi: 10.1183/23120541.00539-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Pisani L., Vercesi V., van Tongeren P.S.I., Lagrand W.K., Leopold S.J., Huson M.A.M., et al. The diagnostic accuracy for ARDS of global versus regional lung ultrasound scores - a post hoc analysis of an observational study in invasively ventilated ICU patients. Intensive Care Med Exp. 2019;7 doi: 10.1186/s40635-019-0241-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Mueller-Peltzer K., Krauss T., Benndorf M., Lang C.N., Bamberg F., Bode C., et al. Pulmonary artery thrombi are co-located with opacifications in SARS-CoV2 induced ARDS. Respir Med. 2020;172 doi: 10.1016/j.rmed.2020.106135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.COVID-19 and VTE-Anticoagulation - Hematology.org 2021. https://www.hematology.org:443/covid-19/covid-19-and-vte-anticoagulation (accessed May 10, 2021)
- 45.Bariteau A., Stewart L.K., Emmett T.W., Kline J.A. Systematic review and Meta-analysis of outcomes of patients with subsegmental pulmonary embolism with and without anticoagulation treatment. Acad Emerg Med. 2018;25:828–835. doi: 10.1111/acem.13399. [DOI] [PubMed] [Google Scholar]
- 46.Therapeutic anticoagulation with heparin in critically ill patients with Covid-19. New England Journal of Medicine. 2021;385:777–789. doi: 10.1056/NEJMoa2103417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Longchamp G., Manzocchi-Besson S., Longchamp A., Righini M., Robert-Ebadi H., Blondon M. Proximal deep vein thrombosis and pulmonary embolism in COVID-19 patients: a systematic review and meta-analysis. Thrombosis Journal. 2021;19:15. doi: 10.1186/s12959-021-00266-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Tan B.K., Mainbourg S., Friggeri A., Bertoletti L., Douplat M., Dargaud Y., et al. Arterial and venous thromboembolism in COVID-19: a study-level meta-analysis. Thorax. 2021 doi: 10.1136/thoraxjnl-2020-215383. thoraxjnl-2020-215383. [DOI] [PubMed] [Google Scholar]
- 49.Middeldorp S, Coppens M, Haaps TF van, Foppen M, Vlaar AP, Müller MCA, et al. Incidence of venous thromboembolism in hospitalized patients with COVID-19. J Thromb Haemost 2020;18:1995–2002. doi: 10.1111/jth.14888. [DOI] [PMC free article] [PubMed]
- 50.Cordeanu E.-M., Lambach H., Heitz M., Di Cesare J., Mirea C., Faller A.-M., et al. Pulmonary embolism and coexisting deep vein thrombosis: a detrimental association? J Clin Med. 2019;8 doi: 10.3390/jcm8060899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Franco-Moreno A., Muñoz-Rivas N., Mestre-Gómez B., Torres-Macho J. Pulmonary embolism and COVID-19: a paradigm change. Rev Clin Esp. 2020;220:459–461. doi: 10.1016/j.rceng.2020.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Konstantinides S.V., Meyer G., Becattini C., Bueno H., Geersing G.-J., Harjola V.-P., et al. 2019 ESC guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS): the task force for the diagnosis and management of acute pulmonary embolism of the European Society of Cardiology (ESC) Eur Heart J. 2020;41:543–603. doi: 10.1093/eurheartj/ehz405. [DOI] [PubMed] [Google Scholar]
- 53.Mekontso Dessap A., Boissier F., Charron C., Bégot E., Repessé X., Legras A., et al. Acute cor pulmonale during protective ventilation for acute respiratory distress syndrome: prevalence, predictors, and clinical impact. Intensive Care Med. 2016;42:862–870. doi: 10.1007/s00134-015-4141-2. [DOI] [PubMed] [Google Scholar]
- 54.COVID-19 | NVIC Nederlandse Vereniging voor Intensive Care 2021. https://nvic.nl/covid-19 (accessed May 11, 2021)
- 55.Akturk U.A., Koak N.D., Ernam D. 2017. The role of multi-organ ultrasonography for diagnosing non-massive pulmonary thromboembolism. [Google Scholar]
- 56.Tulleken A.M., Gelissen H., Lust E., Smits T., van Galen T., Girbes A.R.J., et al. UltraNurse: teaching point-of-care ultrasound to intensive care nurses. Intensive Care Med. 2019;45:727–729. doi: 10.1007/s00134-018-05512-x. [DOI] [PubMed] [Google Scholar]
- 57.Arbelot C., Dexheimer Neto F.L., Gao Y., Brisson H., Chunyao W., Lv J., et al. Lung ultrasound in emergency and critically ill patients: number of supervised exams to reach basic competence. Anesthesiology. 2020;132:899–907. doi: 10.1097/ALN.0000000000003096. [DOI] [PubMed] [Google Scholar]
- 58.Hasan A.A., Ghanem M., Makhlouf H., Alkarn A. Acute pulmonary thromboembolism in emergency room: Doppler ultrasound evaluation. Eur Respir J. 2016;48 doi: 10.1183/13993003.congress-2016.PA2453. [DOI] [PubMed] [Google Scholar]
- 59.Görg C., Kring R., Bert T. Transcutaneous contrast-enhanced sonography of peripheral lung lesions. Am J Roentgenol. 2006;187:W420–W429. doi: 10.2214/AJR.05.0890. [DOI] [PubMed] [Google Scholar]
- 60.Bartelt S., Trenker C., Görg C., Neesse A. Contrast-enhanced ultrasound of embolic consolidations in patients with pulmonary embolism: a pilot study: CEUS of embolic consolidations. J Clin Ultrasound. 2016;44:129–135. doi: 10.1002/jcu.22313. [DOI] [PubMed] [Google Scholar]
- 61.Haaksma M.E., Smit J.M., Heldeweg M.L.A., Nooitgedacht J.S., de Grooth H.J., Jonkman A.H., et al. Extended lung ultrasound to differentiate between pneumonia and atelectasis in critically ill patients: a diagnostic accuracy study. Crit Care Med. 2021 doi: 10.1097/CCM.0000000000005303. [DOI] [PubMed] [Google Scholar]
- 62.van den Berg M.J.W., Waanders D., Kooistra E.J., Kox M., Pickkers P. The value of D-dimer to predict pulmonary embolism in critically ill COVID-19 patients. J Crit Care. 2021;64:18–21. doi: 10.1016/j.jcrc.2021.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Nazerian P., Volpicelli G., Gigli C., Becattini C., Sferrazza Papa G.F., Grifoni S., et al. Diagnostic performance of wells score combined with point-of-care lung and venous ultrasound in suspected pulmonary embolism. Acad Emerg Med. 2017;24:270–280. doi: 10.1111/acem.13130. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary material