Abstract
Background
Although lung ultrasound has been reported to be a portable, cost-effective, and accurate method to detect pneumonia, it has not been widely used because of the difficulty in its interpretation. Here, we aimed to investigate the effectiveness of a novel artificial intelligence-based automated pneumonia detection method using point-of-care lung ultrasound (AI-POCUS) for the coronavirus disease 2019 (COVID-19).
Methods
We enrolled consecutive patients admitted with COVID-19 who underwent computed tomography (CT) in August and September 2021. A 12-zone AI-POCUS was performed by a novice observer using a pocket-size device within 24 h of the CT scan. Fifteen control subjects were also scanned. Additionally, the accuracy of the simplified 8-zone scan excluding the dorsal chest, was assessed. More than three B-lines detected in one lung zone were considered zone-level positive, and the presence of positive AI-POCUS in any lung zone was considered patient-level positive. The sample size calculation was not performed given the retrospective all-comer nature of the study.
Results
A total of 577 lung zones from 56 subjects (59.4 ± 14.8 years, 23% female) were evaluated using AI-POCUS. The mean number of days from disease onset was 9, and 14% of patients were under mechanical ventilation. The CT-validated pneumonia was seen in 71.4% of patients at total 577 lung zones (53.3%). The 12-zone AI-POCUS for detecting CT-validated pneumonia in the patient-level showed the accuracy of 94.5% (85.1%– 98.1%), sensitivity of 92.3% (79.7%– 97.3%), specificity of 100% (80.6%– 100%), positive predictive value of 95.0% (89.6% - 97.7%), and Kappa of 0.33 (0.27–0.40). When simplified with 8-zone scan, the accuracy, sensitivity, and sensitivity were 83.9% (72.2%– 91.3%), 77.5% (62.5%– 87.7%), and 100% (80.6%– 100%), respectively. The zone-level accuracy, sensitivity, and specificity of AI-POCUS were 65.3% (61.4%– 69.1%), 37.2% (32.0%– 42.7%), and 97.8% (95.2%– 99.0%), respectively.
Interpretation
AI-POCUS using the novel pocket-size ultrasound system showed excellent agreement with CT-validated COVID-19 pneumonia, even when used by a novice observer.
Introduction
The novel coronavirus disease 2019 (COVID-19) emerged in China in 2019, and has since spread rapidly worldwide [1]. This highly infectious disease, which often causes fatal pneumonia, has caused 6 million deaths as of March 2022 [2–4]. This rapid increase in the number of patients with COVID-19 has strained medical resources in many countries [5, 6]. In such situations, detailed screening in medical facilities for all patients is not realistic; as such, there is an increasing demand for portable, easy, and cost-effective screening tools.
Point-of-care lung ultrasound (POCUS) is a simple diagnostic test with high diagnostic accuracy for lung disease [7, 8]. A previous systematic review reported that the presence of multiple B-lines could identify the presence of pneumonia with a sensitivity of 90.4%, and a specificity of 88.4% [9]. Although this examination has the potential to serve as a useful screening tool in COVID-19 patients, lung POCUS is not sufficiently widespread, mainly because of technical difficulties in image interpretation.
Artificial intelligence, especially machine learning including deep learning, is an emerging computer technology that automatically recognizes images with excellent accuracy [10–12]. This technology may have the potential to help interpret medical images and enable such examinations, even by novice observers. In this study, we investigated the effectiveness of a novel artificial intelligence-based automated B-line counting system for detecting COVID-19 pneumonia using a pocket-size ultrasound device [13].
Methods
Patient enrollment
We enrolled consecutive patients with COVID-19 who were admitted to our hospital and underwent computed tomography (CT) between August and September 2021. Lung POCUS using a pocket-size ultrasound device (Lumify; Philips Ultrasound, Inc., Bothell, WA) with a novel machine learning-based application (guided B-line quantification; Philips Ultrasound, Inc., Bothell, WA) was performed in all patients within 24 h of the CT scan (artificial intelligence-based point-of-care lung ultrasound; AI-POCUS). This machine learning program was developed by the company and is now commercially available. Detailed development process (training, validation, model architecture, etc.) is not publicly open. Patients aged <20 years and pregnant women were excluded. We further performed AI-POCUS in 15 consecutive subjects without lung diseases who underwent chest CT for the purpose of medical checkup, as controls.
AI-POCUS and CT protocols
AI-POCUS was performed in a 12-zone manner: eight zones for the ventral and four zones for the dorsal chest wall using a 1.0–4.0 Hz sector probe (S4-1; Philips Ultrasound, Inc., Bothell, WA), with a depth setting of 6–12 cm [8, 14, 15]. As shown in Fig 1, the ventral chest wall was segmented by the midline and anterior axillary lines, and further divided into the upper and lower zones. The dorsal chest wall was divided into the left-right and upper-lower zones. For those who were too sick to change their positions (such as intubated patients), only ventral eight zones were scanned [15]. The AI-POCUS movies were automatically analyzed using a real-time automated B-line counting application. All AI-POCUS scans were performed by a blinded novice observer who had experienced fewer than 10 cases of lung POCUS before the start of the study. This machine-learning-based application automatically counts the number of B-lines in the movie, and records the maximum number of B-lines. Three or more B-lines in each zone were considered abnormal (Fig 2).
Fig 1. Lung segmentations in 12-zone lung POCUS.
The chest was divided into eight zones for the ventral wall and four zones for the dorsal chest wall, divided by the posterior axillary line (PAL). The parasternal line (PSL) and paravertebral line (PVL) to two hemithorax. The ventral chest wall was segmented by the midline and anterior axillary lines (AAL) and further divided into upper and lower zones. The dorsal chest wall was divided into left-right and upper-lower zones at the height of the tip of the scapula.
Fig 2. Ultrasound B-lines.
B-lines were automatically analyzed using the real-time deep learning-based application, which counted the number of B-lines (indicated by the white lines in the upper panel) in the movie, and recorded the maximum number of B-lines. Three or more B-lines in each zone were considered abnormal (red zone in lower panel). Patients with one or more abnormal zones were considered positive for AI-POCUS.
Non-contrast chest CT scans were acquired using commercially available equipment (Aquilion ONE, Canon, Japan) at inspiration, and reconstructed as axial images with a 5-mm slice thickness, 5-mm interval, and 120 peak kilovoltage. A board-certified pulmonologist, who was blinded to patient characteristics and AI-POCUS findings, read the CT and diagnosed the presence of pneumonia in each of the 12 zones of the lung that corresponded with the lung POCUS segmentation.
Zone-level concordance was assessed between AI-POCUS and CT results for each zone. In contrast, patient-level AI-POCUS results were defined as negative when the patient had no sign of pneumonia in any of the acquired zones, and patient-level concordance was assessed for each patient.
Statistical analyses
Data are presented as the mean ± standard deviation or median [1st and 3rd quartile] for continuous variables, and as the frequency (%) for categorical variables. Differences in patient characteristics between groups were tested using the Welch t-test and Mann-Whitney test for normally and non-normally distributed variables, respectively, and Fisher’s exact test for categorical variables. The primary endpoint of the study was patient-level accuracy, sensitivity, and specificity of AI-POCUS in comparison with CT-validated pneumonia. Wilson’s method was used to calculate the 95% confidence interval for each metric [16]. All statistical analyses were performed with R version 4.0.2 (The R Foundation for Statistical Computing, Vienna, Austria). A two-tailed p value of <0.05 indicated statistical significance.
Ethics
The study protocol complied with the Declaration of Helsinki and was approved by the institutional review board (IRB) at Juntendo University Hospital (#E21-0197). Written informed consent was waived due to its purely observational nature by the IRB based on the "Ethical Guidelines for Medical and Health Research Involving Human Subjects" issued by Japanese Ministry of Health, Labor and Welfare.
Results
Study population
During the study period, 45 patients with COVID-19 were admitted to the hospital. After excluding 4 patients who did not undergo lung POCUS due to admission on weekends, 41 patients were enrolled in the study. Including 15 control subjects, a total of 56 subjects (41 patients and 15 controls) were included in the study (Fig 3). The baseline patient characteristics are summarized in Table 1. Overall, the mean age was 59.4 ± 14.8 years, 23% were female, and the mean body mass index was 25.3 ± 4.7 kg/m2. The patients were significantly younger than the controls. The median time from the onset to the CT scan was 9 [5 – 12] days, and 20% of the patients were maintained on mechanical ventilation at the time of CT and AI-POCUS.
Fig 3. Patient inclusion chart.
Only 4 patients (8.9%) of cases were excluded and 56 patients (41 cases and 15 controls) were included in the final analysis.
Table 1. Patient characteristic.
| Overall (N = 56) | Case (N = 41) | Control (N = 15) | p value | |
|---|---|---|---|---|
| Age, years old | 59.4 ± 14.8 | 56.3 ± 14.6 | 67.6 ± 12.5 | 0.008 |
| Female | 13 (23%) | 8 (20%) | 5 (33%) | 0.30 |
| Body mass index, kg/m2 | 25.3 ± 4.7 | 25.6 ± 5.1 | 24.7 ± 3.5 | 0.50 |
| Days from onset | 9.0 [5.0–12.0] | |||
| Mechanical Ventilation | 8 (20%) |
AI-POCUS and CT availability
CT evaluation was successful for all 12 zones for all subjects (492 zones for cases and 180 for controls), whereas lung POCUS was not available for the dorsal chest zones in 15 patients who were unable to change their positions. One patient was placed in the prone position, and only the dorsal zones were scanned. Clear images were not available in the left lower inside zone in the ventral chest of 26 patients because the heart was close to the chest wall, and the lung was therefore not visible from this position. Finally, lung POCUS movies of 397 zones for patients and 180 zones for controls were analyzed.
Concordance of pneumonia identification
A CT scan identified pneumonia in 395 zones (58.7%) of 40 subjects, whereas no pneumonia was detected in the other 16 subjects. Pneumonia was present almost evenly in each zone, with a slightly higher prevalence in the dorsal zones (Fig 4).
Fig 4. Distribution of pneumonia.
The X-axis represents the zone of the lung (R1, right anterior superior zone; R2, right anterior inferior zone; R3, right lateral superior zone; R4, right lateral inferior zone; R5, right superior dorsal zone; R6, right inferior dorsal zone; Ls are left zones corresponding to Rs), and the Y-axis represents the number of patients with pneumonia in each zone.
Panel A in Fig 5 demonstrates the patient-level concordance between the AI-POCUS and CT-validated pneumonia. One subject was excluded from this analysis because AI-POCUS was not possible for the dorsal zones, although he only had pneumonia there. AI-POCUS was able to diagnose the presence or absence of pneumonia in 52 of 55 subjects, while pneumonia was overlooked in three patients. The overall accuracy, sensitivity, specificity, PPV, NPT, and Kappa of the AI-POCUS were 94.5% (85.1%– 98.1%), 92.3% (79.7%– 97.3%), 100% (80.6%– 100%), 100% (90.4%– 100%), 84.2% (62.4–94.5%), and 0.89 (0.70–0.98), respectively.
Fig 5. Accuracy of AI-POCUS versus CT.
Each panel is a confusion matrix showing the concordance between the AI-POCUS results and CT-validated pneumonia diagnosis.
The screening performance of AI-POCUS using only the ventral eight zones was evaluated as a simpler approach that can be used even for those who cannot change their position. With only 8-zone AI-POCUS, pneumonia was overlooked in 9 patients, whereas 31 of 40 patients with pneumonia were still classified as positive (Fig 5, Panel B). The accuracy, sensitivity, NPV, and Kappa decreased to 83.9% (72.2%– 91.3%), 77.5% (62.5%– 87.7%), 64.0% (44.5–79.8%), and 0.68 (0.43–0.85), respectively; however, the specificity and PPV remained at 100% for both (80.6%– 100% and 89.0%– 100%).
In the zone-level analysis, AI-POCUS was correct in 377 of 577 zones (Fig 5, Panel C). The zone-level accuracy, sensitivity, specificity, PPV, NPV, and Kappa of the AI-POCUS were 65.3% (61.4%– 69.1%), 37.2% (32.0%– 42.7%), 97.8% (95.2%– 99.0%), 95.5% (89.6%– 97.7%), 57.5% (52.9%– 61.9%), and 0.31 (0.23–0.38), respectively.
Finally, we assumed that the prevalence of pneumonia in COVID-19 positive patients is 5% based on the public data from the Japanese Ministry of Health, Labour and Welfare and calculated the estimated PPV and NPV in the real-world. Given the sensitivity of 92.3% and specificity of 100%, and 5% of prior probability, PPV was 100% (99.9%– 100%) and NPV was 99.6% (99.6%– 99.6%).
Discussion
The main findings of our study were as follows: (1) AI-POCUS had excellent performance at detecting patients with COVID-19 pneumonia (sensitivity 92.3% and specificity 100%), and (2) the zone-level sensitivity was moderate (37.3%), although specificity was very high (97.8%). Even with moderate sensitivity in each zone, pneumonia in COVID-19 usually spreads to multiple lung zones, leading to a high sensitivity for each patient. Notably, these excellent results were achieved with AI-POCUS by a novice observer who had minimal experience in lung-POCUS.
With the recent miniaturization of ultrasound devices and advancements in image quality, lung-POCUS is becoming a popular examination, especially in intensive care [8, 17]. Although the lung itself, which is filled with air, cannot usually be observed by ultrasound beams, the noise and artifacts generated by ultrasound beams provide useful clinical information in lung-POCUS. Ultrasound B-lines are one of the most useful artifacts that present with pulmonary congestion, either by pneumonia or cardiogenic pulmonary edema. These reverberation artifacts originate at the pleura, reflecting an air-fluid mixture, which occurs when the subpleural interlobular septa surrounded by subpleural air-filled alveoli become edematous. Previous studies have reported that three or more B-lines visible in a single ultrasound plane are fully sensitive and specific to demonstrate subpleural thickened interlobular septa and/or ground-glass areas with a CT scan as a reference [18].
Diagnosis of pneumonia using lung-POCUS techniques, including the B-line, is expected to be an effective tool in the ongoing COVID-19 pandemic. Reports have already shown that B-lines are not only sensitive, but also associated with disease severity, future deterioration, and treatment effects in COVID-19 [15, 19–23]. However, chest radiography and CT remain the leading examinations used in the management of COVID-19 worldwide, and lung-POCUS has not been sufficiently used. Technical difficulties and problems in interpreting images are major concerns when using lung-POCUS; the number of experts is not large enough to teach and supervise the use of lung-POCUS by clinicians, including general practitioners, although this technique is relatively new and may be difficult for novice observers.
In the present study, we demonstrated that with AI technology, lung-POCUS can be effective to diagnose pneumonia, with excellent accuracy, even by a novice observer. Recent advancements in AI technology, more specifically machine learning, have enabled automated image recognition with similar or even higher accuracy compared to expert clinicians, and the application of such technologies to POCUS has been enthusiastically studied [10, 11, 24, 25]. The automated B-line counting application that we used in this study was also developed using machine learning technology, and showed excellent agreement with expert readings [26, 27].
The moderate zone-level accuracy observed herein may be due to the limited ability to detect slight pneumonia in the central area of the lung far from the body surface. Such slight pneumonia in a deep area of the lung may not affect the pleura, and would therefore not generate a B line. However, pneumonia caused by COVID-19 usually spreads to multiple regions of the lungs, appearing in the form of acute respiratory distress syndrome [23]. Therefore, diffuse pneumonia caused by COVID-19 is a suitable target for AI-POCUS.
COVID-19 is expected to persist for some time. AI-POCUS allows non-specialist general practitioners to diagnose pneumonia, a problem in viral infections, in an access-free manner, which may be useful in the treatment of pneumonia during a pandemic, and is expected to become more widespread.
Limitations
This study has some limitations. First, this was a single-center study conducted in a university hospital, and included a small number of cases. Larger studies involving various settings are necessary before the current results can be widely implemented. Second, we did not check the accuracy of AI-POCUS compared with traditional lung POCUS by an expert. However, a previous study showed that AI-POCUS has over 90% sensitivity and specificity for detecting expert-annotated B-lines [26]. Next, we studied only hospitalized patients who were considered sicker than those who were cared for at home or at other facilities. The performance of the present application should be tested in less intensively sick patients to uncover it’s applicability for screening purposes.
Conclusions
AI-POCUS showed excellent performance in diagnosing patients with CT-validated COVID-19 pneumonia, even by a novice observer.
Abbreviations
- COVID-19
coronavirus disease 2019
- POCUS
point-of-care ultrasound
- AI-POCUS
artificial intelligence-based point-of-care lung ultrasound
- CT
computed tomography
Data Availability
The data are not publicly open due to privacy concerns. Sharing the present data with researchers/companies outside the author’s research group is not approved by the IRB at Juntendo University, and IRB application and approval upon an appropriate request are necessary before sharing the data (contact information: Nobuyuki Kagiyama; kgnb_27_hot@yahoo.co.jp and Data Coordinator of Department of Cardiovascular Biology and Medicine; junkanki@juntendo.ac.jp).
Funding Statement
Kagiyama and Daida are affiliated with a department funded by Philips Japan, Asahi KASEI Corporation, Inter Reha Co., Ltd, and Toho Holdings Co., Ltd., based on collaborative research agreements. Other authors have no conflict of interest to declare. This does not alter our adherence to PLOS ONE policies on sharing data and materials. This work was partially supported by the Japan Society for Promotion of Science KAKENHI (by the Japanese government), with a Grant Number 21K18086. There was no additional external funding received for this study.
References
- 1.Organization WH. WHO Coronavirus Disease (COVID-19) Dashboard. 2022. [cited 2022 April 26]. Available from: https://covid19.who.int/. [Google Scholar]
- 2.Carvalho T, Krammer F, Iwasaki A. The first 12 months of COVID-19: a timeline of immunological insights. Nat Rev Immunol. 2021;21(4):245–56. Epub 2021/03/17. doi: 10.1038/s41577-021-00522-1 ; PubMed Central PMCID: PMC7958099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hu B, Guo H, Zhou P, Shi ZL. Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol. 2021;19(3):141–54. Epub 2020/10/08. doi: 10.1038/s41579-020-00459-7 ; PubMed Central PMCID: PMC7537588. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC. Pathophysiology, Transmission, Diagnosis, and Treatment of Coronavirus Disease 2019 (COVID-19): A Review. Jama. 2020;324(8):782–93. Epub 2020/07/11. doi: 10.1001/jama.2020.12839 . [DOI] [PubMed] [Google Scholar]
- 5.Ji Y, Ma Z, Peppelenbosch MP, Pan Q. Potential association between COVID-19 mortality and health-care resource availability. Lancet Glob Health. 2020;8(4):e480. Epub 2020/02/29. doi: 10.1016/S2214-109X(20)30068-1 ; PubMed Central PMCID: PMC7128131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kavanagh MM, Erondu NA, Tomori O, Dzau VJ, Okiro EA, Maleche A, et al. Access to lifesaving medical resources for African countries: COVID-19 testing and response, ethics, and politics. Lancet. 2020;395(10238):1735–8. Epub 2020/05/11. doi: 10.1016/S0140-6736(20)31093-X ; PubMed Central PMCID: PMC7252104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Mojoli F, Bouhemad B, Mongodi S, Lichtenstein D. Lung Ultrasound for Critically Ill Patients. Am J Respir Crit Care Med. 2019;199(6):701–14. Epub 2018/10/30. doi: 10.1164/rccm.201802-0236CI . [DOI] [PubMed] [Google Scholar]
- 8.Volpicelli G, Elbarbary M, Blaivas M, Lichtenstein DA, Mathis G, Kirkpatrick AW, et al. International evidence-based recommendations for point-of-care lung ultrasound. Intensive Care Med. 2012;38(4):577–91. Epub 2012/03/07. doi: 10.1007/s00134-012-2513-4 . [DOI] [PubMed] [Google Scholar]
- 9.Xia Y, Ying Y, Wang S, Li W, Shen H. Effectiveness of lung ultrasonography for diagnosis of pneumonia in adults: a systematic review and meta-analysis. Journal of Thoracic Disease. 2016;8(10):2822–31. doi: 10.21037/jtd.2016.09.38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, et al. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020;26(6):900–8. Epub 2020/05/20. doi: 10.1038/s41591-020-0842-3 . [DOI] [PubMed] [Google Scholar]
- 11.Kagiyama N, Shrestha S, Farjo PD, Sengupta PP. Artificial Intelligence: Practical Primer for Clinical Research in Cardiovascular Disease. J Am Heart Assoc. 2019;8(17):e012788. Epub 2019/08/28. doi: 10.1161/JAHA.119.012788 ; PubMed Central PMCID: PMC6755846. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Roy S, Menapace W, Oei S, Luijten B, Fini E, Saltori C, et al. Deep Learning for Classification and Localization of COVID-19 Markers in Point-of-Care Lung Ultrasound. IEEE Trans Med Imaging. 2020;39(8):2676–87. Epub 2020/05/15. doi: 10.1109/TMI.2020.2994459 . [DOI] [PubMed] [Google Scholar]
- 13.Baloescu C, Chen A, Raju B, Evans N, Moore CL. 19 Automated Quantification Of B-Lines in Lung Ultrasound On COVID-19 Patients. Ann Emerg Med. 2021;78(2):S9–S10. Epub 2021/08/04. doi: 10.1016/j.annemergmed.2021.07.020 PMC8335439. [DOI] [Google Scholar]
- 14.Levy Adatto N, Preisler Y, Shetrit A, Shepshelovich D, Hershkoviz R, Isakov O. Rapid 8-Zone Lung Ultrasound Protocol is Comparable to a Full 12-Zone Protocol for Outcome Prediction in Hospitalized COVID-19 Patients. J Ultrasound Med. 2021. Epub 2021/10/27. doi: 10.1002/jum.15849 ; PubMed Central PMCID: PMC8661589. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Haurylenka D, Damantsevich V, Filustsin A, Damantsevich A. Diagnostic value of 12-zone lung ultrasound protocol for diagnosing COVID-19-associated pneumonia in outpatients. J Ultrason. 2021;21(87):e271–e6. Epub 2022/01/01. doi: 10.15557/JoU.2021.0046 ; PubMed Central PMCID: PMC8678640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wilson EB. Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association. 1927;22(158):209–12. doi: 10.1080/01621459.1927.10502953 [DOI] [Google Scholar]
- 17.Bhagra A, Tierney DM, Sekiguchi H, Soni NJ. Point-of-Care Ultrasonography for Primary Care Physicians and General Internists. Mayo Clin Proc. 2016;91(12):1811–27. Epub 2016/11/09. doi: 10.1016/j.mayocp.2016.08.023 . [DOI] [PubMed] [Google Scholar]
- 18.Lichtenstein D, Mézière G, Biderman P, Gepner A, Barré O. The comet-tail artifact. An ultrasound sign of alveolar-interstitial syndrome. Am J Respir Crit Care Med. 1997;156(5):1640–6. Epub 1997/12/31. doi: 10.1164/ajrccm.156.5.96-07096 . [DOI] [PubMed] [Google Scholar]
- 19.Zieleskiewicz L, Markarian T, Lopez A, Taguet C, Mohammedi N, Boucekine M, et al. Comparative study of lung ultrasound and chest computed tomography scan in the assessment of severity of confirmed COVID-19 pneumonia. Intensive Care Med. 2020;46(9):1707–13. Epub 2020/07/31. doi: 10.1007/s00134-020-06186-0 ; PubMed Central PMCID: PMC7388119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rubio-Gracia J, Giménez-López I, Garcés-Horna V, López-Delgado D, Sierra-Monzón JL, Martínez-Lostao L, et al. Point-of-care lung ultrasound assessment for risk stratification and therapy guiding in COVID-19 patients: a prospective noninterventional study. Eur Respir J. 2021;58(3). Epub 2021/02/13. doi: 10.1183/13993003.04283-2020 ; PubMed Central PMCID: PMC7877323 interest: I. Giménez-López has nothing to disclose. Conflict of interest: V. Garcés-Horna has nothing to disclose. Conflict of interest: D. López-Delgado has nothing to disclose. Conflict of interest: J.L. Sierra-Monzón has nothing to disclose. Conflict of interest: L. Martínez-Lostao has nothing to disclose. Conflict of interest: C. Josa-Laorden has nothing to disclose. Conflict of interest: F. Ruiz-Laiglesia has nothing to disclose. Conflict of interest: J.I. Pérez-Calvo has nothing to disclose. Conflict of interest: S. Crespo-Aznarez has nothing to disclose. Conflict of interest: J. García-Lafuente has nothing to disclose. Conflict of interest: N. Peña-Fresneda has nothing to disclose. Conflict of interest: B. Amores Arriaga has nothing to disclose. Conflict of interest: B. Gracia-Tello has nothing to disclose. Conflict of interest: M. Sánchez-Marteles has nothing to disclose. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Volpicelli G, Gargani L, Perlini S, Spinelli S, Barbieri G, Lanotte A, et al. Lung ultrasound for the early diagnosis of COVID-19 pneumonia: an international multicenter study. Intensive Care Med. 2021;47(4):444–54. Epub 2021/03/21. doi: 10.1007/s00134-021-06373-7 ; PubMed Central PMCID: PMC7980130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Wang M, Luo X, Wang L, Estill J, Lv M, Zhu Y, et al. A Comparison of Lung Ultrasound and Computed Tomography in the Diagnosis of Patients with COVID-19: A Systematic Review and Meta-Analysis. Diagnostics (Basel). 2021;11(8). Epub 2021/08/28. doi: 10.3390/diagnostics11081351 ; PubMed Central PMCID: PMC8394642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Zheng Y, Wang L, Ben S. Meta-analysis of chest CT features of patients with COVID-19 pneumonia. J Med Virol. 2021;93(1):241–9. Epub 2020/06/25. doi: 10.1002/jmv.26218 ; PubMed Central PMCID: PMC7361361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Sengupta PP, Shrestha S, Berthon B, Messas E, Donal E, Tison GH, et al. Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council. JACC Cardiovasc Imaging. 2020;13(9):2017–35. Epub 2020/09/12. doi: 10.1016/j.jcmg.2020.07.015 ; PubMed Central PMCID: PMC7953597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Seetharam K, Kagiyama N, Sengupta PP. Application of mobile health, telemedicine and artificial intelligence to echocardiography. Echo Res Pract. 2019;6(2):R41–r52. Epub 2019/03/08. doi: 10.1530/ERP-18-0081 ; PubMed Central PMCID: PMC6432977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Baloescu C, Toporek G, Kim S, McNamara K, Liu R, Shaw MM, et al. Automated Lung Ultrasound B-Line Assessment Using a Deep Learning Algorithm. IEEE Trans Ultrason Ferroelectr Freq Control. 2020;67(11):2312–20. Epub 2020/08/04. doi: 10.1109/TUFFC.2020.3002249 . [DOI] [PubMed] [Google Scholar]
- 27.Moore CL, Wang J, Battisti AJ, Chen A, Fincke J, Wang A, et al. Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device. J Ultrasound Med. 2021. Epub 2021/12/30. doi: 10.1002/jum.15935 . [DOI] [PubMed] [Google Scholar]