Abstract
Background and aim:
During the first wave of the Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2) pandemic, we faced a massive clinical and organizational challenge having to manage critically ill patients outside the Intensive Care Unit (ICU). This was due to the significant imbalance between ICU bed availability and the number of patients presenting Acute Hypoxemic Respiratory Failure caused by SARS-CoV-2-related interstitial pneumonia. We therefore needed to perform Non-Invasive Ventilation (NIV) in non-intensive wards to assist these patients and relieve pressure on the ICUs and subsequently implemented a new organizational and clinical model. This study was aimed at evaluating its effectiveness and feasibility.
Methods:
We recorded the anamnestic, clinical and biochemical data of patients undergoing non-invasive mechanical ventilation while hospitalized in non-intensive CoronaVirus Disease 19 (COVID-19) wards. Data were registered on admission, during anesthesiologist counseling, and when NIV was started and suspended. We retrospectively registered the available results from routine arterial blood gas and laboratory analyses for each time point.
Results:
We retrospectively enrolled 231 patients. Based on our criteria, we identified 46 patients as NIV responders, representing 19.9% of the general study population and 29.3% of the patients that spent their entire hospital stay in non-ICU wards. Overall mortality was 56.2%, with no significant differences between patients in non-intensive wards (57.3%) and those later admitted to the ICU (54%).
Conclusions:
NIV is safe and manageable in an emergency situation and could become part of an integrated clinical and organizational model. (www.actabiomedica.it)
Keywords: COVID-19, Hypoxia, HFNC, NIV, ICU
Introduction
During the early months of 2020, the COVID-19 pandemic swept over healthcare facilities around the world, with a wave of patients being hospitalized for pneumonia caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). As of May 8th, 2020, the World Health Organization (WHO) had already reported 1,654,345 confirmed cases of CoronaVirus Disease 19 (COVID-19) in Europe causing 152,179 deaths, of which nearly 20% had occurred in Italy (29,958)(1). Between March 8th and May 8th, approximately 2300 patients were admitted to the Ospedale Maggiore in Parma, Italy for SARS-CoV-2-related interstitial pneumonia. From a clinical and organizational point of view, the management of patients showing the advanced stages of the disease was undoubtedly one of the most relevant criticalities we had to deal with during the first wave of the COVID-19 pandemic. Furthermore, the high number of cases in relation to the limited number of beds available in Intensive Care Units (ICUs) led to the need to evaluate and treat patients with acute hypoxemic respiratory failure (AHRF) requiring high-intensity care in non-intensive wards.
In this emergency context, anesthesiologists, together with specialists in internal medicine and pneumology, responded promptly and competently to the assistance needs determined by the COVID-19 pandemic, adapting to a new, non-conventional organizational and clinical model determined by the extreme pressure on the national health service. Adequate respiratory support for critically ill patients outside the ICU became an emergent priority, and on the basis of anamnestic, clinical, instrumental, and laboratory parameters Non-Invasive Ventilation (NIV) was introduced(2–9). The use of NIV by highly specialized personnel in a non-intensive, and therefore “unconventional”, context was each time carried out with focus on the safety of both the healthcare personnel and the patients, guaranteeing the latter the clinical benefits provided by this method(2).
Nevertheless, we cannot ignore the fact that NIV is not always effective in the presence of AHRF caused by infective pneumonia, with a reported success rate as low as 7.6 % in patients presenting with Middle East Respiratory Syndrome (MERS) (10–12).
Objectives
The aim of this observational study was to record the clinical course and main hospital outcome of critically ill patients undergoing NIV while being treated in non-intensive wards during the study period.
Our primary objective was the observation of any changes in respiratory index (RI) values (ratio between arterial blood partial oxygen pressure (PaO2) and oxygen inspiratory fraction (FiO2)).
As a secondary objective, we aimed to determine whether it was possible to identify early predictive elements that could help detect patients with an increased risk of negative clinical outcome (ICU admission and hospital mortality).
Materials and Methods
This study was conceived as a monocentric, observational, retrospective study, and obtained approval of the Local Ethics Committee. “Strengthening the Reporting of Observational studies in Epidemiology” (STROBE) guidelines were followed for the drafting of this work.
Data collection was carried out maintaining the anonymity of each patient identified by an alpha-numeric code. The inclusion criteria considered were as follows:
- Hospitalized patients diagnosed with SARS-CoV-2 interstitial pneumonia
- NIV treatment
- Age over 18 years old
In our study, we retrospectively collected clinical records, laboratory test results and radiological examinations from patients with a confirmed diagnosis of SARS-CoV-2-related interstitial pneumonia undergoing non-invasive mechanical ventilation while hospitalized in non-intensive COVID-19 wards at the Ospedale Maggiore in Parma, Italy from March 8th to May 8th, 2020. Data were collected using a case record form (CRF). We registered data regarding age, sex, body mass index (BMI), past medical history (chronic cardiac disease, chronic pulmonary disease, cerebrovascular disease, chronic neurological disorder, diabetes, malignancy), vital signs at admission (temperature, respiratory rate), and laboratory values (inflammatory markers, ABG analysis, blood glucose). The flowchart (figure 1) illustrates the step-by-step treatment we applied to patients with respiratory insufficiency: initially, incremental oxygen support was provided using Ventimask, with a flow up to 15 l / min. In the presence of non-response, support with a reservoir mask and nasal cannula (15 l / min + 15 l / min) was used, delivering oxygen with an estimated FiO2 of 70%. In the presence of further failure, after evaluation by the anesthesiologists’ team, the patients underwent NIV trial. (Figures 1 and 2).
Figure 1.
The study flow chart in line with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement (http://www.strobestatement.org). Step-by-step treatment of patients with respiratory insufficiency. Initially an incremental oxygen support was set up with a ventimask with flow up to 15 l / min. In case of non-response, a support with a reservoir mask and nasal cannulae (15 l / min + 15 l / min) was used, delivering oxygen with an estimated FiO2 of 70%. In case of further failure, after evaluation by the ICU team, the patients underwent the NIV trial.
Figure 2.
Management of COVID 19 patients in hospital wards and non-invasive ventilation failure criteria
Data collection was performed on hospital admission (time point 0), during anesthesiologist counseling (1), early after NIV trial start (2), and at the moment of NIV interruption (3).
Clinical outcomes were expressed as follows: length of hospital and ICU stay (days), transfer to ICU (yes or no), and in-hospital death (yes or no). In the event of a transfer, the day of admission to the ICU, with respect to hospitalization and the length of stay in the ward, were recorded. Hospitalization outcome was classified as discharge or in-hospital death.
Possible complications of NIV treatment: axillary pressure sores, upper-limb edema, sense of gastric distension, agitation or poor compliance with the device, others (any further complications not listed above).
All parameters were subsequently expressed in absolute value and in terms of mean and standard deviation or median and 25th and 75th percentile (interquartile range, IQR). Comorbidities were recorded according to commonly used clinical nomenclature. The number of patients with SARS-CoV-2 interstitial pneumonia undergoing NIV is expressed in terms of absolute value and percentage in different sub-populations.
Three main types of NIV interface were used: high-flow nasal cannula (HFNC), oral-nasal and full-face Venturi masks, and NIV helmets. Patients were closely monitored, with regular checks being carried out by healthcare staff and the anesthesiologists (10–12).
NIV settings and parameters were recorded as follows: type of interface: helmet, mask or HFNC. Positive End Expiratory Pressure (PEEP) was expressed as cmH2O and FiO2 as percentages. NIV treatment modality was encoded as follows: Continuous Positive Airway Pressure (CPAP), Bi-level Positive Airway Pressure (BPAP), and HFNC. NIV duration and other time intervals were expressed in days.
We included the maximum number of patients who met the inclusion criteria. We expressed descriptive data as mean, median and standard deviation (SD) for continuous variables, while categorical variables were reported as number and percentage (%). IBM SPSS software was applied for all analyses. We assessed the differences between groups using Mann-Whitney, Spearman and Pearson tests. We performed a logistic regression analysis to assess the main risk factors for eventual in-hospital death and/or admission to the ICU. The following factors were initially entered into the model: age; BMI; C reactive protein (CRP), procalcitonin (PCT) and D-dimer levels before NIV trial; RI ratio before and after NIV trial as an improvement marker; and the ratio of admission RI to the last available value before outcome occurred or the case was censored. Akaike’s and Bayesian information criteria were used to guide model analysis (factors removed until criteria decreased ≤10 units). The optimized logistic regression model did not include CRP and D-dimer levels and had an area under the receiver operating characteristics curve of 0.75, with age and overall RI variation ratio as statistically significant independent factors.
Results
We retrospectively enrolled 231 patients in our cohort study (Table 1 and figure 1). Mean age was 64±11 years (31-90), with a majority of male patients (n=149, 64.5%). When possible BMI was calculated, with the observed mean value being 30±6.62. We observed a statistically significant difference between deceased and discharged patients (p=0.021) for this parameter. Hypertension was the most common comorbidity (prevalence: 59.3%), followed by diabetes (28.1%), and heart disease (17.3%) (Table 2).
Table 1.
Anthropometric, admission laboratory data and main clinical outcomes in general study population
| n=231 | n | % | Mean | SD | Med | 25 | 75 |
| Age (years) | 64.0 | 10.3 | 65.0 | 58 | 71 | ||
| Weight (kg) | 86.1 | 19.2 | 84.0 | 75 | 95 | ||
| BMI (kg/m2) | 30.0 | 6.62 | 28.7 | 25.8 | 32.9 | ||
| Hospital stay (days) | 25.4 | 23.0 | 19.0 | 10 | 29.5 | ||
| Admission - NIV start (days) | 4.2 | 4.8 | 3.0 | 1 | 6 | ||
| NIV duration (days) | 6.5 | 5.2 | 5.0 | 3 | 9 | ||
| Admission – ICU transfer (days) | 8.7 | 6.8 | 8.0 | 4 | 10 | ||
| ICU stay (days) | 18.3 | 18.6 | 14.0 | 3 | 26.8 | ||
| CRP at admission (mg/l) | 142.3 | 69.0 | 129.2 | 95 | 201.3 | ||
| PCT at admission (ng/ml) | 0.57 | 0.83 | 0.30 | 0.15 | 0.62 | ||
| Glycemia at admission (mg/dl) | 150 | 71 | 126 | 108 | 163 | ||
| D-dimer at admission (µg/ml) | 1829 | 2539 | 823 | 606 | 1236 | ||
| Gender (F) | 82 | 35.5 | |||||
| ICU transfer | 74 | 32.0 | |||||
| Mortality | 130 | 56.3 |
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; BMI, body mass index; NIV, non-invasive ventilation; ICU, intensive care unit; CRP, C reactive protein, PCT, procalcitonin; F, female
Table 2.
Comorbidities in general study population, in survived/deceased and non-ICU/ICU patient subgroups.
| n (%) | |||||||||||
| General population (n=231) | Non-ICU (n=157) | ICU (n=74) | Survived (n=101) | Deceased (n=130) | p* | ||||||
| Hypertension | 137 | 59.3% | 103 | 65.6% | 34 | 45.9% | 47 | 46.5% | 90 | 69.2% | <0.001 |
| Diabetes | 65 | 28.1% | 55 | 35% | 10 | 13.5% | 28 | 27.7% | 37 | 28.5% | 0.901 |
| Cardiac disease | 40 | 17.3% | 34 | 21.6% | 6 | 8.1% | 10 | 9.9% | 30 | 23.1% | 0.001 |
| COPD | 24 | 10.4% | 19 | 12.1% | 5 | 6.8% | 16 | 15.8% | 8 | 6.2% | 0.017 |
| Other respiratory diseases | 31 | 13.4% | 24 | 15.2% | 7 | 9.4% | 19 | 18.8% | 12 | 9.2% | OSAS: 0.865 Asthma: 0.276 |
| CKD | 8 | 3.5% | 7 | 4.5% | 1 | 1.4% | 1 | 1% | 7 | 5.4% | 0.070 |
| Liver disease | 14 | 6.1% | 10 | 6.3% | 4 | 5.6% | 5 | 5.0% | 9 | 6.9% | 0.533 |
| Vasculopathy | 9 | 3.9% | 7 | 4.5% | 2 | 2.8% | 3 | 3.0% | 6 | 4.6% | 0.522 |
| Malignancy | 13 | 5.6% | 8 | 5.1% | 5 | 6.7% | 3 | 3.0% | 10 | 7.7% | 0.122 |
| DVT/PE | 6 | 2.6% | 5 | 3.2% | 1 | 1.4% | 0 | 0.0% | 6 | 4.6% | 0.029 |
| Stroke/ TIA | 6 | 2.6% | 3 | 1.9% | 3 | 4.2% | 2 | 2.0% | 4 | 3.1% | 0.629 |
n, number; ICU, intensive care unit; Cardiac disease: structural, valvular or arrhythmic; COPD, chronic obstructive pulmonary disease; Other respiratory disease, asthma and OSAS (Obstructive Sleep Apnea Syndrome); CKD, chronic kidney disease; DVT, deep vein thrombosis; PE, pulmonary embolism; TIA, transient ischemic attack. *p value has been calculated for survived and deceased patients.
The median time between hospital admission and first clinical evaluation performed by an anesthesiologist was 2 days (interquartile range, IQR: 1-3), while on average 4.2 days passed between hospitalization and NIV initiation (Table 1)
Helmet CPAP was the most common ventilation method used in 78.3% of patients, as this interface is compatible with the oxygen sources available in non-ICU wards and is therefore easier to manage in an emergency setting. On the other hand, 10.8 % of patients were treated with a high-flow nasal cannula, while a similar percentage (10.9%) was supported with BPAP.
NIV was carried out for a mean period of 6.5±5.2 days, with values ranging from failed trials lasting less than one day to a maximum of 30-day treatment based on NIV cycles.
22 patients (9.5%) developed superior limbs edema during NIV treatment, and we observed 8 cases (3.5%) of superior limbs deep vein thrombosis (DVT), 4 of both pneumothorax (PNX) and pleural effusion (1.7%) We also recorded 3 episodes of nausea or vomiting in patients undergoing NIV (1.3%) and 2 cases of hemoptysis (0.9%).
Median hospital stay was 19 days (IQR: 10-29.5); this high variability was due to some cases of early-death leading to a precocious outcome (minimum: 2 days) as well as some extremely long hospital stays (maximum:123 days). Overall mortality was 56.2%, which was comparable between the patients in non-intensive wards (57.3%) and those admitted to the ICU (54%). This cohort included 74 patients (32% of the general population), mainly males (n=53, 71.7%), with a moderately younger mean age (59.4 years) and a lower prevalence of comorbidities (Tables 3-4). On average, patients were transferred on the 9th day of hospital stay (1-39). Mean ICU stay was 18.3 days, with values ranging from death on the day of admission to a maximum of 83 days. D-dimer mean value at admission was noticeably higher in patients who were not transferred to the ICU (p=0.002) (Table 5).
Table 3.
Anthropometric, admission laboratory data and main clinical outcomes in survived and deceased patient subgroups.
| Survived (n=101) | Deceased (n=130) | ||||||||||||||
| n | % | Mean | SD | Med | 25 | 75 | n | % | Mean | SD | Med | 25 | 75 | P | |
| Age (years) | 59.1 | 11.1 | 60.0 | 53.0 | 66.0 | 67.8 | 9.0 | 68.0 | 62.3 | 72.0 | |||||
| Weight (kg) | 90.2 | 19.2 | 90.0 | 77.0 | 106.5 | 82.5 | 18.5 | 80.0 | 70.9 | 90.0 | |||||
| BMI (kg/m2) | 31.3 | 6.3 | 30.2 | 26.6 | 35.5 | 29.1 | 6.8 | 28.0 | 25.0 | 31.1 | 0.021 | ||||
| Hospital stay | 38.7 | 27.2 | 27 | 20.0 | 48.0 | 15.1 | 11.3 | 11.0 | 7.0 | 20.0 | |||||
| Admission -NIV | 3.4 | 3.7 | 3 | 1.0 | 4.8 | 4.7 | 5.5 | 4.0 | 1.0 | 6.0 | |||||
| NIV duration | 6.6 | 4.6 | 6 | 3.3 | 9.0 | 6.3 | 5.6 | 4.0 | 3.0 | 9.0 | |||||
| Admission - ICU transfer | 7.1 | 4.4 | 7 | 3.5 | 9.0 | 10.0 | 8.1 | 8.0 | 4.5 | 11.5 | |||||
| ICU stay | 28.3 | 21.2 | 27 | 12.5 | 37.0 | 10.4 | 11.4 | 4.0 | 1.5 | 20.5 | |||||
| Gender (F) | 36 | 35.6 | 46 | 35.4 | |||||||||||
| ICU transfer | 34 | 33.7 | 40 | 30.8 | |||||||||||
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; ICU, intensive care unit; BMI, body mass index; NIV, non-invasive ventilation; F, female; Admission-NIV, NIV duration, Admission-ICU transfer and ICU stay are expressed in days.
Table 4.
Anthropometric, admission laboratory data and main clinical outcomes in non-ICU and ICU patient’s subgroups
| Non-ICU (n=157) | ICU (n=74) | ||||||||||||||
| n | % | Mean | SD | Med | 25 | 75 | n | % | Mean | SD | Med | 25 | 75 | P | |
| Age (years) | 66.1 | 10.6 | 68.0 | 60.0 | 72.0 | 59.4 | 9.9 | 61.0 | 53.0 | 65.0 | |||||
| Weight (kg) | 86.6 | 20.8 | 84.0 | 75.0 | 97.0 | 85.1 | 15.5 | 84.5 | 74.6 | 91.1 | |||||
| BMI (kg/m2) | 30.1 | 6.6 | 29.1 | 25.8 | 33.1 | 29.9 | 6.7 | 28.0 | 25.7 | 31.6 | 0.673 | ||||
| Hospital stay | 18.9 | 13.2 | 16.0 | 9.0 | 25.0 | 39.3 | 31.8 | 28.5 | 18.3 | 58.8 | |||||
| Admission -NIV | 4.3 | 5.1 | 3.0 | 1.0 | 5.3 | 4.0 | 4.1 | 3.0 | 1.0 | 6.0 | |||||
| NIV duration | 7.1 | 5.2 | 6.0 | 3.0 | 9.3 | 5.1 | 5.1 | 4.0 | 2.0 | 6.0 | |||||
| Gender (F) | 61 | 38.9 | 21 | 28.4 | |||||||||||
| Mortality | 90 | 57.3 | 40 | 54.1 | |||||||||||
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; ICU, intensive care unit; BMI, body mass index; NIV, non-invasive ventilation; F, female; Admission-NIV, NIV duration, are expressed in days.
Table 5.
Main laboratory data in non-ICU and ICU patient subgroups.
| Non-ICU (n=157) | ICU (n=74) | ||||||||||
| Mean | SD | Med | 25 | 75 | Mean | SD | Med | 25 | 75 | P | |
| CRP 0 | 148.6 | 71.3 | 130.5 | 99.7 | 225.21 | 128.7 | 62.7 | 123 | 78.3 | 156.9 | 0.296 |
| CRP 2 | 148.8 | 80.5 | 144.1 | 96.1 | 232.6 | 176.6 | 73.8 | 161.2 | 137 | 250 | |
| CRP 3 | 122.2 | 105.9 | 96.5 | 16 | 248.5 | 150.8 | 105.9 | 193.8 | 35.2 | 249.7 | |
| PCT 0 | 0.66 | 0.97 | 0.33 | 0.16 | 0.65 | 0.36 | 0.31 | 0.27 | 0.15 | 0.54 | 0.382 |
| PCT 2 | 1.37 | 2.57 | 0.63 | 0.22 | 1.24 | 1.12 | 1.12 | 0.68 | 0.38 | 1.7 | |
| PCT 3 | 0.92 | 1.11 | 0.57 | 0.07 | 1.16 | 1.03 | 1.53 | 0.4 | 0.22 | 0.72 | |
| Glyc 0 | 156 | 77 | 127 | 109 | 173 | 137 | 57 | 124 | 104 | 139 | 0.076 |
| Glyc 2 | 160 | 64 | 142 | 116 | 185 | 139 | 40 | 128 | 116 | 151 | |
| Glyc 3 | 170 | 74 | 160 | 115 | 208 | 140 | 40 | 128 | 113 | 168 | |
| D-d 0 | 2270 | 2842 | 937 | 685 | 2044 | 974 | 1519 | 644 | 532 | 862 | 0.002 |
| D-d 2 | 4399 | 3604 | 2398 | 1130 | 9000 | 4314 | 3824 | 1572 | 1356 | 9000 | |
| D-d 3 | 5571 | 3608 | 6775 | 1472 | 9000 | 6153 | 3149 | 7273 | 3880 | 9000 | 0.296 |
n, number;SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; ICU, intensive care unit; Med, median; CPR, C reactive protein (mg/l), PCT, procalcitonin (ng/ml); Glyc, Glycemia (mg/dl); D-d, D-dimer (µg/ml); 0, Admission; 2, NIV start; 3, NIV suspension.
The inflammatory reactants CRP and PCT turned out to be already altered on admission (Table 6), with a significant difference between discharged and deceased patients (106.2 vs 148.6 mg/l, p=0.023 and 0.24 vs 0.61 ng/ml, p=0.013, respectively (Table 7).
Table 6.
Main laboratory data in general study population.
| n=231 | Mean | SD | Med | 25 | 75 | P |
| CRP 0 | 142.3 | 69.0 | 129.2 | 95 | 201.3 | p 0.085 |
| CRP 2 | 157.3 | 79.0 | 158.0 | 102.2 | 250 | |
| CRP 4 | 129.1 | 105.7 | 133.8 | 19.1 | 249.7 | |
| PCT 0 | 0.57 | 0.83 | 0.30 | 0.15 | 0.62 | p 0.005 |
| PCT 2 | 1.28 | 14.1 | 0.63 | 0.26 | 1.48 | |
| PCT 4 | 0.96 | 1.25 | 0.49 | 0.16 | 1.15 | |
| Glyc 0 | 150 | 71 | 126 | 108 | 163 | p 0.844 |
| Glyc 2 | 153 | 57 | 137 | 116 | 178 | |
| Glyc 4 | 159 | 65 | 144 | 113 | 196 | |
| D-d 0 | 1829 | 2539 | 823 | 606 | 1236 | p 0.002 |
| D-d 2 | 4371 | 3646 | 2052 | 1157 | 9000 | |
| D-d 4 | 5774 | 3442 | 6866 | 1699 | 9000 |
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; CRP, C reactive protein (mg/l); PCT, procalcitonin (ng/ml): (0 vs 2 – p=0.003; 0 vs 4 – p= 0.643; 4 vs 2 – p=0.011); Glyc, glycemia (mg/dl); D-d, D-dimer (µg/ml): (0 vs 2 – p=0.029 0 vs 4 – p=0.001 4 vs 2 – p=0.195); 0, Hospital admission; 2, NIV start; 4, NIV suspension.
Table 7.
Main admission laboratory data in survived and deceased patient subgroups
| Survived (n=101) | Deceased (n=130) | ||||||||||
| Mean | SD | Med | 25 | 75 | Mean | SD | Med | 25 | 75 | p | |
| CRP 0 | 106.2 | 74.1 | 70.7 | 102.5 | 208.4 | 148.6 | 66.5 | 131.3 | 65.7 | 129.6 | 0.023 |
| PCT 0 | 0.24 | 0.34 | 0.15 | 0.17 | 0.63 | 0.61 | 0.87 | 0.34 | 0.08 | 0.18 | 0.013 |
| Glyc 0 | 142 | 58 | 123 | 111 | 164 | 156 | 80 | 127 | 104 | 159 | 0.210 |
| D-d 0 | 1963 | 3019 | 746 | 612 | 1267 | 1804 | 2464 | 823 | 541 | 948 | 0.488 |
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; CPR, C reactive protein (mg/l), PCT, procalcitonin (ng/ml); Glyc, glycemia 0 (mg/dl); D-d, D-dimer (µg/ml); 0, Hospital admission.
Mean RI on admission was 233±88, which subsequently worsened at anesthesiologist counseling (94). Evaluating the parameter trend separately in deceased and discharged patients, we observed that starting from similar values on admission (230 vs. 238 respectively) the difference between the two groups became more relevant at time point 1 (87 vs. 105) and was even higher during NIV (97 vs. 142) and at treatment discontinuation (77 vs. 166). A similar trend was seen for patients admitted to the ICU (Figure 3).
Figure 3.
Respiratory Index variation during NIV treatment, based on outcome (in-hospital death AND/OR ICU admission). Patients with the highest ratio between post- and pre-NIV RI had lower mortality and ICU admission rates (p<0.001), but the figure highlights the fact that the effect of NIV becomes more significant on a long-term basis than at the end of the first trial, suggesting that its success should be assessed over a longer period of time. Age was another statistically significant risk factor for negative hospital outcome (p=0.03).
In the background of our criteria (Figure 2), we identified 46 patients as NIV trial responders, representing 19.9 % of the general study population and 29.3 % of patients who spent their entire hospital stay in non-intensive wards. In this group, we observed significantly lower PCT at hospitalization (p=0.008) but higher mean BMI (p=0.017). (Table 8)
Table 8.
Anthropometric, admission laboratory data and main clinical outcomes in NIV trial responder and non-responder patient subgroups.
| Responder (n=46) | Non responder (n=172) | |||||||||||||||
| n | % | Mean | SD | Med | 25 | 75 | N | % | Mean | SD | Med | 25 | 75 | P | ||
| Age (years) | 58.5 | 10.5 | 59.0 | 54 | 63.8 | 65.7 | 10.5 | 67.0 | 60.8 | 72 | ||||||
| Weight (kg) | 93.0 | 19.9 | 90.0 | 77 | 109 | 84.1 | 18.1 | 83.0 | 74 | 92 | ||||||
| BMI (kg/m2) | 32.3 | 6.3 | 30.6 | 27.6 | 36.2 | 29.5 | 6.6 | 28.1 | 25.2 | 31.6 | 0.017 | |||||
| Hospital stay | 27.1 | 15.2 | 24.0 | 18 | 31 | 23.7 | 23.7 | 16.5 | 8 | 28 | ||||||
| Admission -NIV | 4.0 | 4.5 | 3.0 | 1 | 5.8 | 4.3 | 5.0 | 3.0 | 1 | 6 | ||||||
| NIV duration | 8.0 | 4.8 | 6.5 | 5 | 11.8 | 5.9 | 5.2 | 5.0 | 2 | 7 | ||||||
| Admission - ICU transfer | 8.9 | 6.9 | 8.0 | 4 | 10 | |||||||||||
| ICU stay | 18.3 | 18.6 | 14.0 | 3 | 26.8 | |||||||||||
| Gender (F) | 19 | 41.3 | 57 | 33.1 | ||||||||||||
| ICU transfer | 74 | 43 | ||||||||||||||
| Mortality | 130 | 75.6 | ||||||||||||||
n, number; SD, standard deviation Med, median; 25, 25th percentile; 75, 75th percentile; BMI, body mass index; NIV, non-invasive ventilation; ICU, intensive care unit; F, female; Admission-NIV, NIV duration, Admission-ICU transfer and ICU stay are expressed in days.
We also recorded pre-existing comorbidities in responder and non-responder subgroups (Table 9)
Table 9.
Comorbidities in NIV trial responder and non-responder subgroups.
| Responder (n=46) | Non responder (n=172) | |||
| n | % | N | % | |
| Diabetes | 15 | 32.6 | 49 | 28.5 |
| Hypertension | 21 | 45.7 | 110 | 64 |
| COPD | 8 | 17.4 | 14 | 8.1 |
| CKD | 0 | 0 | 8 | 4.7 |
| Cardiac disease | 5 | 10.9 | 33 | 19.2 |
| Vasculopathy | 0 | 0 | 9 | 5.2 |
| DVT/PE | 0 | 0 | 6 | 3.5 |
| Stroke/TIA | 0 | 0 | 6 | 3.5 |
| Malignancy | 0 | 0 | 12 | 7 |
| Liver disease | 1 | 2.2 | 13 | 7.6 |
| Others respiratory diseases | 9 | 19.6 | 19 | 11.0 |
n, number; COPD, chronic obstructive pulmonary disease; CKD, chronic kidney disease; DVT, deep vein thrombosis; PE, pulmonary embolism; TIA, transient ischemic attack.
The logistic regression of factors associated with in-hospital death and ICU admission after NIV trial showed age to be an obvious risk factor for negative outcome (p=0.03), as well as the fact that patients with the highest ratio between post- and pre-NIV RI had lower mortality and ICU admission rates (p<0.001) (Figure 3).
Discussion
Our study aimed to evaluate the clinical and organizational efficacy and safety of an incremental respiratory support protocol in a population of COVID-19 patients initially hospitalized in non-ICU wards. In the face of a full-scale emergency, characterized by an enormous imbalance between the request for ICU beds and their availability, studies from different countries show that 11%–62% of patients diagnosed with severe or critical SARS-CoV-2-related interstitial pneumonia were treated with non-invasive ventilation(13–15). In Italy, NIV was widely used, particularly outside the ICU, to deal with the enormous assistance burden we were facing at the peak of the first pandemic wave during the spring of 2020 introducing a novel organizational model (15, 16)
Even in the presence of contrasting evidence regarding NIV application in the context of other respiratory virus pandemics, with success rates varying from 7.6 % to 90% (10–12), our results indicate that NIV might be effective, especially in overweight and obese patients who are more prone to suffering the negative effects of invasive ventilation, if coupled with an integrated approach to the patient. In fact, mortality rate was comparable among patients who were assisted in ordinary wards and those who were transferred to the ICUs (57.3% vs 54%). This result is in-line with those provided by other studies (17, 18). Based on our criteria, 19.9 % of patients responded positively to NIV, avoiding the need for invasive ventilation and eventual death. This result is similar to the evidence already present in the literature (19), and we report a higher success rate compared with other studies, which showed failure rates up to 91.5% (20),
One of the first difficulties we had to face in our real-life experience was that NIV is an aerosol generating procedure (AGP) that can expose doctors, nurses and other healthcare workers to the risk of SARS-CoV-2 infection, as reported in previous studies describing the severe acute respiratory syndrome (SARS) epidemic (10, 21–23). However, there were no cases of transmission to the members of our medical team who correctly used personal protective equipment (PPE); this shows that with adequate precautions, NIV is safe in non-intensive settings, as already reported in other studies (19, 24–27)
Analysis of our data seems to suggest that supporting patients affected by SARS-CoV-2-related interstitial pneumonia with NIV from an early stage can contribute to improved oxygenation, and consistently reduce the need for ICU beds. This has multiple positive effects both on patient and caregiver perspective. On the one hand, non-invasive ventilation can decrease respiratory muscle fatigue, therefore the need to resort to endotracheal intubation (ETI), invasive ventilation-related complications, such as ventilator acquired pneumonia (VAP), delirium, ICU–acquired syndrome, and oversedation(28). On the other hand, we implemented an organizational model based on the cooperation between ward doctors and anesthesiologists, who acted as counselors and treated severely ill patients in 300-plus bed inpatient facilities. In this setting, anesthesiologists planned and followed NIV treatment, monitoring the clinical course and identifying those patients at risk of unfavorable outcome as quickly as possible to arrange transfer to an ICU. In the presence of contraindications, or when care intensity escalation was not possible, they administered palliative sedation.
On the other hand, NIV could be associated with relevant detrimental effects, such as acute lung injury caused by excessive tidal volume (10, 29)
Our results seem to suggest that NIV, delivered mainly by helmet in a non-intensive environment, might be highly effective for the treatment of COVID-19-related AHRF, if coupled with attentive and continuous counseling.
One of our most interesting findings was the fact that at time points 1 and 2 RI was not significantly different between patients who later developed a negative hospital outcome (in-hospital death or ICU admission) and those who were discharged from ordinary wards, indicating that the effect of NIV becomes evident over a longer period (Figure 3). This finding could prove to be useful to assess NIV effectiveness in multiple contexts, as its use has become more and more widespread in recent times, both in the pre-ICU and post-extubation setting,(6) and randomized clinical trials confirmed its beneficial effects in the presence of AHRF (3), even when caused by community acquired pneumonia (3, 30). Evidence also supports its implementation to prevent, or treat, postoperative respiratory failure (31), especially in obese patients (32).
In addition, mortality observed in patients who underwent endotracheal intubation after failing NIV (n=74, 32%) did not exceed the value we registered in patients who spent their entire hospital stay in non-intensive wards (54.1% vs. 57.3%), which was consistent with findings in other studies (33). Our mortality was lower compared with other centers in Italy(16).
These data, even if calculated on quite a small population of patients (n=231), appears to support the claim that a possible delay in ETI in COVID-19 patients, who did not show an effective response to the first NIV trial, should not be considered a risk-factor for death, and is not responsible for a worse outcome than in those subjects treated entirely in a non-invasive setting.
We performed analyzed CPR, PCT, D-dimer, glycemia and RI in an obese and non-obese population with regard to negative hospital outcome (in-hospital death and/or ICU admission) (Figure 4) finding a statistically significant difference for D-dimer at admission (p=0.014), which was lower in subjects with a BMI>30, who survived until hospital discharge and were not transferred to the ICU. This evidence, coupled with a higher mean BMI observed in the responder and survived patient groups (p=0.017 and 0.021, respectively), may suggest that those patients who were tendentially hospitalized earlier during their clinical course due to this risk factor might have reaped the benefit of prompter respiratory support initiation.
Figure 4.
Comparison between normal/overweight and obese population, based on unfavourable hospital outcome (in-hospital death and/or ICU admission). Normal/overweight patients: BMI < 30 kg/m2. Obese patients: BMI 30 kg/m2. Logistic regression was applied on flogistic reactants, blood glucose and D-dimer at hospital admission. Logistic regression: D-dimer p=0.014 Glu p=0.034 CRP p=0.4 Pct p=0.6 P/F p=0.3.
This study presents several limitations: it was designed as a retrospective, monocentric observational study and based on data collected during a full-scale emergency. Moreover, due to the overwhelming influx of patients, hospitalization was frequently possible only at an advanced-disease phase, characterized by an already relevant flogistic state that could have had a negative impact on patient outcome, even if pCO2 and pH on admission (35.5±7.8 mmHg and 7.46±0.05, respectively) showed that respiratory muscle exhaustion had not yet developed. This claim appears to be supported by our laboratory data, showing already significantly increased flogistic reactants values at hospital admission.
Conclusion
In conclusion, this approach is feasible and safe for healthcare workers, and has made it possible to reduce, or at least slow down, a simultaneous overwhelming ICU influx of patients with COVID-19-related ARHF(6, 8, 34) and mortality outcomes observed in our study population are comparable to the evidence found in the literature on the same topic (35).
It will be important to extend the use of the NIV method during possible future pandemic waves, including anthropometric and laboratory parameters and lung ultrasound (LUS (33, 36)), in a comprehensive patient evaluation model aimed at optimizing ventilatory support, as RI alone cannot be considered an accurate benchmark.
Artificial intelligence (AI) and machine learning (ML (37) ) will undoubtedly help to create the diagnostic and therapeutic predictive models that we desperately need to properly and fairly allocate resources to those patients who could reap the greatest prognostic benefit.
Conflict of Interest:
Each author declares that he or she has no commercial associations (e.g. consultancies, stock ownership, equity interest, patent/licensing arrangement etc.) that might pose a conflict of interest in connection with the submitted article
References
- Coronavirus disease 2019 (COVID-19) Situation Report – 67 [Google Scholar]
- Miller DC, Beamer P, Billheimer D, Subbian V, Sorooshian A, Campbell BS, et al. Aerosol risk with noninvasive respiratory support in patients with COVID-19. J. Am. Coll. Emerg. Physicians Open. 2020;1:521–526. doi: 10.1002/emp2.12152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Antonelli M, Conti G, Esquinas A, Montini L, Maggiore SM, Bello G, et al. A multiple-center survey on the use in clinical practice of noninvasive ventilation as a first-line intervention for acute respiratory distress syndrome. Crit. Care Med. 2007;35:18–25. doi: 10.1097/01.CCM.0000251821.44259.F3. [DOI] [PubMed] [Google Scholar]
- Villarreal-Fernandez E, Patel R, Golamari R, Khalid M, DeWaters A, Haouzi P. A plea for avoiding systematic intubation in severely hypoxemic patients with COVID-19-associated respiratory failure. Crit. Care. 2020;24:337. doi: 10.1186/s13054-020-03063-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paternoster G, Sartini C, Pennacchio E, Lisanti F, Landoni G, Cabrini L. Awake pronation with helmet continuous positive airway pressure for COVID-19 acute respiratory distress syndrome patients outside the ICU: A case series. Med. Intensiva. 2020 doi: 10.1016/j.medin.2020.08.008. S0210569120302734. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Demoule A, Chevret S, Carlucci A, Kouatchet A, Jaber S, Meziani F, et al. oVNI Study Group, REVA Network (Research Network in Mechanical Ventilation). Changing use of noninvasive ventilation in critically ill patients: trends over 15 years in francophone countries. Intensive Care Med. 2016;42:82–92. doi: 10.1007/s00134-015-4087-4. [DOI] [PubMed] [Google Scholar]
- Alviset S, Riller Q, Aboab J, Dilworth K, Billy P-A, Lombardi Y, et al. Continuous Positive Airway Pressure (CPAP) face-mask ventilation is an easy and cheap option to manage a massive influx of patients presenting acute respiratory failure during the SARS-CoV-2 outbreak: A retrospective cohort study. PloS One. 2020;15:e0240645. doi: 10.1371/journal.pone.0240645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brusasco C, Corradi F, Di Domenico A, Raggi F, Timossi G, Santori G, et al. Continuous positive airway pressure in Covid-19 patients with moderate-to-severe respiratory failure. Eur. Respir. J. 2020:2002524. doi: 10.1183/13993003.02524-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oranger M, Gonzalez-Bermejo J, Dacosta-Noble P, Llontop C, Guerder A, Trosini-Desert V, et al. Continuous positive airway pressure to avoid intubation in SARS-CoV-2 pneumonia: a two-period retrospective case-control study. Eur. Respir. J. 2020;56:2001692. doi: 10.1183/13993003.01692-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Alhazzani W, Møller MH, Arabi YM, Loeb M, Gong MN, Fan E, et al. Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19) Intensive Care Med. 2020:1–34. doi: 10.1007/s00134-020-06022-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bellani G, Laffey JG, Pham T, Madotto F, Fan E, Brochard L, et al. Noninvasive Ventilation of Patients with Acute Respiratory Distress Syndrome. Insights from the LUNG SAFE Study. Am. J. Respir. Crit. Care Med. 2017;195:67–77. doi: 10.1164/rccm.201606-1306OC. [DOI] [PubMed] [Google Scholar]
- Alraddadi BM, Qushmaq I, Al-Hameed FM, Mandourah Y, Almekhlafi GA, Jose J, et al. Saudi Critical Care Trials Group. Noninvasive ventilation in critically ill patients with the Middle East respiratory syndrome. Influenza Other Respir. Viruses. 2019;13:382–390. doi: 10.1111/irv.12635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guan L, Zhou L, Le Grange JM, Zheng Z, Chen R. Non-invasive ventilation in the treatment of early hypoxemic respiratory failure caused by COVID-19: considering nasal CPAP as the first choice. Crit. Care. 2020;24:333. doi: 10.1186/s13054-020-03054-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet Elsevier. 2020;395:497–506. doi: 10.1016/S0140-6736(20)30183-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Longhini F, Bruni A, Garofalo E, Navalesi P, Grasselli G, Cosentini R, et al. Helmet continuous positive airway pressure and prone positioning: A proposal for an early management of COVID-19 patients. Pulmonology. 2020;26:186–191. doi: 10.1016/j.pulmoe.2020.04.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lucchini A, Giani M, Isgrò S, Rona R, Foti G. The “helmet bundle” in COVID-19 patients undergoing non invasive ventilation. Intensive Crit. Care Nurs. 2020;58:102859. doi: 10.1016/j.iccn.2020.102859. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang D, Hu B, Hu C, Zhu F, Liu X, Zhang J, et al. Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus–Infected Pneumonia in Wuhan, China. JAMA. 2020;323:1061. doi: 10.1001/jama.2020.1585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pagano A, Porta G, Bosso G, Allegorico E, Serra C, Dello Vicario F, et al. Non-invasive CPAP in mild and moderate ARDS secondary to SARS-CoV-2. Respir. Physiol. Neurobiol. 2020;280:103489. doi: 10.1016/j.resp.2020.103489. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Raoof S, Nava S, Carpati C, Hill NS. High-Flow, Noninvasive Ventilation and Awake (Nonintubation) Proning in Patients With Coronavirus Disease 2019 With Respiratory Failure. Chest. 2020;158:1992–2002. doi: 10.1016/j.chest.2020.07.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duca A, Memaj I, Zanardi F, Preti C, Alesi A, Della Bella L, et al. Severity of respiratory failure and outcome of patients needing a ventilatory support in the Emergency Department during Italian novel coronavirus SARS-CoV2 outbreak: Preliminary data on the role of Helmet CPAP and Non-Invasive Positive Pressure Ventilation. EClinicalMedicine. 2020;24:100419. doi: 10.1016/j.eclinm.2020.100419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lynch JB, Davitkov P, Anderson DJ, Bhimraj A, Cheng VC-C, Guzman-Cottrill J, et al. Infectious Diseases Society of America Guidelines on Infection Prevention for Health Care Personnel Caring for Patients with Suspected or Known COVID-19. Clin. Infect. Dis. Off. Publ. Infect. Dis. Soc. Am. (Internet) 2020 doi: 10.1093/cid/ciaa1063. (cited 2021 Jan 28); Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7454357/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fowler RA, Guest CB, Lapinsky SE, Sibbald WJ, Louie M, Tang P, et al. Transmission of severe acute respiratory syndrome during intubation and mechanical ventilation. Am. J. Respir. Crit. Care Med. 2004;169:1198–1202. doi: 10.1164/rccm.200305-715OC. [DOI] [PubMed] [Google Scholar]
- Tran K, Cimon K, Severn M, Pessoa-Silva CL, Conly J. Aerosol generating procedures and risk of transmission of acute respiratory infections to healthcare workers: a systematic review. PloS One. 2012;7:e35797. doi: 10.1371/journal.pone.0035797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Privitera D, Angaroni L, Capsoni N, Forni E, Pierotti F, Vincenti F, et al. Flowchart for non-invasive ventilation support in COVID-19 patients from a northern Italy Emergency Department. Intern. Emerg. Med. 2020;15:767–771. doi: 10.1007/s11739-020-02370-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Montrief T, Ramzy M, Long B, Gottlieb M, Hercz D. COVID-19 respiratory support in the emergency department setting. Am. J. Emerg. Med. 2020;38:2160–2168. doi: 10.1016/j.ajem.2020.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Winck JC, Ambrosino N. COVID-19 pandemic and non invasive respiratory management: Every Goliath needs a David. An evidence based evaluation of problems. Pulmonology. 2020;26:213–220. doi: 10.1016/j.pulmoe.2020.04.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nielsen Jeschke K, Bonnesen B, Hansen EF, Jensen J-US, Lapperre TS, Weinreich UM, et al. Guideline for the management of COVID-19 patients during hospital admission in a non-intensive care setting. Eur. Clin. Respir. J. 2020;7:1761677. doi: 10.1080/20018525.2020.1761677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rochwerg B, Brochard L, Elliott MW, Hess D, Hill NS, Nava S, et al. Official ERS/ATS clinical practice guidelines: noninvasive ventilation for acute respiratory failure. Eur. Respir. J. 2017;50:1602426. doi: 10.1183/13993003.02426-2016. [DOI] [PubMed] [Google Scholar]
- Chiumello D, Brochard L, Marini JJ, Slutsky AS, Mancebo J, Ranieri VM, et al. Respiratory support in patients with acute respiratory distress syndrome: an expert opinion. Crit. Care Lond. Engl. 2017;21:240. doi: 10.1186/s13054-017-1820-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nicolini A, Ferraioli G, Ferrari-Bravo M, Barlascini C, Santo M, Ferrera L. Early non-invasive ventilation treatment for respiratory failure due to severe community-acquired pneumonia: Pneumonia and non-invasive ventilation. Clin. Respir. J. 2016;10:98–103. doi: 10.1111/crj.12184. [DOI] [PubMed] [Google Scholar]
- Nava S, Hill N. Non-invasive ventilation in acute respiratory failure. 2009;374:11. doi: 10.1016/S0140-6736(09)60496-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Imber DA, Pirrone M, Zhang C, Fisher DF, Kacmarek RM, Berra L. Respiratory Management of Perioperative Obese Patients. Respir. Care Respiratory Care. 2016;61:1681–1692. doi: 10.4187/respcare.04732. [DOI] [PubMed] [Google Scholar]
- Vetrugno L, Baciarello M, Bignami E, Bonetti A, Saturno F, Orso D, et al. The “pandemic” increase in lung ultrasound use in response to Covid-19: can we complement computed tomography findings? A narrative review. Ultrasound J. (Internet) 2020 doi: 10.1186/s13089-020-00185-4. (cited 2021 Feb 9); 12Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7422672/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- Faraone A, Beltrame C, Crociani A, Carrai P, Lovicu E, Filetti S, et al. Effectiveness and safety of noninvasive positive pressure ventilation in the treatment of COVID-19-associated acute hypoxemic respiratory failure: a single center, non-ICU setting experience. Intern. Emerg. Med. (Internet) 2020 doi: 10.1007/s11739-020-02562-2. (cited 2021 Jan 28); Available from: http://link.springer.com/10.1007/s11739-020-02562-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cerceo E, Fraimow H. Lessons Learned From the Front Line: Outcomes of Noninvasive Ventilation for Coronavirus Disease 2019 Pneumonia in China. 2020;48:3. doi: 10.1097/CCM.0000000000004484. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nouvenne A, Zani MD, Milanese G, Parise A, Baciarello M, Bignami EG, et al. Lung Ultrasound in COVID-19 Pneumonia: Correlations with Chest CT on Hospital admission. Respiration. 2020:1–8. doi: 10.1159/000509223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ferrari D, Milic J, Tonelli R, Ghinelli F, Meschiari M, Volpi S, et al. Machine learning in predicting respiratory failure in patients with COVID-19 pneumonia-Challenges, strengths, and opportunities in a global health emergency. PloS One. 2020;15:e0239172. doi: 10.1371/journal.pone.0239172. [DOI] [PMC free article] [PubMed] [Google Scholar]