Abstract
Purpose
In recent years, a variety of acute respiratory distress syndrome (ARDS) evaluation systems have been developed worldwide; however, they are not so convenient for the doctors clinically, we decided to establish and evaluate a simplified evaluation system of ARDS (SESARDS).
Materials and Methods
Data from 140 ARDS patients (derivation data set) were collected to screen for prognostic factors affecting outcomes in ARDS patients. By logistic regression analysis, scores were allocated to corresponding intervals of the variables, respectively, by means of analysis of the frequency distribution to establish a preliminary scoring system. Based on this primary scoring system, a definitive evaluation scheme was created through consultation with a panel of experts. The scores for the validation data set (92 cases) were assigned and calculated by their predictive mortality with the SESARDS and acute physiology and chronic health evaluation II (APACHE II). The performance of SESARDS was compared with that of APACHE II by means of statistical analysis.
Results
The factors of age, pH, Glasgow coma scale (GCS), oxygenation index (OI), and the lobes of lung were associated with prognosis of ARDS respectively. The sensitivity and specificity of SESARDS for the validation data set were 96.43% and 58.33%, respectively. On the AUC, no significant difference between APACHE II and SESARDS was detected. There were no significant differences between the prediction and the actuality obtained by SESARDS for the validation data set the SESARDS scores were positively correlated with the actual mortality.
Conclusion
SESARDS was shown to be simple, accurate and effective in predicting ARDS progression.
Keywords: Acute respiratory distress syndrome, evaluation of severity, scheme evaluation
INTRODUCTION
Presently, the mortality for acute respiratory distress syndrome (ARDS) is high.1,2 Jiang and Hu3 proposed that the currently available methods for evaluating ARDS disease severity are not reflective of the actual severity and can be inconsistent. However, these evaluation systems can be effective in indicating the degree of critical illness as well as in evaluating treatment and outcomes. In recent years, a variety of ARDS evaluation systems, such as acute physiology and chronic health evaluation (APACHE), simplified acute physiology score (SAPS), mortality probability models and multi-organ-dysfunction score, have been used worldwide; however, they are not so convenient for the doctors clinically, especially for those working in undeveloped and developing regions of the world. Recently, ARDS score was developed based on lung injury score at an American-European Consensus Conference in 1994; however, it has its limitations. On one hand, it is difficult to assess early lung injury in ARDS patients without mechanical ventilation, while on the other hand, positive end expiratory pressure (PEEP) for improvement of an oxygenated effect is dependent on time and the individual. Heffner, et al.4 reported that both lung injury score and ARDS score are accurate and affirmative in the diagnosis of ARDS, but they are no more effective than SAPS score in predicting mortality. Therefore, a more practical and simplified evaluation system for ARDS would be of great benefit.
MATERIALS AND METHODS
Subjects and grouping
From June 1999 to October 2008, data from the medical records of 232 patients from the Department of Respiratory Diseases of Nanfang Hospital, Shunde First People's Hospital, Guangzhou Chest Hospital; Department of Respiratory Diseases of Huizhou People's Central Hospital; Department of Respiratory Diseases, Pearl River Hospital, Southern Medical University; Xinhui People's Hospital, the Fifth Affiliated Hospital of Sun Yat-sen University; Xiangzhou People's Hospital of Zhuhai; Gaoming People's Hospital of Foshan; Guizhou Hospital of Foshan and Shenzhen Third People's Hospital were included in our study. Data were randomized into derivation data set (n=140) and validation data set (n=92). The derivation data set was subdivided into death group (n=90; male 67, female 23; averaged 41.85+14.51 years) and survival group (n=50; male 35, female 15; averaged 41.30+13.86 years); the validation data set was also divided into a death group (n=56; male 38, female 18; averaged 49.05+20.54 years) and survival group (n=36; male 26, female 10; averaged 46.56+18.36 years). Use patient records for our study was approved by Nanfang Hospital and patient confidentiality was maintained.
Diagnosis criteria for ARDS patients
Diagnosis of ARDS was made according to the criteria proposed at the American-European Consensus Conference on ARDS in 1994. In all 232 eligible patients, diagnostically, the ratio of the partial pressure of arterial oxygen (PaO2) to the fraction of inspired oxygen was less than 200 (adjusted if the altitude exceeded 1000 m), and bilateral infiltrations were evident on chest radiography, consistent with the presence of pulmonary edema without evidence of left atrial hypertension.5
Methods
Data collection
Physiological measurements, age and previous health status were used to calculate APACHE II score. Clinical parameters for ARDS were obtained within the first 24 hours after the diagnosis of ARDS was made, and included 1) age, sex, and comorbidities; 2) respiratory rate, blood pressure, body temperature, and heart rate; 3) white cell count and hematocrit; 4) serum level of sodium, potassium, creatinine and glucose; 5) arterial blood gas including pH, PaO2, PaCO2, and oxygenation index (OI); 6) Glasgow coma scale (GCS); 7) presence of ARDS (p); 8) number of risk factors for ARDS; 9) number of lobes involved; 10) amount of extra-pulmonary organ dysfunction or failure; and 11) vital status at discharge.
Assignment for indexes
In 1998, Gattinoni, et al.6 reported that pathological changes in pneumonia-induced ARDS were significantly different, in addition to the effects on PEEP from ARDS caused by abdominal disease. Accordingly, he classified ARDS into two categories: ARDS (p) and ARDS (exp), and then assigned scores of 1 to the former and 0 to the latter, when applying them to statistical treatment. For our statistical purpose, likewise, we handled the score assignments in this study for the four lobes of the lung (left-up, left-down, right-up and right-down) according to the standards for evaluating acute lung lesions. According to the scope of inflammation on chest imaging, we assigned scores of 1-4 for the four lobes, respectively. Then, we assigned scores of 1-5 for the total number of indicators of nonpulmonary organ dysfunction present, including alimentary tract hemorrhage, acute renal failure, liver function failure, nervous function failure, and function failure of the hematological system.
Statistical analysis
Statistical analysis was conducted based on the retrospective case-control study, and results of the descriptive statistics were presented as means±SD. The comparison between two means was analyzed by independent-sample t-test, and the frequency distributions in categorized variables were compared using Pearson's chi-square test. The risk factors affecting ARDS prognosis were screened by univariate logistic regression and multivariate logistic regression (forward conditional). Then, a probabilistic equation for a simplified evaluation system of acute respiratory distress syndrome (SESARDS) was established by full-factor logistic regression, and thereafter, the quartile frequency distribution method was used to establish a preliminary valuation system, from which a definitive evaluation scheme of SESARDS was developed and revised after consultation with a panel of experts. Finally, SESARDS and APACHE II were compared in regarding to validity, sensitivity, and specificity using receiver operating characteristics curves (ROC) curves. SPSS software version 13.0 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis and plotting of graphs, scatter plots and ROC curves, as well as calculation of the AUC of ROCs.
RESULTS
Baseline demographics
Statistically, the differences between the two sets were insignificant in regards to age, sex, etiology of ARDS and mortality (Table 1).
Table 1.
Baseline Demographics for Both Data Sets
ARDS, acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.
Results of Univariate Logistic Regression Analysis (Table 2)
Table 2.
Results of Univariate Logistic Regression Analysis
GCS, Glasgow coma scale; OI, oxygenation index; DBP, diastolic blood pressure; MBP, mean blood pressure; Hct, hematocrit.
*Amount of nonpulmonary organ dysfunction.
Factors influencing ARDS prognosis
After univariate logistic regression analysis, the variables of diastolic blood pressure, P(A-a)O2, pH, hematocrit, GCS, OI, the sum of lung field, amount of nonpulmonary organ dysfunction and MBP were used in a logistic regression model to screen for factors influencing the prognosis of ARDS, which included age, pH, GCS, OI and lobes of the lung (Table 3).
Table 3.
Factors Influencing ARDS Prognosis on Multivariate Logistic Regression Analysis
ARDS, acute respiratory distress syndrome; GCS, Glasgow coma scale; OI, oxygenation index.
Development of SESARDS
Frequency analysis
The five factors for the derivation data set were treated with frequency analysis for their respective values when the percentiles were set at 25%, 50% and 75%.
The results of frequency analysis were used to establish the grades of each factor, taking into consideration the actual values of all five factors. For the actual age and lobes of the lung, we assigned to 0-25% a score of 0, 25-50% a score of 1, 50-75% a score of 2 and to 75-100% a score of 3, for both variables. For the two variables of pH and OI, we assigned to 0-25% a score of 3, 25-50% a score of 2, 50-75% a score of 1 and to 75-100% a score of 0. For the GCS variable we assigned to 0-75% a score of 1 and to 75-100% a score of 0.
The weights of the variables
Referring to Wang,7 based on the non-parametric statistics, we sent e-mails to 15 experienced experts asking them to sort these five variables according to their level of importance as indicators on the prognosis of patients with ARDS. The order of the sorted items was to be marked as serial numbers: from number 1 (as the most important) to number 5 (as the least important). To calculate the weight of each variable, we combined the serial numbers (1 to 5) with the Weight Calculation Sheet and used the following formula:
F formulation: F=∑[(n+1-i)fi], where n=5, i=serial number and fi=the frequency of rank i (Table 6).
Table 6.
The Weights of the Variables
GCS, Glasgow coma scale; OI, oxygenation index.
Establishment of SESARDS
All the weights of the variables together with their grades were revised to establish the final SESARDS (Table 7).
Table 7.
The SESARDS Severity of Disease Classification System
GCS, Glasgow coma scale; OI, oxygenation index.
The probability of death equation
Subsequently, the scores of SESARDS, as an independent variable, were applied to logistic regression using the possibility of death as a dependent variable, to work out the probability of death equation:
 |
Comparisons between SESARDS and APACHE II
The Validity of SESARDS
The ROC curves of SESARDS and APACHE II
ROC was used to test the validity of SESARDS. The results showed no significant differences between SESARDS and APACHE II, statistically (Fig. 1).
Fig. 1.
ROC curves of SESARDS and APACHE II. ROC, receiver operating characteristics curves; SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.
The sensitivities, specificities and accuracies of SESARDS and APACHE II
SESARDS and APACHE II were used on the validation data set for the comparisons of prediction of survival/death and actuality of survival/death (Table 8).
Table 8.
Comparisons between SESARDS and APACHE II in Sensitivity, Specificity and Accuracy
SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.
The reliability of SESARDS
Comparisons between SESARDS and APACHE II were made in regards to their reliability by way of Lemshow-Hosmer test.
The relationship between SESARDS score and actual mortality
SESARDS was used to evaluate and score the cases studied from which actual mortalities were calculated. Finally, the SESARDS scores and the actual mortalities were compared, shown to be positively correlated (Table 10, Fig. 2).
Table 10.
The Relationship between SESARDS Score and Actual Mortality
SESARDS, simplified evaluation system of acute respiratory distress syndrome.
Fig. 2.
The co-relationship line between SESARDS score and actual mortality. SESARDS, simplified evaluation system of acute respiratory distress syndrome.
There were significant differences between the survival group and death group for both data sets, in view of SESARDS score (10.726±3.296 vs. 6.162±3.136)
Comparison between SESARDS and APACHE II (Table 11)
Table 11.
Comparison between SESARDS and APACHE II
SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.
DISCUSSION
An effective prediction system can accurately predict changes in patients, but should consist of relatively simple variables. This paper presents a newly developed SESARDS as well as the validation results thereof. As a severity of disease classification system, SESARDS uses points based upon the initial values of age, pH, GCS, OI, and lobes of the lung. In this study, an increasing score (range 0 to 18) was shown to be closely related with subsequent risk of hospital death in ARDS. When SESARDS scores were evaluated for the ability to determine ARDS prognoses, the results were strong and stable. Our data further indicated that classification could be made appropriately if done at an early point in time. This would allow for the severity classification to be more independent from treatment.
APACHE II, established by Knaus, et al.8 in 1985, is one of the most widely used evaluation systems and also the most authoritative method for scoring critical illness at present. It is characterized by excellent performance in predicting mortality in ARDS patients over lung injury score, focusing only on the severity of respiratory failure. APACHE II and SAPS II scores have been previously reported to be independent predictors of hospital mortality in ARDS patients.9-12 According to Oh, et al.,13 APACHE II score is closely related to mortality in patients in the Intensive Care Unit (r=0.81). However, APACHE II is obtained by dividing the aggregate sum by the number of components used, and the 14 components used in this system may make it more difficult for doctors to use it clinically.14
After development, we applied SESARDS to the validation data set for evaluation of its reliability and validity in comparison to APACHE II, and found that there were no significant differences in accuracy in predicting ARDS incidence, nor sensitivity or specificity. Importantly, the AUC of the ROC for SESARDS was not different from that of APACHE II statistically. Moreover, it was found that there was no significant difference between the predictive results and the actual results when SESARDS was used on the validation data set, indicating that SESARDS was satisfactory in regards to goodness-of-fit. At last, our study provided evidence that SESARDS score was positively correlated with the actual mortality ratio. Moreover, SESARDS is relatively simpler than APACHE II (5 variables compared to 14).
It should be emphasized that in the first 24 hours SESARDS did not perfectly predict death rates in individual patients. However, our data indicated that misclassification rates became smaller as the probability of death increased (Fig. 1). Nevertheless, the specificity of SESARDS (58.33%) was unsatisfactory, though it is based on a larger number of samples, compared to several other ARDS-specific studies.15,16
In conclusion, SESARDS was shown to be simple, accurate and effective in predicting ARDS progression.
Table 4.
Results of Frequency Analysis
GCS, Glasgow coma scale; OI, oxygenation index.
Table 5.
The Initial SESARDS
SESARDS, simplified evaluation system of acute respiratory distress syndrome; GCS, Glasgow coma scale; OI, oxygenation index.
Table 9.
The Reliabilities of SESARDS and APACHE II Via Lemshow-Hosmer Test
SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.
Footnotes
The authors have no financial conflicts of interest.
References
- 1.Luhr OR, Antonsen K, Karlsson M, Aardal S, Thorsteinsson A, Frostell CG, et al. The ARF Study Group. Incidence and mortality after acute respiratory failure and acute respiratory distress syndrome in Sweden, Denmark, and Iceland. Am J Respir Crit Care Med. 1999;159:1849–1861. doi: 10.1164/ajrccm.159.6.9808136. [DOI] [PubMed] [Google Scholar]
- 2.Qiu H, Chen D, Liu D. Investigation of acute respiratory distress syndrome: predictors of mortality and clinical therapeutic strategies. Chin Criti Care Med. 1998;9:523. [Google Scholar]
- 3.Jiang X, Hu N. Chinese version of "critical illness scoring system" computer software. Chin Criti Care Med. 2000;4:246–247. [Google Scholar]
- 4.Heffner JE, Brown LK, Barbieri CA, Harpel KS, DeLeo J. Prospective validation of an acute respiratory distress syndrome predictive score. Am J Respir Crit Care Med. 1995;152(5 Pt 1):1518–1526. doi: 10.1164/ajrccm.152.5.7582287. [DOI] [PubMed] [Google Scholar]
- 5.Bernard GR, Artigas A, Brigham KL, Carlet J, Falke K, Hudson L, et al. The American-European Consensus Conference on ARDS. Definitions, mechanisms, relevant outcomes, and clinical trial coordination. Am J Respir Crit Care Med. 1994;149(3 Pt 1):818–824. doi: 10.1164/ajrccm.149.3.7509706. [DOI] [PubMed] [Google Scholar]
- 6.Gattinoni L, Pelosi P, Suter PM, Pedoto A, Vercesi P, Lissoni A. Acute respiratory distress syndrome caused by pulmonary and extrapulmonary disease. Different syndromes? Am J Respir Crit Care Med. 1998;158:3–11. doi: 10.1164/ajrccm.158.1.9708031. [DOI] [PubMed] [Google Scholar]
- 7.Wang LD. Sort indicators, one of the methods to weigh the variables. J Hubei Sports Sci (Chinese) 1993;4:51–54. [Google Scholar]
- 8.Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818–829. [PubMed] [Google Scholar]
- 9.Monchi M, Bellenfant F, Cariou A, Joly LM, Thebert D, Laurent I, et al. Early predictive factors of survival in the acute respiratory distress syndrome. A multivariate analysis. Am J Respir Crit Care Med. 1998;158:1076–1081. doi: 10.1164/ajrccm.158.4.9802009. [DOI] [PubMed] [Google Scholar]
- 10.Page B, Vieillard-Baron A, Beauchet A, Aegerter P, Prin S, Jardin F. Low stretch ventilation strategy in acute respiratory distress syndrome: eight years of clinical experience in a single center. Crit Care Med. 2003;31:765–769. doi: 10.1097/01.CCM.0000055402.68581.DC. [DOI] [PubMed] [Google Scholar]
- 11.Vieillard-Baron A, Girou E, Valente E, Brun-Buisson C, Jardin F, Lemaire F, et al. Predictors of mortality in acute respiratory distress syndrome. Focus On the role of right heart catheterization. Am J Respir Crit Care Med. 2000;161:1597–1601. [PubMed] [Google Scholar]
- 12.Brun-Buisson C, Minelli C, Bertolini G, Brazzi L, Pimentel J, Lewandowski K, et al. Epidemiology and outcome of acute lung injury in European intensive care units. Results from the ALIVE study. Intensive Care Med. 2004;30:51–61. doi: 10.1007/s00134-003-2022-6. [DOI] [PubMed] [Google Scholar]
- 13.Oh TE, Hutchinson R, Short S, Buckley T, Lin E, Leung D. Verification of the Acute Physiology and Chronic Health Evaluation scoring system in a Hong Kong intensive care unit. Crit Care Med. 1993;21:698–705. doi: 10.1097/00003246-199305000-00013. [DOI] [PubMed] [Google Scholar]
- 14.Knaus W, Wagner D, Draper E. APACHE III study design: analytic plan for evaluation of severity and outcome in intensive care unit patients. Development of APACHE. Crit Care Med. 1989;17(12 Pt 2):S181–S185. [PubMed] [Google Scholar]
- 15.Zilberberg MD, Epstein SK. Acute lung injury in the medical ICU: comorbid conditions, age, etiology, and hospital outcome. Am J Respir Crit Care Med. 1998;157(4 Pt 1):1159–1164. doi: 10.1164/ajrccm.157.4.9704088. [DOI] [PubMed] [Google Scholar]
- 16.Doyle RL, Szaflarski N, Modin GW, Wiener-Kronish JP, Matthay MA. Identification of patients with acute lung injury. Predictors of mortality. Am J Respir Crit Care Med. 1995;152(6 Pt 1):1818–1824. doi: 10.1164/ajrccm.152.6.8520742. [DOI] [PubMed] [Google Scholar]