Mortality Prediction in Rural Kenya: A Cohort Study of Mechanical Ventilation in Critically Ill Patients

Importance:

Critical care is expanding in low- and middle-income countries. Yet, due to factors such as missing data and different disease patterns, predictive scores often fail to adequately predict the high rates of mortality observed.

Objectives:

We evaluated multiple prognostic models for the outcome of mortality in critically ill, mechanically ventilated patients in rural Kenya and examined factors contributing to mortality in our setting.

Design, Setting, and Participants:

A prospective cohort study was conducted on mechanically ventilated patients in rural Kenya. Consecutive patients 16 years old and older initiated on mechanical ventilation between January 1, 2016, and April 30, 2017, at Tenwek Hospital were included. Demographic data, clinical characteristics, and patient outcomes were collected during routine clinical care.

Main Outcomes and Measures:

We assessed the discrimination and calibration of multiple previously-described models for mortality: Acute Physiology and Chronic Health Evaluation II, Sequential Organ Failure Assessment, quick Sequential Organ Failure Assessment, Simplified Acute Physiology Score II, Modified Early Warning Score, Tropical Intensive Care Score, Rwanda-Mortality Predictive Model, Vitals score (validated in Tanzania), and Vitals score for sepsis (validated in Uganda). Factors most associated with mortality were analyzed in our cohort utilizing stepwise regression.

Results:

Among the final cohort of 300 patients, the overall mortality rate was 60.7%, the average age was 39.9 years, 65% were male, and 33% were seen at an outside facility prior to admission to the critical care unit. Missing variables occurred in patients for numerous models but were complete in most adapted to resource-limited settings. Models displayed moderate prediction of mortality and variable discrimination area under the receiver operating characteristic curves (and Hosmer-Lemeshow chi-square statistic) of 0.77 (22.4) for Acute Physiology and Chronic Health Evaluation II, 0.70 (3.4) for Modified Early Warning Score, 0.65 (0.16) for quick Sequential Organ Failure Assessment, 0.55 (18.4) for Simplified Acute Physiology Score II and 0.74 (9.2) for Rwanda-Mortality Predictive Model, 0.72 (0.12) for Vitals Tanzania, 0.68 (14.7) for Vitals Uganda, and 0.65 (13.9) for Tropical Intensive Care Score. Variables associated with increased mortality in our population were hypotension, infection, traumatic brain injury, and hematocrit.

Conclusions and Relevance:

Overall, survival for critically ill patients in rural Kenya was poor, but predictable with contributing factors. Models designed for resource-constrained settings had favorable discrimination and better calibration for mortality prediction than high-resource models in our population of mechanically ventilated, critically ill patients in rural Kenya.

Critical care is expanding in low- and middle-income countries (LMICs) (1, 2). The majority of ICUs are concentrated in large referral hospitals in urban areas (3). Mechanical ventilation is increasingly available and used in resource-constrained environments, but little data has emerged regarding its use in such settings (4, 5). Reported mortality rates in critical care units are high (40–80%) in LMICs, especially among ventilated patients (6–9).

Despite the increase in critical care services, prognostic models for mortality prediction in resource-limited settings are limited due to their validation in only high-income countries, the frequency of missing variables, and the distinct clinical situations and characteristics (10, 11). Many of these scores require numerous variables and data points (Table 1) that may not be readily available in low-resource settings. A handful of predictive scores have been developed and adapted in resource-constrained settings; however, most scores have not been validated in similar settings (7, 15–18). Yet, the ability to prognosticate is especially important in resource-constrained settings where critical care is not widely available and decisions regarding resource utilization take on even greater importance (19–21). The objectives of this descriptive cohort study were to review the experience with mechanical ventilation at a hospital in rural Kenya, assess the discrimination and calibration of multiple prognostic models for the outcome of mortality, and investigate factors associated with mortality in our setting.

TABLE 1.

Various Critical Care Scores and the Necessary Variables

MATERIALS AND METHODS

We prospectively collected data on patients who presented to Tenwek Hospital and were initiated on mechanical ventilation between January 1, 2016, and April 30, 2017. Prior to data collection, approval for the study was obtained through the Tenwek Hospital Institutional Review and Ethics Committee. Tenwek Hospital is a teaching and referral hospital in rural Kenya serving a large population with critical care services since 2005. To review and describe the experience of a cohort of patients who were initiated on mechanical ventilation in our resource-constrained setting, we collected demographic data, clinical characteristics, and outcomes.

The primary outcome was defined as mortality during hospitalization. Patients were followed until death or hospital discharge. If an outpatient post-admission encounter was present, then the date of last follow-up was obtained. The exposure was numerous available predictive models for mortality. To assess each model’s applicability to our setting, we did not obtain additional laboratory investigations or other data relevant to scoring systems during the care of patients and instead continued with routine clinical management. Although the absence of data could introduce bias into the scores’ predictive ability, this allowed understanding for how models would routinely perform in our real-world, resource-constrained setting. Variables that we attempted to collect included age, sex, primary service, date of admission and discharge from the hospital and critical care unit, date of intubation and extubation, re-intubation, days of ventilation, whether the patient was transferred from an outside facility, any complications of ventilation, the indication for ventilation, presence of trauma at admission, discharge location, and all of the variables from the predictive scores listed in Table 1. These variables were handled and grouped as determined by their use in each predictive score. All patients admitted to the critical care unit who were initiated on mechanical ventilation were included. Patients who underwent cardiac surgery were excluded as they were routinely managed for significant portions of their hospitalization in the postoperative recovery area and predictive critical care scores often do not reflect their mortality risk in other settings that have been evaluated (22). In the analysis of predictive scores, we excluded all patients under the age of 16 years to remain consistent with other published reports (15). The inclusion and exclusion criteria for the study were designed a priori to improve generalizability to other centers with similar resources. To externally validate predictive scores, we decided to evaluate our cohort after determining there would be at least 100 events and 100 nonevents, accounting for potential missing information (23). The population of patients undergoing mechanical ventilation was determined to avoid selection bias of patients admitted to a critical care unit without critical care needs, such as postoperative patients for routine monitoring (24).

Predictive scores were calculated using various models. There were no planned analyses to adjust for missing data because the objective was to review how the scores would perform for patients with the information available during routine clinical care. Prior to analysis, charts were retrospectively reviewed for any missing data points. With available data, we assessed discrimination and calibration (25) of multiple previously-described models: Acute Physiology and Chronic Health Evaluation (APACHE) II (12), Sequential Organ Failure Assessment (SOFA) and quick Sequential Organ Failure Assessment (qSOFA) (13), Simplified Acute Physiology Score (SAPS) II (14), Rwanda-Mortality Predictive Model (R-MPM) (developed and validated in Rwanda) (15), Vitals score (validated in Tanzania) (7), Vitals score for sepsis (validated in Uganda) (16), Tropical Intensive Care Score (TropICS) (17), and Modified Early Warning Score (MEWS) (16, 18). These models were selected due to their frequency in critical care literature (APACHE II, SOFA and qSOFA, SAPS II, MEWS) or the comparability with our resource-constrained setting (R-MPM, TropICS, Vitals scores in Uganda and Tanzania). Scores were calculated per their defined criteria. Logistic regression with the binary outcome of mortality, defined as death at hospital discharge, was evaluated with each score to determine an odds ratio. Wald test was used for statistical significance of the predictor score. Receiver operating characteristic (ROC) curves were formed to calculate the area under the curve for discrimination of the model (AUROC), and Hosmer-Lemeshow goodness-of-fit tests were performed to assess calibration. Higher values for Hosmer-Lemeshow goodness-of-fit tests demonstrate a lack of fit, or discrimination, for the model, so acceptable calibration is typically defined by a nonsignificant Hosmer-Lemeshow value (p > 0.05). A subset analysis was performed on the vitals score for sepsis from Uganda (16) and the qSOFA score on patients who presented with signs of or concern for infection.

To understand factors associated with mortality in our cohort, we performed logistic regression for mortality prediction with backward stepwise elimination. Each potential variable was assessed for association with mortality and included in the model if a statistically significant (p < 0.05) association was found. To identify contributing variables, the probability of entry to the predictive model was alpha equals to 0.05 and removal at alpha equals to 0.055. We completed analyses using Stata software Version 14.2 (StataCorp LP, College Station, TX).

RESULTS

Among 334 consecutive patients, 300 were age 16 years or older. The overall mortality rate was 60.7%. Demographics, laboratory values, and clinical characteristics among the entire cohort, survivors, and nonsurvivors are displayed in Table 2. The median duration on mechanical ventilation was 3 days (interquartile range [IQR], 2–5 d), ICU stay was 4 days (IQR, 3–8 d), and hospital stay was 7 days (IQR, 4–12 d). Admission details and indications for ventilation are displayed in Table 3. Of the survivors, 63% had documentation of 30-day follow-up, and all but one were alive. Among patients with signs of or concerns for infection (n = 94), the AUROC curve was 0.51 for the vitals score from Uganda and 0.57 for the qSOFA score.

TABLE 2.

Demographics and Clinical Characteristics of a Critically Ill Population in Rural Kenya Undergoing Mechanical Ventilation

TABLE 3.

Indication for Mechanical Ventilation Among Survivors and Nonsurvivors

Missing variables for patients occurred in numerous models: SOFA (285), SAPS II (300), APACHE II (212), and TropICS (300). ROC curves of the evaluated models are shown in Table 4 and graphically displayed in Figure 1. Factors associated with mortality are presented in Table 5.

TABLE 4.

Discrimination and Calibration of Prediction Models in a Critically Ill Population in Kenya

Figure 1.

Discrimination of mortality prediction scores. Receiver operating characteristic curves where area under the curve (AUROC) demonstrates accuracy of predicted mortality for each scoring system evaluated. APACHE = Acute Physiology and Chronic Health Evaluation, MEWS = Modified Early Warning Score, qSOFA = Quick Sequential Organ Failure Assessment, R-MPM = Rwanda-Mortality Predictive Model, SAPS = Simplified Acute Physiology Score, SOFA = Sequential Organ Failure Assessment, TropICS = Tropical Intensive Care Score.

TABLE 5.

Factors Associated With Mortality in Mechanically Ventilated Patients in Rural Kenya As Determined by Stepwise Regression

DISCUSSION

Overall, survival for critically ill patients on mechanical ventilation in rural Kenya was poor but predictable. Numerous models were comparable with moderate prediction. Scores developed in resource-constrained settings similar to ours had the best discrimination and calibration, in particular, the R-MPM and Vitals score from Tanzania, in comparison to scores developed in high-resource settings. The APACHE II score had the best discrimination for mortality prediction (AUROC: 0.77) in our population of mechanically ventilated, critically ill patients in rural Kenya. However, it was poorly calibrated with a significant Hosmer-Lemeshow value, possibly due to missing data. However, it is noteworthy that in the 88 patients for whom the datasets were complete for calculation of APACHE II score, this score actually had a worse mortality prediction. We identified a number of factors to be associated with mortality in our setting.

Data from our population was missing in numerous models including those developed for both high-income (APACHE II, SAPS II, SOFA) and a low- or middle-income country (TropICS). Our study evaluated the routine care of patients and how the scores might apply to current care. However, the study did not evaluate the potential of the scores if all variables were collected (e.g., ordering an arterial blood gas or bilirubin to complete the score instead of to guide clinical decision-making). Given the concerns in other resource-limited settings of minimizing nonessential resource use, this seems to be a strength of our study to demonstrate how various scores might perform in a real-world resource-limited setting.

The study may be limited by the decision to include only mechanically ventilated patients. To avoid inappropriate comparisons to other institutions and to undertake the resource-intensive collection of the necessary data, we elected to exclude patients admitted to critical care units without mechanical ventilation. Future studies may benefit from evaluation of all patients admitted to the critical care unit with the understanding that such inclusion may not be generalizable to other similar settings with different thresholds for admission to the critical care unit. Often critical care decisions are based on a system of triage with ICUs being full of the sickest patients currently in the hospital. If there were more beds and resources available, other patients would also qualify for admission.

Our mortality rate, 61%, appears comparable to other resource-constrained settings. In a review from Nigeria, their overall mortality rate was 32%, but was much higher, 63%, among mechanically ventilated patients (26). A similar pattern was observed in Northern Uganda, where the overall mortality was 27%, and ventilation, which was used sparingly, had a 53% mortality (27). Mbarara regional referral hospital reported an overall mortality rate of 38%, but had a 74% mortality rate among ventilated patients (28). These reports seem consistent with the challenges faced while providing critical care with limited resources and our results would likely be generalizable to setting similar to ours throughout rural Africa. Beyond the scores’ abilities to prognosticate, improving survival in the intensive care settings of LMICs requires ongoing advances to critical care services (29), barriers to which include lack of training, lack of nurses, and low wages (30).

As shown in our cohort, the indication for ventilation was often predictive of survival. Survival was noted to be poor in patients intubated who had hypotension or abnormal pH. A lower hematocrit was associated with worsened survival perhaps reflecting the chronicity of problems in a setting where delayed presentation is common. Certain conditions were associated with survival as postoperative patients (p = 0.03) and patients who self-poisoned, typically with organophosphates (p < 0.001), had improved survival. Trauma (31), specifically traumatic brain injury (32), accounts for a high burden of disease with poor survival in critical care units (9) and is a major burden of disease in resource-constrained settings (33).

Some scores, such as SOFA and qSOFA, have been developed to aid clinical decision-making and not for prediction, whereas others, such as the Vitals (Uganda), were developed for specific conditions such as sepsis. For comparison, we examined these scores in our population. Further delineation of patient disease could improve predictive scores; however, this must be balanced with ease of use and applicability of the score to the setting. Other centers have demonstrated the impact of sepsis on survival, with mortality as high as 80% in resource-constrained environments (8). Our findings in patients with suspected or confirmed infection are similar as mortality in this subset of patients was 84% (p < 0.001). The Vitals (Uganda) (3) and qSOFA was designed for sepsis and suspicion of infection, but we evaluated its predictive ability for all patients. In a subset of 94 patients with infection in our cohort, the ROC for the vitals score was worse in comparison to all patients. The presence of infection is highly correlated with mortality (p < 0.001) and could be one area to focus improvements. The limitations of providers’ ability to diagnose infection (34) could be an area to improve as infection was defined in this study based upon providers’ notes. Our results highlight the need to improve management of sepsis in our context with limited resources (35).

Because of the already limited resources in hospitals and intensive care settings in LMICs, a prognostic model for mortality, specifically applicable to these settings, is an especially useful tool. Although predictive models are recommended to undergo regional customizations (11), the models often do not apply to resource-constrained settings. Further, the models that have been developed for LMICs have rarely been assessed or validated in other similar settings. As predictive models are developed (36), these should also be examined and validated in resource-constrained settings. The ability to benchmark critical care requires intensivists to use common language and scores. Similar to other studies from resource-limited settings (7, 16), our study is limited by the number of patients and the information available. Future studies would benefit from collaboration among multiple centers to improve generalizability. As scores from the region (R-MPM and the Vitals [Uganda and Tanzania] scores) performed moderately well and were more likely to have complete data than scores designed for high-income countries, future quality improvement studies may benefit from adaptation of scores appropriate for resource-constrained settings. Overall, the R-MPM and Vitals (Tanzania) scores had favorable calibration, discrimination, and few data points missing. Given the ease of use of the Vitals (Tanzania) score, implementation in our setting should be feasible.

CONCLUSIONS

Although survival for patients undergoing mechanical ventilation in critical care units in rural Kenya was lower than that reported in high-resource settings, it was comparable to other similar, resource-limited settings. Further, the outcome was often predictable given the available scores. Data were missing for multiple scores, demonstrating their ineffectiveness in our resource-constrained setting. Scores adapted for similar settings had similar or better predictive value than those developed in high-resource settings, but with better calibration.