Monitoring Anesthesia Care Delivery and Perioperative Mortality in Kenya Utilizing a Provider-driven Novel Data Collection Tool

Abstract

Background

Perioperative mortality rate is regarded as a credible quality and safety indicator of perioperative care, but its documentation in low and middle income countries is poor. We developed and tested an electronic, provider report driven method in an East African country.

Methods

We deployed a data collection tool in a Kenyan tertiary referral hospital that collects case-specific perioperative data, with asynchronous automatic transmission to central servers. Cases not captured by the tool (‘non observed’) were collected manually for the last two quarters. We created logistic regression models to analyze the impact of procedure type on mortality.

Results

Between January 2014 and September 2015, 8,419 cases out of 11,875 were captured. Quarterly data capture rates ranged from 423 (26%) to 1,663 (93%) in the last quarter. There were 93 (1.53%) deaths reported at 7 days. Compared to 4 (0.53%) deaths in Cesarean section, general surgery (42; 3.65%, odds ratio [OR] 15.8, 95%CI: 5.20–48.1; p<0.001), neurosurgery (19; 2.41%, OR 14.08, 95%CI: 4.12–48.1; p<0.001), and emergency surgery (25; 3.63%, OR 4.40, 95%CI: 2.46–7.86; p<0.001) carried higher risk of mortality. The ‘non observed’ group did not differ from electronically captured cases in 7-day mortality (1; 0.23% versus 16; 0.58%, OR 3.95, 95%CI: 0.41–38.2; p=0.24).

Conclusions

We created a simple solution for high-volume, prospective electronic collection of perioperative data in a lower middle income setting. We successfully used the tool to collect a large repository of cases from a single center in Kenya, and observed mortality rate differences between surgery types.

Introduction

Non-communicable diseases are an increasing disease burden in many low and middle income countries.^1–3 Providing basic surgical and anesthesia care in such countries could avert as many as 1·4 million deaths yearly.¹ It is important to assess current outcomes in order to identify and remedy unsafe practices or poor quality care along the way to improving surgery and anesthesia in low resource countries. Perioperative mortality rate (defined as percentage of patients dying during or shortly after surgery) is a credible quality and safety indicator of perioperative care.^3,4 Perioperative mortality rate (henceforce referred to as ‘postoperative mortality’) data are crucial to direct capacity-building efforts for safe surgical care in low and middle income countries.

Postoperative mortality is a principal indicator of the safety of surgery and anesthesia care in a country or health region.^3–5 Risk adjustment for factors such as the American Society of Anesthesiologists physical status (ASA status), age, admission urgency, and type of procedure can refine interpretation of postoperative mortality outcomes.⁵ Postoperative mortality is well-documented in high income countries but rarely used as outcome metrics in low and middle income countries.^3–5 Postoperative mortality in high income settings is 9-fold lower than in low and middle income countries (0.38 vs. 3.44 per 100 admissions, respectively),⁵ and postoperative mortality has declined more in high income countries than in low and middle income ones over the past 3 to 4 decades.⁶ A systematic review of postoperative mortality reporting in low and middle income countries from 1995–1999 revealed a median in-hospital mortality rate of 1.2% for elective procedures and 10.1% for emergency procedures.⁷ Studies of postoperative mortality have been limited by several factors, including: a) small sample size, b) hospital-level surveys instead of patient level data, c) focus on a specific surgery, such as Cesarean section or cleft palate repair, and d) data collected retrospectively from surgical case logbooks, which are often incomplete.^8–14

We designed and implemented a simple electronic data collection solution to gather information on anesthesia, surgery, and perioperative outcomes in a prospective, near real-time manner. We present the methodology for creating and implementing this postoperative mortality tool in Kenya and demonstrate 21 months of case-by-case postoperative mortality data from a rural tertiary referral hospital.

Materials and Methods

Study Design

The Vanderbilt University Medical Center (Nashville, Tennessee, USA) and the Kijabe Hospital Institutional Review Boards (Kijabe, Kenya) approved the study without requiring written informed consent. AIC Kijabe Hospital is a 285-bed, rural teaching hospital that serves a catchment area of 45 million persons.¹⁵ It was chosen to be the first center for implementation of the data collection tool because of its clinical and educational capacity in surgery and anesthesia. Results of the prototype are intended to support an ongoing project to expand anesthesia and surgery capacity in government and non-government hospitals throughout Kenya.

Questionnaire

A case report form was assembled using two guiding principles. First, we included all data fields believed to be important for understanding baseline characteristics of surgical care and outcomes in low and middle income countries. Second, we limited the number of data fields to make the case report form reasonable to complete by the anesthetist during a typical surgical case. These somewhat competing principles were iteratively applied employing a modified Delphi technique with Vanderbilt University (BS, MDM, JPW, WSS) and Kijabe Hospital (MWN, JK, MM) experts. The result contains five sections of questions, yielding a final case report form with 132 data fields (Appendix 1 and 2).

Data Collection Tool

Kenyan anesthetists performed initial data collection using paper forms while we developed the electronic data collection system. We created the electronic system on the Research Electronic Data Capture (REDCap) platform, a free, customizable web based data collection instrument that typically requires Internet connectivity for functionality.¹⁶ Internet connectivity can be unreliable in rural Kenya, so we created a version of the REDCap tool supporting offline data capture with asynchronous data upload when an Internet connection was available (Appendix 3). During the implementation phase, Kenyan registered nurse anesthetists and student nurse anesthetists received training on principles of quality improvement, real-time data acquisition, data management, and professionalism. We collected data from January 2014 to September 2015, organized into seven 3-month quarters.

Mortality

Data collectors followed patients prospectively from preoperative assessment until 7 days into the postoperative period (Appendix 2e). When patients were discharged prior to the 7th day, a follow up phone call was made to the patient or designated family member. In-patient mortality data were independently verified using existing in-hospital mortality logs. Cases that were missed, meaning not collected using the REDCap tool, were identified using the surgical logbook (henceforth referred to as ‘non observed’). Key outcome data for ‘non observed’ cases were manually collected retrospectively for the last 6 months of the data collection period. This manual collection of data did not employ the REDCap tool.

Statistical Analysis

We described baseline characteristics and demographics using counts and percentages for all categorical data. Ear, nose and throat, endoscopy, gynecology, ophthalmology, oral/maxillofacial, and urology surgeries were grouped together and referred to as ‘other procedures’ under the classification of procedure type. Demographic and clinical characteristics were compared between ‘observed’ and ‘non observed’ cases in the last two quarters using the Fisher exact test or Kruskal-Wallis test for categorical and quantitative factors, respectively.

Two types of missing data were identified: the ‘non observed’ and, for observed cases, missing mortality status. Both types of missing data can introduce bias in the estimate of postoperative mortality and its association with procedure type and other factors. Missing mortality records and missing records for key covariates were treated using multiple imputation with the chained equations approach.¹⁷ Specifically, this method was used to account for uncertainty due to missing values for emergency and trauma status, time of procedure (day/night/weekend), and mortality status at 24 hours, 48 hours, and 7 days. Records with missing values for other covariates were excluded from analysis. We multiply imputed each of these variables using the remaining variables, including procedure type. One hundred chains were initialized and evaluated for 30 iterations. All subsequent statistical analyses were evaluated using each of the 100 completed data sets. The results of these analyses were combined using Rubin’s rules.¹⁸

Inverse probability weighting was used to compute marginal estimates of 24-hour, 48-hour, and 7-day postoperative mortality, accounting for the possibility of bias due to missing cases that were not captured in REDCap. Specifically, in the completed Q2–Q3 2015 case data, the probability of failed capture by the REDCap system was estimated using logistic regression, adjusting for procedure type, emergency, trauma, and pediatric status, and time of procedure. The fitted model was then used to estimate the probability of failed capture for all observed procedure records. The reciprocals of these probabilities were used to weight estimate of 24-hour, 48-hour, and 7-day postoperative mortality. All mortalities are reported cumulatively, meaning deaths in 24-hours were counted in 48-hour and 7-day mortality, and deaths in 48 hours were counted in 7-day mortality.

The adjusted associations between mortality (24-hour, 48-hour, and 7-day) and procedure type, emergency, trauma, and pediatric status, time of procedure, and date (days from the start of data collection) were assessed using unweighted logistic regression analysis. The effect of procedure date was modelled using a restricted cubic spline with three degrees of freedom and knots spaced evenly along the procedure date quantiles. Probabilities and odds ratios were summarized using Wald-type 95% confidence intervals. All statistical analyses were implemented using the open-source software package “R” (http://www.r-project.org) with the add-on package “mice”, for multiple imputation.^17,19

Results

From January 2014 to September 2015 data from 8,419 cases were captured with the REDCap tool, representing 71% of the 11,875 cases recorded on the surgical book at Kijabe Hospital during that period. We excluded 553 (6.56%) cases with missing demographics and procedure type data, and entered 7,866 cases into the postoperative mortality analysis (Figure 1). Table 1 gives details of patient, provider, and case demographics. Most patients, 7,444 (94.6%), were healthy, as defined by ASA status equal to 1 or 2. Most cases, 7,635 (97.1%), were performed by Kenyan registered nurse anesthetists as is customary at Kijabe Hospital. Because Kijabe Hospital is a training center, 6,679 (84.9%) of the cases included nurse anesthetist students under the direct supervision of registered nurse anesthetists and anesthesiologists in a task sharing, anesthesia team model.

Figure 1.

Figure 1 lists number of cases with missing data. 7,866 cases were used for evaluating patient demographics, while mortality analyses for 24 hours, 48 hours, and 7 days were performed on 7,820, 7,686, and 6,164 cases, respectively.

Abbreviations: n – number of cases, SAS – surgical apgar score.

Table 1.

Patient demographics

Category	Number (percentage)
Total Number of Patients	7,866
In-room Providers
KRNA	7,635 (97.1%)
Anesthesia student	6,679 (84.9%)
Clinical officer	140 (1.8%)
ASA Classification
ASA 1	3,370 (42.8%)
ASA 2	4,074(51.8%)
ASA 3	388 (4.9%)
ASA 4	25 (0.32%)
ASA 5	9 (0.11%)
Male	3,948 (50.2%)
Age*
< 1 months	355 (4.5%)
< 3 months	555 (7.1%)
< 3 years	1,250 (15.9%)
< 12 years	2,075 (26.4%)
< 18 years	2,442 (31.0%)
18–65 years	4,621(58.7%)
> 65 years	803 (10.2%)
Anesthesia Type
General	4,491 (57.1%)
Regional / spinal	2,895 (36.8%)
MAC	480 (6.1%)
Monitors Used
Pulse oximetry	7,813 (99.3%)
Blood pressure	7,737 (98.4%)
3-lead ECG	7,517 (95.6%)
End tidal CO2	4,829 (61.4%)
Temperature probe	1,319 (16.8%)
Peripheral nerve stimulator	147 (1.9%)
Safe Surgery Checklist	7,822 (99.4%)

^*For patients <18 years old, the age categories reported contain all patients less than the age listed in each line; thus, the summated percentages will be >100%. Categories for ages <18 years, 18–65 years, and >65 years equal 100%.

Abbreviations: ASA classification – American Society of Anesthesiologists physical status classification, ECG – electrocardiogram, KRNA – Kenyan registered nurse anesthetist, MAC – monitored anesthesia care.

Pulse oximetry was used in 7,813 (99.3%) of cases, blood pressure measurement at least every 5 minutes in 7,737 (98.4%), and a 3-lead electrocardiogram in 7,517 (95.6%). Only 4,829 (61.4%) cases utilized end tidal CO₂ monitoring, and only 1,319 (16.8%) reported monitoring of patient temperature (Table 1). In 7,822 (99.4%) of the cases, a Safe Surgery Checklist was performed, designed to reduce postoperative mortality.²⁰

Overall, 1,702 (21.6%) cases entered into REDCap were missing 7-day mortality data (Figure 1). After a research assistant was hired at the end of the first quarter of 2015 to focus solely on collection of 7-day mortality data, missing data for this statistical element dropped to 5 (0.32%) in the final quarter (Figure 2).

Figure 2.

Non observed and 7-day mortality missing data. Figure 2 shows percentage alive, dead, and missing data for 7-day mortality per quarter. Black dotted line between quarters 1 and 2 in 2014 represents transition from paper form of REDCap to electronic, while red dotted line between quarters 1 and 2 in 2015 indicates the period when a research assistant dedicated to the collection of 7-day mortality data was hired.

Abbreviations: n – number of patients, Q – quarter, REDCap – Research Electronic Data Capture.

Concerning postoperative mortality, there were 50 (0.64%) mortalities reported at 24 hours, 66 (0.87%) at 48 hours, and 93 (1.53%) at 7 days after surgery (Table 2). Patients who underwent Cesarean section had the lowest 24-hour (2; 0.23%), 48-hour (3; 0.34%), and 7-day (4; 0.53%) mortality, while patients who underwent general surgery ([20; 1.23%], [29; 1.83%], and [42; 3.65%]) and emergency surgery ([13; 1.53%], [17; 2.04%], and [25; 3.63%]) showed significantly higher 24-hour, 48-hour, and 7-day mortality, respectively (Table 2). Logistic regression analysis for 24-hour mortality showed that as compared to Cesarean section, general surgery (OR 10.68, 95%CI: 2.32–49.13; p=0.002), neurosurgery (OR 8.45, 95% CI: 1.56–45.8; p=0.01), orthopedic surgery (OR 6.81, 95%CI: 1.31–35.4; p=0.02), and emergency surgery (OR 4.30, 95%CI: 1.97–9.41; p=0.0003) were associated with higher risk of mortality (Table 3). Trauma, pediatrics, and ‘other surgeries’ were not associated with increased risk of 24-hour mortality. In addition, the number of days following the start of data collection was also significantly associated with 24-hour mortality. Specifically, the odds ratio associated with day 450 vs. day 0 was 0.55 (95% CI: 0.25–1.23; p=0.15). The corresponding odds ratio for day 600 vs. day 450 was 0.11 (95% CI: 0.02–0.71; p=0.02) (Figure 3). These associations with mortality were also seen in 48-hour and 7-day mortality (Table 3).

Table 2.

Absolute numbers for 24-hour, 48-hour, and 7-day mortality by procedure type and other categories

	24 hour			48 hour			7 day
Surgery Type	Mortality (%)*	Alive	Missing	Mortality (%)	Alive	Missing	Mortality (%)	Alive	Missing
All surgeries	50 (0.64)	7,770	46	66 (0.87)	7,620	180	93 (1.53)	6,071	1,702
By Procedure Type
C-sections	2 (0.23)	876	5	3 (0.34)	870	10	4 (0.53)	758	121
General surgery	20 (1.23)	1,622	4	29 (1.83)	1,586	31	42 (3.65)	1,151	453
Neurosurgery	9 (0.88)	1,025	6	11 (1.08)	1,014	15	19 (2.41)	789	232
Orthopedics	11 (0.73)	1,499	14	12 (0.83)	1,447	65	14 (1.20)	1,162	348
Other surgeries+	8 (0.29)	2,748	17	11 (0.41)	2,703	59	14 (0.63)	2,211	548
By Other Categories
Emergency surgery	13 (1.53)	847	5	17 (2.04)	832	16	25 (3.63)	689	151
Pediatrics (age <18 y)	18 (0.75)	2,408	16	24 (1.01)	2,375	42	31 (1.66)	1,863	547
Trauma	6 (0.84)	716	7	8 (1.15)	693	28	9 (1.66)	543	178

^*Denominator used for calculation of percent mortality is the number of patients alive, without taking missing data into account.

⁺Other surgeries include: ear nose and throat, endoscopy, gynecology, ophthalmology, oral/maxillofacial, and urology surgeries.

Abbreviations: C-sections – Cesarean sections, y – years.

Table 3.

Logistic regression analysis: Odds ratio of mortality at 24 hours, 48 hours, and 7 days as compared to C-section for entire study period

	24 hours			48 hours			7 days
Variable	Odds Ratio	95%CI	P-value	Odds Ratio	95%CI	P-value	Odds Ratio	95%CI	P-value
C-section	REF	REF	REF	REF	REF	REF	REF	REF	REF
Procedure Type
General surgery	10.68	[2.32, 49.13]	0.002	10.09	[2.86, 35.6]	0.0003	15.8	[5.20, 48.1]	<0.0001
Orthopedics	6.81	[1.31, 35.4]	0.02	4.45	[1.09, 18.1]	0.04	5.79	[1.69, 19.9]	0.005
Neurosurgery	8.45	[1.56, 45.8]	0.01	6.45	[1.56, 26.6]	0.01	14.08	[4.12, 48.1]	<0.0001
Other surgeries	3.09	[0.60, 15.9]	0.18	2.67	[0.69, 10.4]	0.16	3.53	[1.07, 11.7]	0.04
Emergency	4.30	[1.97, 9.41]	0.0003	3.95	[1.97, 7.91]	0.0001	4.40	[2.46, 7.86]	<0.0001
Pediatrics	1.02	[0.52, 2.03]	0.95	1.05	[0.59, 1.89]	0.86	0.76	[0.45, 1.28]	0.30
Trauma	0.90	[0.32, 2.53]	0.84	1.11	[0.44, 2.77]	0.83	0.79	[0.35, 1.77]	0.57
Date of Surgery
Day 450 vs Day 0	0.55	[0.25, 1.23]	0.15	0.43	[0.24, 0.76]	0.003	0.58	[0.38, 0.88]	0.01
Day 600 vs Day 450	0.11	[0.02, 0.71]	0.02	0.09	[0.03, 0.33]	<0.001	0.13	[0.05, 0.35]	<0.001
Time of Surgery (N/W)*	1.02	[0.38, 2.76]	0.97	1.13	[0.47, 2.74]	0.86	1.18	[0.57,2.43]	0.66

^*N/W – night or weekend case.

C-section was used as the reference variable, as it is the most commonly performed surgical procedure in low and middle income countries. P-values <0.05 are indicated in bold. Abbreviations: 95%CI – 95% confidence interval, C-section – Cesarean section, REF – reference value.

Figure 3.

The effect of date of surgery on the odds of mortality. This figure illustrates the effect of date on the log odds of postoperative mortality rate, relative to the earliest time period point (January 2014). There does not appear to be much effect until roughly March of 2015 (with exception of 48 hours), where there is a sharp decline in postoperative mortality rate, adjusting for the effects of all other factors in the multiple regression. The postoperative mortality odds ratio associated with day 450 versus day zero (January 1, 2014) is 0.55 (95% CI [0.25, 1.23], p-value: 0.15, 0.43 (95% CI [0.24, 0.76], p-value: 0.003), and 0.58 (95% CI [0.38, 0.88], p-value: 0.01), for 24-hour, 48-hour, and 7-day mortality, respectively. The corresponding odds ratio for day 600 versus day 450 is 0.11 (95% CI [0.02, 0.71], p-value: 0.02), 0.09 (95% CI [0.03, 0.33], p-value: <0.001), and 0.13 (95% CI [0.05, 0.35], p-value: <0.001), for 24-hour, 48-hour, and 7-day mortality, respectively.

During the study period, 3,456 cases from the surgical log book were not captured by the REDCap tool (‘non observed’). The first three months of data collected via paper form showed the lowest capture rate, at 26%, while after implementation of electronic tool the average data capture rate was at 77%, increasing to 93% of all cases in the final quarter (Figure 4). Comparison of observed and ‘non observed’ data for the last two quarters revealed there were more night and weekend cases (28.9% and 40.3% vs 3.5% and 1.1%, p<0.001), emergency cases (52.2% vs 8.6%, p<0.001), and trauma cases (15.6% vs 11.1%, p=0.008) in the ‘non observed’ group than in the ‘observed’ group. Procedure type distribution also differed between groups, with fewer orthopedic and pediatric cases in the ‘non observed’ group (Table 4).

Figure 4.

Figure 4 shows the number of cases collected with REDCap tool compared to surgical logbook (‘non observed’). Black dotted line between quarters 1 and 2 in 2014 represents transition from paper form of REDCap to electronic, while red dotted line between quarters 1 and 2 in 2015 indicates the period when a research assistant dedicated to the collection of 7-day mortality data was hired.

Abbreviations: Q – quarter, REDCap – Research Electronic Data Capture.

Table 4.

Data summary for observed versus non-observed quarters 2 and 3 of 2015

Variable	Observed (N=2,740)	Non Observed (n=429)	P-value
Procedure Type			<0.001
C-section	259 (9.5%)	154 (35.9%)
General surgery	428 (15.6%)	50 (11.7%)
Neurosurgery	297 (10.8%)	19 (4.4%)
Orthopedics	548 (20.0%)	60 (14.0%)
Other Surgery	1,208 (44.1%)	146 (34.0%)
Pediatrics	807 (29.5%)	47 (11.0%)	<0.001
Emergency	237 (8.6%)	224 (52.2%)	<0.001
Trauma	303 (11.1%)	67 (15.6%)	0.008
Surgery Time			<0.001
Day	2,495 (91.1%)	132 (30.8%)
Night	96 (3.5%)	124 (28.9%)
Weekend	30 (1.1%)	173 (40.3%)
24-hour mortality	9 (0.33%)	0 (0%)	0.24
48-hour mortality	12 (0.44%)	1 (0.23%)	0.54
7-day mortality	16 (0.58%)	1 (0.23%)	0.35

Data presented as absolute number with percent (%) of total cases in parentheses.

Regression analysis of the ‘non observed’ group revealed general surgery, neurosurgery and emergency surgery associated with higher 48-hour and 7-day mortality as compared to Cesarean section, similar to the finding with the observed group (Table 5). Adjusting for procedure, emergency status, trauma status, and time of surgery (day, night, weekend), there was no evidence that being in the ‘non observed’ group was associated with 48-hour (OR 3.52, 95%CI: 0.34–36.7; p=0.29) or 7-day (OR 3.95, 95%CI: 0.41–38.2; p=0.24) mortality. Because there were no deaths in 24-hour mortality in the ‘non observed’ group for these quarters, we were unable to perform regression analysis for 24-hour mortality.

Table 5.

Odds ratio of 48-hour and 7-day mortality for observed and non observed cases for quarters 2 and 3 in 2015

	48 hour			7 day
Variable	Odds Ratio	95%CI	P-value	Odds Ratio	95%CI	P-value
C-section	REF	REF	REF	REF	REF	REF
Procedure Type
General surgery	25.5	[3.07, 211.2]	0.003	33.7	[4.13, 274.4]	0.001
Orthopedics	3.58	[0.27, 46.7]	0.33	8.27	[0.74, 92.7]	0.09
Neurosurgery	15.4	[1.36, 173]	0.03	34.8	[3.42, 353.6]	0.002
Other
Emergency	4.99	[1.08, 23.02]	0.04	6.45	[1.72, 24.2]	0.006
Pediatrics	1.97	[0.54, 7.14]	0.30	1.19	[0.36, 3.98]	0.78
Trauma	3.83	[0.79, 18.64]	0.10	2.17	[0.51, 9.23]	0.29
Time of Surgery (N/W)*	1.46	[0.23, 9.29]	0.69	1.07	[0.20, 5.82]	0.94
Observed	3.52	[0.34, 36.7]	0.29	3.95	[0.41, 38.2]	0.24

To assess if being ‘non observed’ affected mortality, we performed a regression analysis only on Q2 and Q3 data in 2015, the same time period manual data were collected for the ‘non observed’ group. C-section was used as the reference variable, as it is the most commonly performed surgical procedure in low and middle income countries. Because there was no mortality reported at 24 hours in the ‘non observed’ group, we were unable to perform this analysis for 24-hour mortality. P-values <0.05 are indicated in bold.

^*N/W – night or weekend case.

Abbreviations: 95%CI – 95% confidence interval, C-section – Cesarean section, REF – reference value.

We used the propensity score model associated with the 2015 quarter 2 and 3 data to compute the likelihood of having been observed for each case in the whole study cohort and used inverse probability weighting to estimate 24-hour, 48-hour, and 7-day mortality. This revealed an adjusted 24-hour mortality of 0.71 (95% CI [0.54–0.92]), 48-hour mortality of 0.99 (95% CI [0.78–1.24]), and 7-day mortality of 1.86 (95%CI [1.52–2.27]), as opposed to an unadjusted mortality of 1.58.

Discussion

Kenyan anesthesia providers achieved near real-time, point of care collection of key perioperative anesthesia and surgical data using the electronic data collection tool we developed. Thus, we have enabled near real-time electronic outcomes capture for large surgical populations in a middle income country. Postoperative mortality at a tertiary referral center in Kenya is lower than previous reports from similar centers. Consistent with previous literature, emergency surgery is associated with higher postoperative mortality than elective cases. Lastly, we documented higher postoperative mortality rates in orthopedics, general surgery, and neurosurgical cases, as compared to Cesarean section, the most common procedure in low- and middle-income countries.

Prospective Large Scale Data Collection Possible in Low and Middle Income Countries

Our postoperative mortality data were collected prospectively, in contrast to other reports from hospitals in Africa.^21,22 We collected perioperative data from 8,419 out of 11,875 cases, providing richer data to interpret observed surgical outcomes. Through continued education and resource allocation, we were able to reduce ‘non observed’ cases to 7% of total in the last quarter of data collection. We manually collected postoperative mortality data on a subgroup of the ‘non observed’ population and were then able to demonstrate that postoperative mortality outcomes of cases captured in the REDCap tool were representative of the whole surgical population. While there is a possibility that there are some cases that were performed but not entered in the surgical logbook, based on our clinical experience at Kijabe Hospital, this happens in very rare circumstances. At this hospital there are personnel dedicated to entering such data daily in the surgical logbook, even on weekends, and the likelihood of selection bias affecting our outcomes analysis is extremely low.

Postoperative Mortality at a Tertiary Referral Center in East Africa

Our overall mortality rate is lower than anticipated. Postoperative mortality in low and middle income countries ranges from 3.44% to 6%.^5,7,21,22 Consistent use of ASA standard monitors, which have been shown to improve perioperative outcomes,^23,24 might reduce mortality in our population. Monitors are less available in most other low income institutions. For example, pulse oximetry use rates of 12% (Tanzania) to 46% (Uganda) are typical.²⁵ In addition Kijabe Hospital is an international training center for both anesthesiology and surgery, attracting skilled practitioners and creating an environment of excellence, which probably influences postoperative mortality. Lastly the use of the Safe Surgery Checklist in 99.4% of operative cases could have contributed to a lower mortality rate.²⁰ It is important to note that the quality of the use of the checklist was not assessed in depth in this study, therefore additional analysis is needed to further investigate this association.²⁶

We observed declining mortality late in the study period (Figure 3). While the exact reason is not known, we speculate the following. First, the Kenyan Registered Nurse Anesthetist workflow was changed in early January 2015 to relieve the anesthetist from clinical care after taking a full night of call. Fewer fatigued providers on the day shift may have lowered mortality.^27,28 Second, during this study we placed significant educational emphasis on the need to monitor patient outcomes and modify clinical practice to improve outcomes. It is possible that emphasis on monitoring and improving outcome metrics engendered increased vigilance and more diligent care during each case.²⁹ Third, it is possible that case mix shifted to healthier patients or lower acuity without being detected in the REDCap tool, independently lowering mortality risk. A future study is needed to prospectively test the possibility that the introduction of an initiative of this type might improve outcomes.

We observed increased mortality in patients undergoing emergency surgery, consistent with previous reports,^4,5,22,30 and postoperative mortality rate was also higher in the general surgery, orthopedics, neurosurgery patient mix compared with Cesarean section patients. Regional referral hospitals naturally attract non-obstetrical patients who have a more guarded prognosis. In addition, delayed presentation for oncologic surgery or fixation of fractures is common in low and middle income countries.^31,32 In contrast, the obstetrical patient is younger, has fewer co-morbidities, and primarily arrives from a short distance, potentially explaining mortality differences. In the future, it will be important to evaluate mortality rates within these surgical specialties among national referral hospitals to see if the case mix and outcomes are similar.

A standardized set of validated metrics for postoperative mortality rate in low and middle income countries does not exist.⁴ Weiser et al recommended using “day of surgery death ratio” and “in-hospital death ratio” as two metrics that could be adopted globally.³³ We chose to report 24-hour, 48-hour, and 7-day mortality. We believe that the 24-hour death ratio functionally represents day-of surgery ratio for most patients.⁴ Additionally, we believe that 24-hour mortality may have an advantage in that it takes into account emergency cases that are completed in the evening. For instance, a case that ends at 10PM would only have a 2 hour period to consider “day of surgery” mortality, as opposed to a full 24 hours postoperatively in our metric. We elected to collect 7-day mortality instead of “in-hospital death ratio” because we found it challenging to identify date of hospital discharge reliably within our data collection system and current personnel, in a setting where there is no electronic medical record and poor record keeping in general. Instead, having a specific date of follow up after surgery ensured consistency of reporting. We believe that 7-day mortality will capture majority of in-hospital mortality, as it is our experience that almost all patients in our setting are discharged by this time point. However, we recognize that future research should address which set of metrics is most valid and reliable in low and middle income countries.

Strengths and Limitations

The mechanism by which postoperative mortality was determined in this large cohort was the result of a simple information technology solution coupled with a robust and environment-appropriate data collection education program for the anesthesia providers. Our data collection tool allows for electronic data collection without the need for continuous Internet connectivity. Transmission to a central server makes data analysis and reporting more robust, facilitating data aggregation as other data collection sites come online throughout Kenya. Considering that most postoperative mortality data are derived from review of logbooks or paper data collection, a prospective, electronic solution could be a major step forward.

It would have been ideal to collect data from 100% of cases, but we encountered barriers. Electronic data entry in the perioperative period changes the anesthesia providers’ workflow, particularly in low and middle income countries, where paper anesthesia charts are the primary record. This additional workload is eased by the presence of students in the theater, but students are not always available. We plan to modify the REDCap tool to produce a simple intraoperative record, along with its other features, minimizing duplication of effort. In addition, data collection and intraoperative charting in general can have the potential to detract from vigilance. This is not unique to our data collection system or the low resource setting, but something that comes with anesthesia practice and knowing what to prioritize. Part of the training we provide to the data collectors includes emphasizing the precedence clinical care takes over documentation.

There are several limitations to our study. Kijabe Hospital is a tertiary referral hospital with surgery specialization in pediatrics and neurosurgery and extensive, highly supervised anesthesia and surgery training programs. As such, generalizations of these observed outcomes to other health facilities and hospitals should not be made without accounting for practice setting differences. Second, given an average capture rate of approximately 75% of all cases, it is possible that our dataset varies from the actual population of cases performed. While we believe we have adequately addressed this issue by collecting a sample of ‘non observed’ data and using inverse propensity weighting to estimate postoperative mortality rate and to account for missing data, it is always possible that the true mortality is different from what we observed. Third, as in all non-automated environments, data entry and accuracy are dependent on the skill and diligence of the data collectors. While in-hospital mortality data are verified by an independent person using hospital mortality logs and phone calls to patients and patient families, data quality is still susceptible to human error and bias. As the tool is further modified, data field restrictions are being implemented to improve data quality. Fourth, in October 2014, Kenya’s economic status was changed from low income to lower middle income according to the World Bank, which could impact the application of this study to a country that is a low income country. Lastly, many patients were discharged before postoperative day 7, reducing our 7-day mortality follow up. We hired a research nurse to collect 7-day follow up data, reducing missing data in REDCap observed cases to 0.32% (Figure 2). This raised costs, but we demonstrated feasibility with only one additional staff member at a high-volume referral center. This need for additional personnel will have cost implications in smaller and under-resourced hospitals. We have found that with adequate training, a health care worker with diploma level training is able to accomplish the task. In addition one worker can cover multiple hospitals, depending on case load, which might alleviate some of the cost burdens.

Future Directions and Conclusions

We plan to expand the scope of this project to include a wide variety of hospitals throughout Kenya, including many government facilities. Additionally, we are keen to engage partners in low and middle income countries who wish to use the tool that we described, both to enter data and to improve its content based on local experience. By expanding this data collection effort nationally and in other low income countries, we seek to provide perioperative outcome metrics in diverse hospital settings as baselines against which to evaluate capacity-building efforts. These data are most useful when shared transparently with healthcare system leadership in the government, non-governmental, and faith-based organization sectors, accompanying an assessment of barriers to providing safe surgery and anesthesia for their representative patient populations. Collection and reporting of perioperative outcome metrics, such as postoperative mortality rate, are vital first steps in the quality improvement cycle so urgently needed for emerging health systems to deliver safe surgery and anesthesia. As a goal in line with the vision of Global Surgery 2030,³ creating a multinational dataset surrounding perioperative outcomes, particularly for bellwether procedures, could help guide the development and deployment of resources in these settings.

We created a simple solution for high-volume, prospective electronic collection of perioperative medical information in an asynchronous, context-sensitive manner. We successfully used the tool to collect a large repository of cases from a single center in East Africa, demonstrating that 24-hour, 48-hour, and 7-day mortality data can be measured along with perioperative process data.

Appendix 1: PDF of 132-item case report form

Abbreviations: a – am, AIC – African Inland Church, bpm – beats per minute, CC – milliliter, CO – clinical officer, C-Section – Cesarean section, CV – cardiovascular, DA – difficult airway, DD – date, Dias – diastole, ENT – ear, nose, or throat, Epid – epidural, ETT – endotracheal tube, G – gauge (e.g., 25G – 25 gauge needle), GA – general anesthesia, GSW – gunshot wound, HDU – high demand unit, HH:MM – hours and minutes, HR – heart rate, ICU – intensive care unit, IV – intravenous, kg – kilogram, LMA – laryngeal mask airway, MAC – monitored anesthesia care, MD – medical doctor, M-F – Monday to Friday, Misc – miscellaneous, MM – month, MVA – motor vehicle accident, OR – operating room, p – pm, PACU – post anesthesia care unit, POD – postoperative day, REDCap – Research Electronic Data Capture, Resp – respiratory, RN – registered nurse, RSI – rapid sequence intubation, Sat – Saturday, Sun – Sunday, Sys – systole, U/S – ultrasound, v. – versus, YOB – year of birth, YYYY – year.

Appendix 2

Appendix 2a–e. REDCap data collection tool.

Abbreviations: a – am, AIC – African Inland Church, bpm – beats per minute, CC – milliliter, CO – clinical officer, C-Section – Cesarean section, CV – cardiovascular, DA – difficult airway, DD – date, Dias – diastole, ENT – ear, nose, or throat, Epid – epidural, ETT – endotracheal tube, G – gauge (e.g., 25G – 25 gauge needle), GA – general anesthesia, GSW – gunshot wound, HDU – high demand unit, HH:MM – hours and minutes, HR – heart rate, ICU – intensive care unit, IV – intravenous, kg – kilogram, LMA – laryngeal mask airway, MAC – monitored anesthesia care, MD – medical doctor, M-F – Monday to Friday, Misc – miscellaneous, MM – month, MVA – motor vehicle accident, OR – operating room, p – pm, PACU – post anesthesia care unit, POD – postoperative day, REDCap – Research Electronic Data Capture, Resp – respiratory, RN – registered nurse, RSI – rapid sequence intubation, Sat – Saturday, Sun – Sunday, Sys – systole, U/S – ultrasound, v. – versus, YOB – year of birth, YYYY – year.

Appendix 3. Close-up of REDCap Data Collection tool

Appendix 3a–c. Close-up of REDCap data collection tool.

Appendix 3a. REDCap tool allows the data collector to enter records offline. Record ID number is displayed in the left column, and the user can see which of the five data collection instruments he/she is entering (highlighted in blue with red outlined box). In the right column, mandatory data points are highlighted with red text (*must provide value). Data entry points include free text, radio buttons, check boxes, and calculated data fields. Free text data entry fields are limited in number to provide the most usable structure to the overall dataset.

Appendix 3b. Under the Record Status Dashboard, a data collector is able to see, by identification number, all the records entered. The record identification (Record ID) number includes the date and time of surgery for ease of identifying a particular record (red box). Whether all data fields within a form are complete is indicated by the circles in the table, with the key being shown in the upper right of the page (blue box).

Appendix 3c. The REDCap software was modified so that in the absence of Internet connection (which is a common occurrence in low- and middle-income countries), the laptop will serve as the server and store patient data. Whenever an Internet connection is available, the user can press the upload data button indicated in the red box in the left column, at which point data are uploaded to the main server. If successful, the confirmation message “Data is uploaded successfully” is displayed.

Abbreviations: FAQ – frequently asked questions, ID – identification, MB – megabyte, REDCap – Research Electronic Data Capture, YOB – year of birth.