The Surgical Apgar Score

Objectives:

To review the current literature evaluating the performance of the Surgical Apgar Score (SAS).

Background:

The SAS is a simple metric calculated at the end of surgery that provides clinicians with information about a patient’s postoperative risk of morbidity and mortality. The SAS differs from other prognostic models in that it is calculated from intraoperative rather than preoperative parameters. The SAS was originally derived and validated in a general and vascular surgery population. Since its inception, it has been evaluated in many other surgical disciplines, large heterogeneous surgical populations, and various countries.

Methods:

A database and gray literature search was performed on March 3, 2020. Identified articles were reviewed for applicability and study quality with prespecified inclusion criteria, exclusion criteria, and quality requirements. Thirty-six observational studies are included for review. Data were systematically extracted and tabulated independently and in duplicate by two investigators with differences resolved by consensus.

Results:

All 36 included studies reported metrics of discrimination. When using the SAS to correctly identify postoperative morbidity, the area under the receiver operating characteristic curve or concordance-statistic ranged from 0.59 in a general orthopedic surgery population to 0.872 in an orthopedic spine surgery population. When using the SAS to identify mortality, the area under the receiver operating characteristic curve or concordance-statistic ranged from 0.63 in a combined surgical population to 0.92 in a general and vascular surgery population.

Conclusions:

The SAS provides a moderate and consistent degree of discrimination for postoperative morbidity and mortality across multiple surgical disciplines.

BACKGROUND

In 2007, Gawande et al¹ proposed an Apgar score for surgical patients. This Surgical Apgar Score (SAS) was to be calculated at the end of surgery providing both a numerical grade of the patient’s condition and prognostic information regarding the patient’s chances of major complications or death. This score was derived from a retrospective analysis of data from the National Surgical Quality Improvement Program database and then validated in 2 prospective cohorts that included 102 colectomy patients and 767 patients undergoing vascular or general surgery procedures. A simple 10-point score based on estimated blood loss (EBL), lowest heart rate, and lowest mean arterial pressure (MAP) was developed. This score was found to be significantly associated with death or major morbidity at 30 days. Those with scores of ≤4 had a 30-day mortality or major morbidity rate of 58.6%, while those with a score of 9 or 10 had a mortality or major morbidity rate of only 3.6%. Following the score’s derivation and validation, the same group, this time led by Regenbogen et al² confirmed that the prognostic value of the score was derived from intraoperative parameters after correcting for preoperative and intraoperative risk. Regenbogen et al^3,4 further validated the score in another group of general and vascular surgery patients and evaluated the ability of the score to predict morbidity following colectomy.

The SAS is calculated from the EBL, lowest MAP, and lowest heart rate. Much like Virginia Apgar’s scoring system to evaluate newborns, the SAS is also a 10-point scale with higher scores indicative of better outcomes.¹ The parameters of the score are provided in Table 1.

TABLE 1.

The 10-Point Surgical Apgar Score

	0 Points	1 Point	2 Points	3 Points	4 Points
Estimated blood loss (mL)	>1000	601–1000	101–600	≤100
Lowest mean arterial pressure (mm Hg)	<40	40–54	55–69	≥70
Lowest heart rate (BPM)	>85*	76–85	66–75	56–65	≤55*

The Surgical Apgar Score is calculated at the end of any general or vascular surgery operation from the estimated blood loss, lowest mean arterial pressure, and lowest heart rate entered in the anesthesia record during the operation. The score is the sum of the points from each category.

*Occurrence of pathologic bradyarrhythmia, including sinus arrest, atrioventricular block or dissociation, junctional or ventricular escape rhythms, and asystole also receive 0 points for lowest heart rate.

BPM indicates beats per minute; mL, milliliters; mm Hg, millimeter of mercury.

Prediction of surgical risk is important for providing patients accurate information and helping to guide clinician decision-making in the perioperative period. Many risk stratification tools have been developed from analysis of surgical databases. Most, however, have relied on a combination of preoperative patient factors and specific surgical considerations. While intraoperative performance is known to impact patient outcomes, this information has been largely left out of current risk stratification tools. The SAS is unique in that it takes into account intraoperative performance to provide an estimation of postoperative morbidity and mortality that clinicians may be able to use to better risk-stratify patients following surgery. Since the inception of the SAS in 2007, it has been evaluated across a number of specific surgical disciplines, large heterogeneous surgical populations, and various countries. This is a review of the current literature that has evaluated the performance of the SAS with the aim of better understanding its discriminatory ability in the perioperative period.

MATERIALS AND METHODS

The Meta-Analysis of Observational Studies in Epidemiology Guidelines for reporting systematic reviews of observational studies were followed when performing this systematic review. Appropriate exclusions were made as data was not meta-analyzed.^5,6 A Preferred Items Checklist was used in the writing of the article and is provided in Appendix 1 (http://links.lww.com/AOSO/A190).⁵ The search strategy and flow chart were adopted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.^7,8

Search Strategy and Study Eligibility

The search strategy was drafted and agreed upon by all authors. A database search was performed on March 3, 2020, and included Ovid MEDLINE, Ovid Healthstar, and Excerpta Medica dataBASE databases. A gray literature search was also performed in accordance with previously published methods.⁹ Results were limited to English language and Journal Articles with no restriction on date of publication. Review articles were excluded as they did not provide new information, and abstract-only publications were excluded as study quality could not be adequately assessed. As this study aimed to evaluate the SAS for performance in discrimination, those that did not report an area under the receiver operating characteristic curve (AUROC) or concordance (c-) statistic were excluded. The search strategy as well as study inclusion and exclusion criteria are described in Appendix 2 (http://links.lww.com/AOSO/A191).

Data Extraction

Two authors performed data extraction blindly and independently. Differences in extracted data were resolved by consensus of all three authors with reference to the article in question. Study details were extracted to a standardized table and included reference details, location, dates of data collection, study design, surgical discipline, patient population, sample size, outcomes measured, objective performance results, the author’s conclusions, sources of funding, and conflicts of interest.

Quality Assessment of Studies

Quality assessment of observational studies was performed in accordance with the Newcastle-Ottawa Scale for Assessing Non-Randomized Studies in Meta-analyses where patient selection, comparability, and outcome are assessed.¹⁰ Patient selection was evaluated in terms of representativeness of the exposed cohort, selection of the nonexposed cohort, ascertainment of exposure, and demonstration that the outcome of interest was not present at study onset. Comparability within the study was evaluated by determining measurement and control of variables between groups. Study outcome was evaluated by considering method of ascertainment of outcome, as well as adequate duration and completeness of follow-up. Two independent investigators assessed study quality blindly during data extraction. Differences were resolved by consensus of all three authors. We determined that 6 or more out of 9 of the aforementioned criteria would constitute sufficient quality for inclusion in this review. The Newcastle-Ottawa Scale is outlined in Appendix 3 (http://links.lww.com/AOSO/A192).

Data Analysis and Statistical Considerations

The performance of the SAS in various populations was evaluated with measures of discrimination. Discrimination, or how well a score correctly identifies an outcome, is reported using the c-statistic or AUROC. Here we will report and refer to the AUROC or c-statistic interchangeably as metrics of discrimination. This metric ranges from 0.5 indicating random concordance to 1.0 indicating perfect concordance.¹¹ Values of less than 0.7 indicate poor performance, while those from 0.7 to 0.9 indicate moderate performance and values greater than 0.9 indicate high performance.¹² If either the AUROC or c-statistic was reported, it was extracted as a measure of discrimination. Descriptions of the relationship between the SAS and patient outcomes or notable observations were extracted for illustration of applicability and future investigations, although they were not included in the analysis. Measures of calibration, or how well the predictive model reflects actual outcome prevalence, are also important in the assessment of a new clinical metric. We chose to limit our review to measures of discrimination as this best illustrates the utility of the score in clinical practice.

RESULTS

Search Results

The formal database search identified 302 articles from primary literature databases. Another 2 articles were identified when searching gray literature databases. After removal of duplicates, 130 articles remained. Titles and abstracts were reviewed resulting in 2 additional duplicates removed, 1 article discarded for irrelevant content, 7 articles discarded for being descriptive/letters or editorials, 3 articles discarded for being review articles, and 44 articles discarded as they were abstract-only publications. The 73 remaining articles were obtained in full text and reviewed in detail. Two articles were excluded for not achieving the minimum quality assessment requirements. Nine articles were excluded as they did not evaluate the original SAS, while 26 articles were excluded as they did not measure SAS performance as indicated by a published AUROC or c-statistic. Reference lists from all 73 articles that were reviewed in full were hand searched with no new articles identified. The 36 remaining articles are included in this review.^{1,3,4,13–45} The authors who first derived and validated the SAS were successfully contacted and confirmed that no additional unpublished studies or data was available. Figure 1 summarizes the review process.

FIGURE 1.

Flow diagram of the review process.

All of the 36 articles included in this review were observational in design with 19 being retrospective cohort analyses and 13 being prospective cohort analyses. Two studies were mixed retrospective and prospective cohort analysis and another 2 articles were retrospective case series.

Quality Assessment

The quality assessment results are included in Supplemental Table 1 (http://links.lww.com/AOSO/A194). Study quality was assessed using the Newcastle-Ottawa scale as all included studies were observational.¹⁰ Two studies were excluded due to inadequate methodological quality, and the remaining 36 articles included in this review were determined to have low risk of bias.

Outcomes Reporting

Outcomes are summarized in Supplemental Table 1 (http://links.lww.com/AOSO/A194). Study outcomes were primarily 30-day morbidity and mortality through 2 studies reported intensive care unit (ICU) admission rates, 1 reported perioperative myocardial infarction, and 2 studies evaluated in-hospital morbidity and mortality. The majority of studies used the National Surgical Quality Improvement Program morbidity definition as used by Gawande et al¹ or the Clavien-Dindo classification system for postsurgical complications.^46,47 Morbidity definitions are summarized in Appendix 4 (http://links.lww.com/AOSO/A193). Twenty-five studies reported mortality rates, which ranged from 0% to 29.5%.

Discrimination

All 36 studies reported metrics of discrimination. When using the SAS to correctly identify any postoperative morbidity, the AUROC or c-statistic ranged from 0.561 in a thoracic surgery (esophagectomy) population to 0.872 in an orthopedic spine (lumbar fusion) surgery population.^31,33 When using the SAS to identify mortality the AUROC or c-statistic ranged from 0.629 in an orthopedic surgery (hip fracture) population to 0.92 in a general/vascular surgery population.^1,24 When considering the ability of the SAS to identify postoperative ICU admission, 2 studies were identified with AUROC or c-statistics of 0.69 and 0.76.^20,22 Supplemental Table 2 (http://links.lww.com/AOSO/A195) lists all extracted measures of discrimination. Results are summarized in Figure 2.

FIGURE 2.

Bar graph of study results. HPB, Hepatobiliary; HEENT, head, eyes, ears, nose and throat.

Results by Surgical Discipline

General and Vascular

In patients undergoing general and vascular surgeries, 14 articles reported measures of discrimination for postoperative morbidity or morbidity and mortality with metrics of discrimination ranging from 0.63 to 0.79.^30,32 Two studies reported discrimination for postoperative ICU admission with AUROC/c-statistics ranging from 0.69 for delayed ICU admission to 0.76 for immediate ICU admission.^20,22 In these same general and vascular surgery populations, 5 studies reported measures of discrimination for postoperative mortality with metrics ranging from of 0.74 to 0.92.^1,41 Notably, the SAS had higher discriminatory ability in younger patient populations undergoing emergent surgeries.^19,32

Orthopedic and Spine Surgery

In patients undergoing orthopedic surgeries, 6 studies measured discrimination for morbidity, with statistics ranging from 0.59 to 0.87.^18,33 Two studies reported discrimination for mortality, with AUROC/c-statistics, reported as 0.63 and 0.76.^24,38 While measures of discrimination for morbidity and mortality in the general orthopedic surgery population were some of the lowest in this review, the subset of orthopedic spine surgery found good performance with morbidity discrimination ranging from 0.75 to 0.87.^25,33

Thoracic Surgery

Two studies measured discrimination for morbidity in the thoracic surgery population. In both reports of those undergoing esophagectomy, they found the SAS to have modest discriminatory metrics of 0.56 and 0.68.^26,31

Hepatobiliary Surgery

Two studies measured discriminatory ability for postoperative morbidity in the hepatobiliary population. In 1 study of patients undergoing hepatectomy for hepatocellular carcinoma, the AUROC/c-statistic was found to be 0.72, while in a group of patients with pancreatic cancer undergoing either a distal pancreatectomy or pancreaticoduodenectomy, the metric was found to be 0.60.^42,44

Urology

In the urological surgery population, 1 study reported discrimination for mortality with an AUROC/c-statistic of 0.75.¹⁶

Neurosurgery

In a group of patients undergoing craniotomy for resection of meningioma, Hsu et al³⁴ found the SAS to predict 30-day morbidity with an AUROC/c-statistic of 0.768.

Gynecologic Surgery

In patients undergoing nonlaparoscopic gynecologic surgery, 1 study reported an AUROC/c-statistic of 0.80 for the SAS in predicting postoperative morbidity and mortality.³⁵

Head and Neck Surgery

Two studies measured discrimination for 30-day morbidity in patients undergoing surgery for head and neck cancer. AUROC/c-statistics were reported as 0.70 and 0.73.^37,43

Mixed Surgical Disciplines

Four studies evaluated the discriminatory ability of the SAS across mixed surgical disciplines. Three studies reported metrics for postoperative morbidity ranging from 0.62 to 0.73.^14,17,39 One study reported on postoperative mortality and found the SAS to be moderately predictive of mortality with a statistic of 0.77.¹⁴ One study evaluated the SAS as a predictor of postoperative troponin leak or myocardial infarction in a mixed surgical population and found it to be moderately predictive with an AUROC/c-statistics of 0.80 and 0.81.²⁹

DISCUSSION

The SAS is a 10-point score derived from the lowest intraoperative heart rate, lowest MAP, and EBL.¹ This metric is unique in that it relies on intraoperative rather than preoperative data to predict postoperative risk of morbidity and mortality. Since first derived and validated by Gawande et al,¹ the SAS has been thoroughly evaluated in various settings. Early on, Regenbogen et al² investigated whether the SAS measured intraoperative performance or if it just reflected preoperative risk. They found that the SAS remained strongly correlated with postoperative outcomes even after accounting for a patient’s acute condition, comorbidities, and operative complexity. They concluded that the SAS appeared to detect differences in intraoperative management. More recently, the SAS has demonstrated consistent discriminatory ability across multiple disciplines. Higher levels of discrimination for both morbidity and mortality have been found in the general, vascular, orthopedic spine, orthopedic trauma, and gynecologic surgery populations as well as in younger patient groups.^{1,3,19,20,22,24,25,32,33,38,40} Lower levels of discrimination for both morbidity and mortality have been noted in general orthopedic surgery populations.^13,18,27 Studies looking at other endpoints such as postoperative ICU admission or the presence of cardiac injury also found that the SAS had a high level of discriminatory ability.^20,22,29

While the SAS has shown to provide moderate discriminatory ability for postoperative morbidity and moderate to high discrimination for postoperative mortality, the score has not performed as well in some general orthopedic surgery populations.^13,18,27 Contrary to this, the SAS performed well in the subset of orthopedic patients undergoing spine surgeries and traumatic hip fracture repair.^24,25,33,38 A possible explanation for this may be that general orthopedic surgeries are often elective, with medically optimized patients and ideal operating conditions where blood loss, hemodynamic compromise, and mortality are infrequent. Alternatively, elective orthopedic procedures are often facilitated with regional anesthesia, which again may influence hemodynamics and blood loss. Of course, an ideal score would provide perfect discrimination across a multitude of scenarios. Given that a predictive model is derived from a specific population, it is to be expected to perform better in circumstances that most closely resemble that initial derivation. The utility of the SAS is illustrated in its ease of use and the fact that it maintains a moderate level of discrimination for postoperative morbidity and a moderate to high level of discrimination for postoperative mortality across a wide range of surgical conditions. Looking at other commonly used predictive models, the performance of the SAS is strikingly similar. For example, the CHADS₂ and CHA₂DS₂-VASc scores used for the prediction of cardioembolic stroke have c-statistics ranging from 0.60–0.80 to 0.64–0.79, respectively.⁴⁸ While it was beyond the scope of this review, a number of investigators have made adjustments to the SAS or combined it with specific preoperative risk factors to better predict postoperative morbidity and mortality.^38,40,43,44 This adaptability again illustrates the strength of this metric. The true and unique value of the SAS is that it clearly shows that intraoperative performance has an impact on postoperative risk and that combining this data with risk stratification tools currently in place may lead to better prediction of postoperative outcome. Of course, the hope is that a better predictive model will influence postoperative care where these complications can be mitigated.

Our study has a number of limitations. First of all, we kept our inclusion criteria broad so as to prevent any limitation on the scope of included data. Exclusion criteria were derived to ensure that studies met prespecified quality requirements and that only new data was included, which directly assessed the SAS. Excluding studies that did not report measures of discrimination improved data analysis at the expense of excluding data on score calibration. Notably, all studies that we found were observational in design. While study quality was assessed and a minimal risk of bias was required for inclusion, observational studies have intrinsically higher rates of bias that is difficult control. Secondarily, the included studies had a high degree of heterogeneity with regards to reported outcomes and statistical analysis. This made it difficult to compare results between studies and impossible to consolidate results into a meta-analysis. In addition, although we performed a gray literature search and contacted the original authors for unpublished data, the majority of studies included were found in the primary database search, and as such, our study remains at risk of publication bias. Finally, despite an exhaustive literature search, a small number of appropriate articles may have been overlooked.

CONCLUSIONS

Risk prediction is a key component of providing safe and informed patient care, and implementation of risk prediction models may improve outcomes by enabling early preventative measures. The SAS is a unique risk prediction metric in that it is simple to calculate and relies only on intraoperative parameters. It has shown to provide moderate discriminatory ability for postoperative morbidity and a moderate to high level of discrimination for postoperative mortality across multiple surgical disciplines. This discriminatory ability may provide significant value to clinicians when determining appropriate postoperative management. While this retrospective data is reassuring, the SAS may not be applicable to all surgical procedures. Further studies are required to determine whether the use of the SAS can influence postoperative management, and if this influence can lead to a positive impact on patient outcomes.

ACKNOWLEDGMENTS

All authors were involved in study design, data extraction, article review, and final article approval. E.S.P. performed the literature search.

Funding Statement

Disclosure: The authors declare that they have nothing to disclose.