Can the Surgical Apgar Score predict morbidity and mortality in general orthopaedic surgery?

Abstract

Purpose

The Surgical Apgar Score (SAS) is a simple tally based on intra-operative heart rate, blood pressure and blood loss; it predicts 30-day major postoperative complications and mortality in different surgical fields, but no validation has been performed in general orthopaedic surgery.

Methods

A prospective assessment of the SAS in 723 consecutive patients undergoing major and intermediate orthopaedic procedures was performed in an 18-month period. The SAS was calculated immediately after surgery, and the occurrence of major complications or death was registered within a 30-day follow-up.

Results

Thirty-seven patients had ≥1 complication (5.12 %). The complication rate did not augment as the score decreased (SAS 9–10 = 6.56 %; SAS 7–8 = 2.62 %; SAS 5–6 = 7.21 %; SAS ≤4 = 10.2 %), the relative risk did not augment as the score decreased and the likelihood ratio did not increase with decreasing SAS values, except in the subgroup of patients undergoing spine surgery. The C-statistic was 0.59 (95 % confidence interval 0.48–0.69), a weak discriminatory value. Using a threshold of 7 to define high-risk and low-risk patients, the SAS allowed risk stratification only for spine surgery.

Conclusions

The SAS does not predict 30-day major complications and death in patients undergoing general orthopaedic surgery, but it is useful in the subgroup of patients undergoing spine surgery.

Introduction

Orthopaedic surgery has experienced significant development in recent decades, and better understanding of perioperative risk is important in the context of an ageing population requiring surgical treatment for musculoskeletal problems and more complex procedures. Although well-established postoperative scoring systems to predict outcomes have been developed [1], their complexity hampers their employment in the day-to-day clinical scenario and, most of the time, postoperative risk evaluation depends on subjective clinical evaluation. An inexpensive, objective and easy to use surgical outcome score that could identify patients at high risk for major complications and death would improve patient safety, aid in research and could become a public health tool for quality improvement programmes.

Recently, a simple surgical score inspired by the Apgar Score of Obstetrics was developed to predict the occurrence of 30-day major postoperative complications and mortality [2]. Originally developed in patients undergoing general and vascular surgery, the Surgical Apgar Score (SAS) was later validated in different surgical specialties, including colorectal, urological and gynaecological surgery [3–5], as well as in different international scenarios [6–9]. This 10-point score, which allows risk stratification in the postoperative setting, is based on three variables: the estimated blood loss (EBL), lowest heart rate (HR) and lowest mean arterial pressure (MAP) during surgery (Table 1). This score has been demonstrated to be a good predictor of the occurrence of major complications or death within 30 days of surgery; a high score is associated with a low risk of postoperative major complications or death, while a low score is associated with an increased risk [2, 8, 10].

Table 1.

The 10-point SAS

	Number of points
	0	1	2	3	4
Estimated blood loss (ml)	>1,000	601–1,000	101–600	≤100	–
Lowest mean arterial pressure (mmHg)	<40	40–54	55–69	≤70	–
Lowest heart rate (bpm)	>85	76–85	66–75	56–65	≤55

Only two studies have evaluated the utility of this score in the orthopaedic field, and only for specific operations: a retrospective study evaluating patients undergoing hip and knee arthroplasties showed that even though the score obtained relevant intra-operative information regarding risk of complications, on its own it did not provide a comprehensive postoperative risk stratification [11]. Another study aiming to evaluate the utility of the SAS in a district general hospital in the UK found that the score did not demonstrate statistical significance for the prediction of major complication or death in patients undergoing lower limb arthroplasty or femoral neck fracture treatment [9].

Lower limb joint replacements and femoral neck fractures, however, represent only a limited subgroup of orthopaedic operations. Our study is a prospective evaluation aiming to assess the utility of the SAS in predicting 30-day major complications and mortality in a patient cohort undergoing a wide mix of orthopaedic surgical procedures.

Patients and methods

Institutional Review Board approval was obtained to perform this study.

A prospective, consecutive series of all patients undergoing major and intermediate orthopaedic surgeries in a university hospital was recruited between 1 January 2011 and 30 June 2012. The inclusion criteria were as follows: inpatients undergoing major and intermediate orthopaedic procedures, including open spine surgery (discectomies, laminectomies and spinal fusion procedures in the cervical, thoracic and lumbosacral spine through either anterior or posterior approaches, independently of the number of levels operated); knee and hip osteotomies and arthroplasties, including partial or total arthroplasties, including primary and revision cases; major lower extremity fracture with open surgical treatment (femoral neck, femoral diaphysis, tibial plateau and tibial fractures); and tumour resection and reconstructions from the pelvis, humerus, femur or tibia. Exclusion criteria were as follows: patients under age 18, minimally invasive procedures, patients that did not have their SAS registered in the operative chart when they left the operating room and patients lost to follow-up.

The data were collected at the time of operation from handwritten anaesthesia records and registered in the operative report at the end of each surgical procedure. Intra-operative parameters were EBL, lowest HR and lowest MAP, and the SAS was calculated immediately after surgery and registered in the operative report. In our institution, the operative records must be completed in a FileMaker (FileMaker Inc., Santa Clara, CA, USA) database before the patients leave the operating room, and it is required that a print copy of the operative record is archived in the patient’s physical chart. The strategy used to avoid data loss included a modification of the FileMaker database such that the operative report for patients undergoing surgery by any member of the Department of Orthopaedic Surgery could not be printed unless EBL, HR, MAP and SAS were registered. That system allowed us to register data at the end of surgery for 767 patients from a total of 794 patients that would have met the criteria during the study period (96.6 %).

The outcomes measured were the occurrence of major complications or death within 30 days of surgery. Major complications, as defined in the original paper of Gawande et al. [2], included the following: acute renal failure, bleeding requiring ≥4 units of red cell transfusion within 72 h after operation, cardiac arrest requiring cardiopulmonary resuscitation, coma for ≥24 h, deep venous thrombosis, myocardial infarction, unplanned intubation, ventilator use ≥48 h, pneumonia, pulmonary embolism, stroke, major wound disruption, surgical site infection, sepsis, septic shock, systemic inflammatory response syndrome, unplanned return to the operating room and death. Superficial surgical site infections and urinary tract infections were not considered major complications. Outcome data were prospectively collected from diagnoses in discharge summaries and ambulatory electronic charts. If no complications were registered, then patients were called 30 days after surgery to determine whether complications had occurred.

Statistical analyses were performed using the SPSS program. To determine the discriminatory accuracy of the SAS, we generated a C-statistic by calculating the area under the receiver-operating characteristic (ROC) curve; the results were expressed with a 95 % confidence interval (CI). A univariate logistic regression analysis (considering the score as an ordinal variable) was also performed to determine the relationship between major complications/death and the SAS; the results were expressed as an odds ratio (OR) with 95 % CI. In addition, we aggregated patients with SAS ≤4 into one subgroup because we had a small number of cases with low scores, and we created three additional subgroups (5–6, 7–8 and 9–10) for analyses, including calculating relative risks (RR) and likelihood ratios (LR) for each SAS subgroup. LRs were used to evaluate the performance of the SAS to detect major complications/death, because LR results are independent of the prevalence of the outcome (complications/death). Finally, the SAS was rationalised into two groups representing low-risk (SAS ≥7) and high-risk patients (SAS <7), using a threshold that had been previously established [9]. Fischer’s exact test was used to analyse differences between these two groups; in addition, RR and LR were calculated.

Results

From the 767 patients with SAS completed at the end of surgery, we were able to obtain complete data 30 days after surgery in 723 cases (5.74 % lost to follow-up). This final cohort included 334 patients undergoing hip surgery, 268 spine surgery cases, 92 knee surgery cases, 24 patients treated for long bone fractures and five patients undergoing major tumour resection procedures. The overall incidence of major complications (including five deaths) was 5.12 % (37/723 patients). Among patients with SAS 9–10, 8/122 (6.56 %) had major complications, with two deaths; in the group of patients with SAS 7–8, 9/344 (2.62 %) experienced major complications, including two deaths; for patients with SAS 5–6, 15/208 (7.21 %) had major complications with no deaths, and for those with SAS ≤4, 5/49 (10.2 %) had complications (one death). The distributions of major complications and death between the SAS categories are shown in Fig. 1.

Fig. 1.

Distribution of 30-day major complications and death between the SAS subgroups

This same evaluation was performed for the cohorts of patients undergoing hip surgery, spine surgery, knee surgery, long bone fracture treatment and major tumour resection procedures; this evaluation revealed that complications increased as the SAS decreased only in the spine surgery cohort (Fig. 2).

Fig. 2.

Distribution of 30-day major complications and death between the SAS subgroups for patients undergoing hip surgery, spine surgery, knee surgery and tumour resection plus long bone fracture treatment

The C-statistic for predicting major complications and deaths was 0.59 (95 % CI 0.48–0.69), a weak discriminatory power (Fig. 3).

Fig. 3.

ROC curve for the SAS as predictor of major complications and death. The area under the curve corresponds to the C-statistic, which was 0.59 Diagonal segments are produced by ties

The OR from the univariate logistic regression analysis was 0.83 (95 % CI 0.70–0.99). Using the stratification by decreasing scores, and employing the subgroup of patients with scores 9–10 as a reference group, the RR did not augment as the score decreased, as shown in Table 2. LR did not increase with decreasing SAS scores; in addition, 95 % CIs showed an overlap among the different subgroups, as shown in Table 2.

Table 2.

Rate of major complications and death, RR and LR for the different SAS categories

Score	0–4	5–6	7–8	9–10
Number of patients	49	208	344	122
Major complications & deaths, n (%)	5 (10.20)	15 (7.21)	9 (2.62)	8 (6.56)
RR (95 % CI)	1.56 (0.54–4.52)	1.1 (0.48–2.52)	0.399 (0.16–1.01)	1 (reference)
LR (95 % CI)	2.11 (0.88–5.27)	1.44 (0.89–3.19)	0.5 (0.17–0.74)	1.3 (0.64–2.90)

Of the 466 patients with SAS ≥7, 17 presented major complications or death, while of the 257 patients exhibiting SAS <7, 20 presented these complications (p = 0.02, Fig. 1). However, this difference was determined only by spine surgery cases; using a threshold of 7 to define high risk and low risk, only the cohort of spine surgery patients exhibited differences among high-risk and low-risk groups (Table 3).

Table 3.

Differences among high-risk and low-risk groups (defined by a threshold of 7 points in SAS)

	Without complications		With complications		p
	SAS <7	SAS ≥7	SAS <7	SAS ≥7
General	237	449	20	17	0.021
Hip	127	194	4	9	0.579
Spine	84	166	14	4	0.000
Long bone fracture	8	14	0	2	0.536
Knee	17	74	0	1	1.000
Tumour	1	1	2	1	1.000

Discussion

The SAS, which was developed in general and vascular surgery, and has been validated in colorectal, urological and gynaecological surgery, is a weak predictor of major complications and death in general orthopaedic surgery. However, it is a useful predictor of major complications and death in the subgroup of patients undergoing spine surgery.

Previously, only two publications had assessed this score in the orthopaedic field. A retrospective study evaluating patients undergoing hip and knee arthroplasties showed that although the score contained relevant intra-operative information regarding the risk of complications, on its own it did not provide a comprehensive postoperative risk stratification [11]. Nevertheless, that study could only determine the in-hospital complications, which do not account for all the cases that presented major complications and mortality 30 days after surgery; in addition, as a retrospective study, the accuracy of the data acquired at the end of surgery could be imperfect. The importance of prospectively obtaining morbidity and mortality data in a thorough way should not be underestimated, since it has been described that administrative databases fail to adequately gather information on postoperative complications [12]. Another study aiming to evaluate the utility of the SAS in a district general hospital in the UK found that the score did not demonstrate statistical significance for the prediction of major complications or death in patients undergoing lower limb arthroplasty or femoral neck fracture treatment, despite its usefulness in general and vascular surgery [9]; however, the results from that study could be attributed to a type II error due to a relatively small population of patients undergoing orthopaedic procedures. To overcome these potential limitations, our study prospectively recruited a cohort of 712 patients undergoing a wide variety of orthopaedic procedures, with data collected at the end of surgery and 30 days follow-up in all them. This mix of orthopaedic procedures including complex and more simple surgeries avoids an over-representation of “high-risk patients” (those with SAS <7). Thus, we could obtain a representative sample of the proportion of complex cases in the orthopaedic field, allowing an assessment of the SAS not only for the most complex settings, but also in usual orthopaedic practice. However, we decided to exclude patients undergoing outpatient surgeries and minor surgeries, which should be associated with a very low risk of major postoperative complications or death.

The SAS did not predict major complications or death in the general orthopaedic surgery cohort. Interestingly, the score was demonstrated to be a good tool to predict major complications or death in the subgroup of patients undergoing spine surgery. It could be hypothesised that the limited utility of the SAS in this cohort of patients undergoing orthopaedic surgery could be secondary to a smaller complication rate compared to general and vascular surgery, since it is not unusual to have a significant number of patients with a risk of death greater than 70 % in general surgery [13]. Our cohort exhibited a lower complication rate compared with the general and vascular surgery validation cohort (5.2 versus 22 %); a low in-hospital complication rate was also reported in a retrospective study assessing the SAS in patients undergoing hip and knee arthroplasties [11]. Nonetheless, that study may have underestimated their complications, since relatively high readmission rates have been observed within 30 days after joint arthroplasties [14]. To overcome this potential limitation, we used the LR, which allows an assessment of the performance of the score independent of the prevalence of morbidity and mortality, also showing that the score could not stratify postoperative risk. Moreover, as shown by Thorn et al. [9], even with a similar complication rate among patients undergoing general and vascular surgery or orthopaedic procedures (24 versus 20 %), the SAS was not able to predict 30-day major complications or death in orthopaedic surgery.

The SAS not only did not allow risk stratification in patients undergoing limb surgery; noteworthy, an important proportion of patients presenting complications or dying had a score of 7 or more (Table 2); hence, clinicians should not underestimate the potential for complications to arise in patients undergoing limb surgery despite a high SAS value. On the other hand, the monotonic relationship between SAS and the risk of major complications or death in spine surgery suggests that patients undergoing spine surgery may benefit from a postoperative triage based on this score (Fig. 2). The different predictive values of the SAS for spine and limb surgery suggest that complications after spine surgery are more related to the cardiovascular effects that a low SAS reflects; conversely, complications after limb surgery may be more related to patients’ previous characteristics, as they can occur in spite of a high score. Interestingly, it has already been reported that predicting the outcome of orthopaedic procedures in injured patients just based on injury severity scores only can be difficult, since host factors appear to be of greater importance [15].

With a growing population requiring orthopaedic procedures worldwide, the need for an easy to apply orthopaedic postoperative risk stratification score is a well-recognised requirement. The SAS has a weak predictive value for 30-day morbidity and mortality in general orthopaedic surgery, but it is a useful tool for this purpose in the subset of patients undergoing spine surgery. Future studies should evaluate potential modifications to this score that could improve its prognostic performance in general orthopaedic surgery.